Image processing method and related equipment

Abstract

The invention provides an image processing method and related equipment. The method comprises the steps that first basic layer data, first enhancement layer data and first metadata are acquired, the first basic layer data are basic layer data corresponding to a first HDR image, and the first enhancement layer data are enhancement layer data corresponding to the first HDR image; converting target data according to the first metadata to obtain target conversion data, wherein the target data is the first basic layer data and / or the first enhancement layer data; and determining a second HDR image according to the target conversion data. According to the technical scheme, a more efficient and more reasonable coding scheme can be selected for the base layer data and / or the enhancement layer data, and then the base layer data and / or the enhancement layer data are converted during decoding, so that the reduction degree of the finally synthesized HDR image is higher, and the HDR image is more adaptive to a display device.

Description

[0001] This application claims the priority of the Chinese patent application filed with the State Intellectual Property Office of China on August 15, 2023, with application number 202311037386.9, and the priority of the Chinese patent application with the invention name “Encoding method and related equipment”, all contents of which are incorporated by reference in this application. Technical Field

[0002] The present application relates to the field of coding and decoding, and more specifically to an image processing method and related equipment. Background Art

[0003] Dynamic range is used in many fields to express the ratio of the maximum value to the minimum value of a variable. In digital images, the dynamic range represents the ratio between the maximum grayscale value and the minimum grayscale value within the displayable range of the image. The dynamic range in nature is quite large. The brightness of a night scene under the starry sky is about 0.001cd / m2, and the brightness of the sun itself is as high as 1,000,000,000cd / m2. Such a dynamic range reaches 1,000,000,000 / 0.001=1013. However, in real scenes in nature, the brightness of the sun and the brightness of the stars are not obtained at the same time. For natural scenes in the real world, the dynamic range is between 10 -3 to 10 6 In most color digital images, each channel of red (R), green (G), and blue (B) is stored in an 8-bit byte. That is, the range of each channel is 0 to 255 gray levels. Here, 0 to 255 is the dynamic range of the image. In the real world, the dynamic range of the same scene is between 10 -3 to 10 6 The dynamic range of an image or video is called a low dynamic range (LDR).

[0004] HDR images or HDR videos may need to be transcoded during the distribution or transmission process. The current HDR transcoding solution is to directly transcode HDR images or HDR videos. After decoding, the HDR image or HDR video obtains HDR pixel values ​​and HDR format identifiers and information. HDR pixel values ​​usually need to be processed (such as image up and down sampling, enhancement, etc.). The HDR format identifier and information, as well as the processed HDR pixel values ​​are then sent to the encoder for re-encoding. The re-encoded HDR image or HDR video is distributed to other devices. This solution of directly transcoding HDR images or HDR videos (which can be called a "single-layer transcoding solution") may cause abnormalities in the transcoded HDR images or HDR videos, affecting the user experience.

[0005] Therefore, how to provide a more efficient encoding / decoding solution is a problem to be solved urgently in the art. Summary of the invention

[0006] The present application provides an image processing method and related equipment, which are conducive to more efficient and reasonable encoding.

[0007] In a first aspect, an embodiment of the present application provides an image processing method, comprising: acquiring first basic layer data, first enhancement layer data and first metadata, wherein the first basic layer data is the basic layer data corresponding to a first HDR image, and the first enhancement layer data is the enhancement layer data corresponding to the first HDR image; converting target data according to the first metadata to obtain target conversion data, the target data being the first basic layer data and / or the first enhancement layer data; and determining a second HDR image according to the target conversion data.

[0008] Through the above technical solution, when processing the double-layer image of the HDR image, the base layer data and / or the enhancement layer data can be first converted according to the metadata, so that the final synthesized HDR image has a higher degree of restoration and is more compatible with the display device.

[0009] In combination with the first aspect, in a possible implementation manner of the first aspect, converting the target data according to the first metadata includes: determining to convert the target data according to the first metadata, and then converting the target data to obtain target converted data.

[0010] Through the above technical solution, when processing the double-layer image of the HDR image, the base layer data and / or the enhancement layer data can be first converted according to the metadata, and then the base layer data and / or the enhancement layer data can be converted according to the determination result, so that the final synthesized HDR image has a higher degree of restoration and is more compatible with the display device.

[0011] In combination with the first aspect, in a possible implementation of the first aspect, the first metadata includes conversion position information, and the conversion position information is used to indicate the location where the conversion occurs. The converting of the target data according to the first metadata includes: converting the target data according to the conversion position information.

[0012] By indicating the conversion location through metadata, the flexibility of target data processing can be improved.

[0013] In combination with the first aspect, in a possible implementation of the first aspect, the first metadata includes first conversion judgment information, and the first indication information is used to indicate that the target data needs to be converted; determining to convert the target data based on the first metadata includes: converting the target data according to the first conversion judgment information.

[0014] In combination with the first aspect, in a possible implementation manner of the first aspect, the conversion includes one or more of the following: color space conversion, color gamut conversion, or color compensation.

[0015] In combination with the first aspect, in a possible implementation of the first aspect, when the target data is the first basic layer data and the conversion includes the color space conversion, the target data is converted to obtain target conversion data, including: performing the color space conversion on the first basic layer data to obtain the target conversion data, and the color space of the target conversion data is an RGB color space.

[0016] In combination with the first aspect, in a possible implementation of the first aspect, when the target data is the first basic layer data and the conversion includes the color gamut conversion, the target data is converted to obtain target conversion data, including: performing the color gamut conversion on the first basic layer data through T() to obtain the target conversion data, where T() is a default matrix or a conversion matrix indicated by the first metadata.

[0017] In combination with the first aspect, in a possible implementation manner of the first aspect, when the target data includes the first enhancement layer data, the conversion further includes a channel conversion, and the channel conversion is used to restore the channel of the first enhancement layer data to the original channel.

[0018] In combination with the first aspect, in a possible implementation manner of the first aspect, the first metadata also includes conversion parameters used when performing the conversion.

[0019] In combination with the first aspect, in a possible implementation manner of the first aspect, when the conversion includes the color compensation, the conversion parameters include: a color compensation matrix; a conversion function; or a compensation value and a compensation method.

[0020] In combination with the first aspect, in a possible implementation manner of the first aspect, the method further includes: performing color gamut conversion and / or color compensation on the second HDR image.

[0021] In a second aspect, an embodiment of the present application provides an image processing method, the method comprising: determining, based on an acquired first HDR image, first basic layer data, first enhancement layer data and first metadata corresponding to the first HDR image, wherein the first metadata is used to convert target data, the target object including the first basic layer data and / or the first enhancement layer data; encoding the target data to obtain a code stream.

[0022] The above technical solution can improve the coding efficiency by transmitting the information for determining the conversion of the base layer data and / or the enhancement layer data through metadata when processing the dual-layer image of the HDR image. The decoding side can determine the conversion of the base layer data and / or the enhancement layer data according to the metadata, and convert the base layer data and / or the enhancement layer data according to the determination result, so that the final synthesized HDR image has a higher degree of restoration and is more compatible with the display device.

[0023] In combination with the second aspect, in a possible implementation manner of the second aspect, the first metadata includes conversion position information, where the conversion position information is used to indicate a position where the conversion occurs.

[0024] In combination with the second aspect, in a possible implementation manner of the second aspect, the first metadata includes first conversion judgment information, and the first indication information is used to indicate that the first basic layer data and / or the first enhancement layer data needs to be converted.

[0025] In combination with the second aspect, in a possible implementation manner of the second aspect, the conversion includes one or more of the following: color space conversion, color gamut conversion, or color compensation.

[0026] In combination with the second aspect, in a possible implementation manner of the second aspect, the first metadata also includes conversion parameters used when performing the conversion.

[0027] In combination with the second aspect, in a possible implementation manner of the second aspect, when the conversion includes the color gamut conversion, the conversion parameter includes a conversion matrix.

[0028] In combination with the second aspect, in a possible implementation manner of the second aspect, when the conversion includes the color compensation, the conversion parameters include: a color compensation matrix; a conversion function; or a compensation value and a compensation method.

[0029] In a third aspect, an embodiment of the present application provides an electronic device, which includes a unit for implementing the first aspect or any possible implementation method of the first aspect.

[0030] In a fourth aspect, an embodiment of the present application provides an electronic device, which includes a unit for implementing the second aspect or any possible implementation method of the second aspect.

[0031] In a fifth aspect, an embodiment of the present application provides a computer device, which includes a processor, wherein the processor is used to couple with a memory, read and execute instructions and / or program codes in the memory to execute the first aspect or any possible implementation of the first aspect.

[0032] In a sixth aspect, an embodiment of the present application provides a computer device, which includes a processor, wherein the processor is used to couple with a memory, read and execute instructions and / or program codes in the memory to execute the second aspect or any possible implementation of the second aspect.

[0033] In the seventh aspect, an embodiment of the present application provides a chip system, which includes a logic circuit, which is used to couple with an input / output interface and transmit data through the input / output interface to execute the first aspect or any possible implementation method of the first aspect.

[0034] In an eighth aspect, an embodiment of the present application provides a chip system, which includes a logic circuit, which is used to couple with an input / output interface and transmit data through the input / output interface to execute the second aspect or any possible implementation method of the second aspect.

[0035] In the ninth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a program code. When the program code stored in the computer storage medium is run on a computer, the computer executes the first aspect or any possible implementation of the first aspect.

[0036] In the tenth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a program code. When the program code stored in the computer storage medium is run on a computer, the computer executes the second aspect or any possible implementation of the second aspect.

[0037] In the eleventh aspect, an embodiment of the present application provides a computer program product, which includes: a computer program code, when the computer program code runs on a computer, enables the computer to execute the second aspect or any possible implementation of the second aspect.

[0038] In a twelfth aspect, an embodiment of the present application provides a computer program product, which includes: a computer program code, when the computer program code runs on a computer, enables the computer to execute the second aspect or any possible implementation of the second aspect.

[0039] In a thirteenth aspect, an embodiment of the present application provides a code stream, which is determined according to the second aspect or any possible implementation method of the second aspect.

[0040] In a fourteenth aspect, an embodiment of the present application provides a code stream, which includes first base layer data, first enhancement layer data, and first metadata, wherein the first metadata is used to convert the first base layer data and / or the first enhancement layer data.

[0041] In a fifteenth aspect, an embodiment of the present application provides a device for storing a code stream, wherein the device is used to store the code stream of the thirteenth aspect or the fourteenth aspect.

[0042] In the sixteenth aspect, an embodiment of the present application provides a device for transmitting a code stream, the device comprising: a transmitter and a receiver, the receiver being used to receive the code stream generated by the method described in the second aspect above, and the transmitter being used to send the code stream to a client device via a transmission medium.

[0043] In the seventeenth aspect, an embodiment of the present application provides a device for transmitting a code stream, the device comprising: a transmitter and at least one storage medium, the at least one storage medium being used to store the code stream generated according to the method described in the second aspect above, the transmitter being used to obtain the code stream from the storage medium and send the code stream to a client device via a transmission medium.

[0044] In an eighteenth aspect, an embodiment of the present application provides a system for distributing a code stream, the system comprising: at least one storage medium for storing a code stream generated according to the method described in the second aspect above, a streaming media server for obtaining a target code stream from the at least one storage medium and sending the target code stream to a client device, wherein the streaming media server comprises a content server or a content distribution server. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is a schematic diagram of dynamic range mapping.

[0046] Figure 2 This is the image of the PQ photoelectric transfer function.

[0047] Figure 3 This is the image of the HLG photoelectric transfer function.

[0048] Figure 4 This is the image of the SLF photoelectric transfer function.

[0049] Figure 5 A schematic block diagram of a coding solution system used in an embodiment of the present application.

[0050] Figure 6 A block diagram of a content providing system for implementing content distribution services used in an embodiment of the present application.

[0051] Figure 7 A schematic diagram of a streaming media system workflow applicable to an embodiment of the present application.

[0052] Figure 8 A schematic diagram describing an exemplary end-to-end process of the present application is provided.

[0053] Fig. 9 An exemplary streaming media system architecture of the present application is described.

[0054] Fig.10 A schematic diagram of a possible system architecture applicable to an embodiment of the present application.

[0055] Fig.11 It is a flowchart of an image processing provided according to an embodiment of the present application.

[0056] Fig.12 It is a schematic flowchart of an image processing method provided according to an embodiment of the present application.

[0057] Fig.13 It is a schematic flowchart of an image processing method provided according to an embodiment of the present application.

[0058] Fig.14 It is a schematic flowchart of an image processing method provided according to an embodiment of the present application.

[0059] Fig.15 It is a schematic structural block diagram of an electronic device provided according to an embodiment of the present application.

[0060] Fig.16 It is a schematic structural block diagram of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0061] The technical solution in this application will be described below in conjunction with the accompanying drawings.

[0062] In the present application, "at least one" means one or more, and "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: including the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0063] First of all, in order to facilitate understanding of the embodiments of the present application, some concepts or terms involved in the embodiments of the present application are explained.

[0064] Color value: A numerical value corresponding to a specific image primary color component (such as R, G, B, or Y).

[0065] Digital code value: A digital expression of an image signal. Digital code value is used to represent nonlinear primary color values.

[0066] Linear color value: A linear color value that is proportional to the light intensity. In an optional case, its value should be normalized to [0,1], referred to as E.

[0067] Nonlinear primary color value: Nonlinear primary color value is the normalized digital expression value of image information, which is proportional to the digital coding value. In an optional case, its value should be normalized to [0,1], referred to as E′.

[0068] Electro-optical transfer function (EOTF): A conversion relationship from nonlinear primary color values ​​to linear primary color values.

[0069] Metadata: Data carried in a video signal that describes the video source information.

[0070] Dynamic metadata: Metadata associated with each frame of an image that changes from frame to frame.

[0071] Static metadata: Metadata associated with an image sequence that remains unchanged within the image sequence.

[0072] Luma: represents the combination of nonlinear primary color signals, symbolized by Y'.

[0073] Luminance mapping: The luminance of the source image is mapped to the luminance of the target system.

[0074] Display adaptation: Processing of video signals to adapt the display characteristics of the target display.

[0075] Source picture: The image input in the HDR pre-processing stage.

[0076] Mastering display: A reference display used when editing and producing video signals to determine the effects of video editing and production.

[0077] Linear scene light signal: In HDR video technology, the HDR video signal with content as scene light refers to the scene light captured by the camera / camera sensor, which is generally a relative value; the linear scene light signal is encoded by hybrid log-gamma (HLG) to obtain the HLG signal, which is a scene light signal and is nonlinear; the scene light signal generally needs to be converted into a display light signal through OOTF before being displayed on the display device.

[0078] Linear display light signal: In HDR video technology, the HDR video signal with content as display light refers to the display light emitted by the display device, which is generally an absolute value in nits. The linear display light signal is encoded by perceptual quantization (PQ) to obtain a PQ signal, which is a display light signal and a nonlinear signal. The display light signal is generally displayed on the display device according to its absolute brightness.

[0079] Opto-optical transfer function (OOTF): A curve used in video technology to convert one optical signal into another.

[0080] Dynamic range: The ratio of the maximum brightness to the minimum brightness in a video signal.

[0081] Luma-chroma-chroma (LCC), separates the three components of a video signal into luminance and chrominance.

[0082] The optical-electro transfer function (OETF) represents the conversion relationship between the linear signal and the nonlinear signal of an image pixel. The commonly used OETFs at this stage include the following three:

[0083] Perceptual quantizer (PQ) photoelectric transfer function, hybrid log-gamma (HLG) photoelectric transfer function and scene luminance fidelity (SLF) photoelectric transfer function. These three photoelectric transfer functions are photoelectric transfer functions specified by the audio video coding standard (AVS).

[0084] Dynamic range is used in many fields to express the ratio of the maximum value to the minimum value of a variable. In digital images, the dynamic range represents the ratio between the maximum grayscale value and the minimum grayscale value within the displayable range of the image. The dynamic range in nature is quite large. The brightness of a night scene under the starry sky is about 0.001cd / m2, and the brightness of the sun itself is as high as 1,000,000,000cd / m2. Such a dynamic range reaches 1,000,000,000 / 0.001=1013. However, in real scenes in nature, the brightness of the sun and the brightness of the stars are not obtained at the same time. For natural scenes in the real world, the dynamic range is between 10 -3 to 10 6 In most color digital images, each channel of R, G, and B is stored in one byte of 8 bits. That is to say, the range of each channel is 0 to 255 gray levels. Here, 0 to 255 is the dynamic range of the image. -3 to 10 6 We call it high dynamic range (HDR), while the dynamic range of ordinary pictures is called low dynamic range (LDR). The imaging process of digital cameras is actually the mapping of the high dynamic range of the real world to the low dynamic range of photos. This is often a nonlinear process.

[0085] Figure 1 is a schematic diagram of dynamic range mapping.

[0086] The PQ photoelectric transfer function is a perceptual quantization photoelectric transfer function proposed based on the brightness perception model of the human eye. Figure 2 , Figure 2is the image of the PQ photoelectric transfer function. The PQ photoelectric transfer function represents the conversion relationship from the linear signal value of the image pixel to the nonlinear signal value in the PQ domain.

[0087] The HLG photoelectric transfer function is an improvement on the traditional gamma curve. Figure 3 , Figure 3 This is an image of the HLG photoelectric transfer function. The HLG photoelectric transfer function uses the traditional gamma curve in the low segment and supplements the log curve in the high segment. The HLG photoelectric transfer function represents the conversion relationship between the linear signal value of the image pixel and the nonlinear signal value in the HLG domain.

[0088] The SLF photoelectric transfer function is the optimal curve obtained based on the brightness distribution of the HDR scene while satisfying the optical characteristics of the human eye. Figure 4 , Figure 4 is the image of SLF photoelectric transfer function. SLF photoelectric transfer curve represents the conversion relationship from linear signal value of image pixel to nonlinear signal value in SLF domain.

[0089] Linear space: In this application, linear space refers to the space where linear optical signals are located.

[0090] Nonlinear space: In this application, nonlinear space refers to the space where linear light signals are converted using nonlinear curves; nonlinear curves commonly used in HDR include PQ EOTF-1 curve, HLG OETF curve, etc., and nonlinear curves commonly used in SDR include gamma curves; it is generally believed that linear light signals are visually linear relative to the human eye after being encoded by the above nonlinear curves. It should be understood that nonlinear space can be considered as visual linear space.

[0091] Gamma correction: Gamma correction is a method of nonlinear color editing of images. It can detect the dark and light parts of the image signal and increase the ratio of the two, thereby improving the image contrast effect. The photoelectric conversion characteristics of current display screens, photographic films and many electronic cameras can be nonlinear. The relationship between the output and input of these nonlinear components can be expressed by a power function, that is, output = (input) γ.

[0092] The nonlinear conversion of the color values ​​output by the device is due to the fact that the human visual system is not linear, and humans perceive visual stimuli through comparison. The external world strengthens the stimulus in a certain proportion, and for humans, this stimulus increases evenly. Therefore, for human perception, the physical quantity that increases in geometric progression is uniform. In order to display the input color according to the laws of human vision, it is necessary to convert the linear color value into a nonlinear color value through the nonlinear conversion in the form of the above power function. The value of gamma γ can be determined according to the photoelectric conversion curve of the color space.

[0093] Color space: Color can be the different perceptions of the eye for light of different frequencies, or it can represent objectively existing light of different frequencies. Color space is a range of colors defined by a coordinate system established by people to represent colors. A color gamut and a color model together define a color space. Among them, a color model is an abstract mathematical model that represents color with a set of color components. A color model may include, for example, a three-primary color light model (red green blue, RGB) and a four-color printing model (cyan magenta yellow key plate, CMYK). A color gamut refers to the sum of colors that a system can produce. Exemplarily, Adobe RGB and sRGB are two different color spaces based on the RGB model. Each device, such as a monitor or printer, has its own color space and can only generate colors within its color gamut. When an image is moved from one device to another, the color of the image may change on different devices because each device converts and displays RGB or CMYK according to its own color space.

[0094] RGB space refers to a space that uses the brightness of the three primary colors of red, green, and blue to quantitatively represent video signals; YCC space is a color space that represents the separation of brightness and color. The three components of the YCC space video signal represent brightness, chromaticity, and chromaticity respectively. Common YCC space video signals include YUV, YCbCr, ICtCp, etc.

[0095] In order to obtain images with a higher dynamic range, the bit width of the image is usually greater than or equal to 10 bits. Common coding standards that support HDR include H.266, H.265 and high efficiency image format (HEIF). The common joint photographic experts group (JPEG) and H.264 only support 8-bit encoding, so they cannot support HDR video and HDR images well.

[0096] HDR images or HDR videos may need to be transcoded during distribution or transmission. The current HDR transcoding solution is to directly transcode HDR images or HDR videos. After decoding, HDR images or HDR videos obtain HDR pixel values ​​and HDR format identifiers and information. HDR pixel values ​​usually need to be processed (such as image up and down sampling, enhancement, etc.). HDR format identifiers and information, as well as processed HDR pixel values ​​are then sent to the encoder for re-encoding. The re-encoded HDR images or HDR videos are distributed to other devices. This solution of directly transcoding HDR images or HDR videos (which can be called a "single-layer transcoding solution") may cause abnormalities in the transcoded HDR images or HDR videos, affecting the user experience. Because when a single-layer coded HDR bitstream is transcoded (decoded and then re-encoded), it depends on the transcoding system to correctly send all the HDR information obtained by the decoder to the encoder. If the information is lost, the transcoding system will fail to re-encode in the correct format, resulting in an erroneous bitstream, which will result in abnormal visual effects when the user watches, seriously affecting the user experience. Because many of the distribution and transcoding systems on the market have not been upgraded to support the correct transmission of HDR information from decoding to encoding, mainstream mobile phone and application manufacturers have received many user complaints in this regard.

[0097] For ease of description, the term "HDR object" is used in some embodiments of the present application. An HDR object can be a static HDR image (also referred to as an HDR image, HDR photo, HDR picture, etc.), an HDR video or other types of dynamic HDR images, or a frame of an HDR video or a dynamic HDR image.

[0098] It is understandable that, for the sake of ease of description, some embodiments of the present application take HDR images as an example to introduce the technical solution of the present application. However, it is understandable that these embodiments can be applied not only to HDR images, but also to other HDR objects, such as HDR videos, dynamic HDR images, and a frame of an HDR video or dynamic HDR image.

[0099] The following combination Figure 5 Describe the coding and decoding system used in this application. Figure 5 A schematic block diagram of a coding and decoding system used in an embodiment of the present application is shown, for example, a video coding and decoding system 10 (or simply referred to as coding and decoding system 10) that can utilize the technology of the present application. The video encoder 20 (or simply referred to as encoder 20) and the video decoder 30 (or simply referred to as decoder 30) in the video coding and decoding system 10 represent devices that can be used to perform various technologies according to various examples described in the present application.

[0100] like Figure 5As shown, the codec system 10 includes a source device 12, which is used to provide coded data such as coded image data 21 to a target device 14 for decoding the coded data.

[0101] The source device 12 includes an encoder 20 . Optionally, the source device 12 may further include an image source 16 , a preprocessor 18 (or a preprocessing unit), and a communication interface or a communication unit 22 .

[0102] The image source 16 may include or may be any type of image capture device for capturing real-world images, etc., and / or any type of image generation device, such as a computer graphics processor for generating computer-animated images or any type of device for acquiring and / or providing real-world images, computer-generated images (e.g., screen content, virtual reality (VR) images, and / or any combination thereof (e.g., augmented reality (AR) images). The image source may be any type of memory or storage that stores any of the above images.

[0103] In order to distinguish the processing performed by the pre-processor 18 or the pre-processing unit 18 , the image or the image data 17 may also be referred to as a raw image or raw image data 17 .

[0104] The preprocessor 18 is used to receive (raw) image data 17 and preprocess the image data 17 to obtain preprocessed image data 19 or preprocessed image data 19. For example, the preprocessing performed by the preprocessor 18 may include trimming, color format conversion (e.g., from RGB to YCbCr), color adjustment, or denoising. It is understood that the preprocessing unit 18 may be an optional component.

[0105] The video encoder 20 is configured to receive the pre-processed image data 19 and provide encoded image data 21 .

[0106] The communication interface 22 in the source device 12 may be used to receive the encoded image data 21 and send the encoded image data 21 (or any other processed version thereof) to another device such as the target device 14 or any other device via the communication channel 13 for storage or reconstruction.

[0107] The target device 14 includes a decoder 30 (eg, a video decoder 30 ) and, in addition or alternatively, may include a communication interface or communication unit 28 , a post-processor 32 (or a post-processing unit 32 ), and a display device 34 .

[0108] The communication interface 28 in the target device 14 is used to receive the encoded image data 21 (or any other processed version) directly from the source device 12 or from any other source device such as a storage device, for example, the storage device is a encoded image data storage device, and provide the encoded image data 21 to the decoder 30.

[0109] The communication interface 22 and the communication interface 28 can be used to send or receive the encoded image data 21 or the encoded data through a direct communication link between the source device 12 and the target device 14, such as a direct wired or wireless connection, or through any type of network, such as a wired network, a wireless network or any combination thereof, any type of private network and public network or any combination thereof. For example, the communication interface 22 can be used to encapsulate the encoded image data 21 into a suitable format such as a message, and / or use any type of transmission coding or processing to process the encoded image data for transmission over a communication link or communication network. The communication interface 28 corresponds to the communication interface 22, for example, and can be used to receive the transmission data, and use any type of corresponding transmission decoding or processing and / or decapsulation to process the transmission data to obtain the encoded image data 21. Both the communication interface 22 and the communication interface 28 can be configured as follows: Figure 5 A unidirectional communication interface, or a bidirectional communication interface, is indicated by an arrow pointing from the source device 12 to the corresponding communication channel 13 of the target device 14, and can be used to send and receive messages, etc. to establish a connection, confirm and exchange any other information related to the communication link and / or data transmission such as encoded image data transmission, etc.

[0110] The decoder 30 is configured to receive the encoded image data 21 and to provide decoded image data 31 or decoded images 31 .

[0111] The post-processor 32 in the target device 14 is used to post-process the decoded image data 31 (also called reconstructed image data) such as the decoded image to obtain the post-processed image data 33 such as the post-processed image. The post-processing performed by the post-processor 32 may include, for example, color format conversion (e.g., conversion from YCbCr to RGB), color adjustment, cropping or resampling, or any other processing for generating the decoded image data 31 for display by the display device 34 or the like.

[0112] The display device 34 in the target device 14 is used to receive the post-processed image data 33 to display the image to a user or viewer, etc. The display device 34 can be or include any type of display for displaying the reconstructed image, such as an integrated or external display screen or display. For example, the display screen can include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro LED display, a liquid crystal on silicon (LCoS), a digital light processor (DLP), or any other type of display screen.

[0113] although Figure 5 The source device 12 and the target device 14 are shown as independent devices, but the device embodiment may also include both the source device 12 and the target device 14 or the functions of both the source device 12 and the target device 14, that is, both the source device 12 or the corresponding functions and the target device 14 or the corresponding functions. In these embodiments, the source device 12 or the corresponding functions and the target device 14 or the corresponding functions may be implemented using the same hardware and / or software or by separate hardware and / or software or any combination thereof.

[0114] According to the description, Figure 5 The presence and (exact) division of different units or functions in the source device 12 and / or the target device 14 shown may vary depending on the actual device and application, as will be apparent to the skilled person.

[0115] Combine the following Figure 6 Describe the content providing system of the content distribution service applied by this application, Figure 6 A block diagram of a content providing system for implementing a content distribution service used in an embodiment of the present application. The content providing system 2100 includes a capture device 2102, a terminal device 2106, and an (optional) display 2126. The capture device 2102 communicates with the terminal device 2106 via a communication link 2104. The communication link may include the above-mentioned communication channel 13. The communication link 2104 includes, but is not limited to, WiFi, Ethernet, wired, wireless (3G / 4G / 5G), USB, or any combination thereof.

[0116] The capture device 2102 may encode the captured data using the encoding method shown in the embodiment of the present application. Alternatively, the capture device 2102 may distribute the captured data to a streaming media server (not shown in the figure), which encodes the data and transmits the encoded data to the terminal device 2106. The capture device 2102 includes but is not limited to a camera, a smart phone or tablet computer, a computer or laptop computer, a video conferencing system, a personal digital assistant (PDA), a vehicle-mounted device, or any combination thereof. In some embodiments, the capture device 2102 may include the above-mentioned source device 12.

[0117] The terminal device 2106 in the content providing system 2100 receives and regenerates the decoded data. The terminal device 2106 can be a device with data receiving and recovery capabilities, such as a smart phone or tablet computer, a computer / laptop 2110, a network video recorder (NVR) / digital video recorder (DVR) 2112, a television 2114, a set top box (STB) 2116, a video conferencing system 2118, a video surveillance system 2120, a personal digital assistant 2122, a vehicle-mounted device 2124 or any combination thereof, or such a device capable of decoding the above-mentioned encoded data. In some embodiments, the terminal device 2106 may include the above-mentioned target device 14. The terminal device 2106 can decode the received data. In some embodiments, the terminal device 2106 can be a video playback application, a streaming media playback application, a streaming media playback platform or a live broadcast platform running on the terminal device.

[0118] For terminal devices with a display, such as a smartphone or tablet, computer or laptop 2110, NVR / DVR 2112, TV 2114, PDA 2122, or vehicle-mounted device 2124, the terminal device can send the decoded data to its display. For terminal devices without a display, such as STB 2116, video conferencing system 3118, or video surveillance system 2120, connect the device to an external display 2126 to receive and display the decoded data.

[0119] When each device in the system performs encoding or decoding, the image encoding device or image decoding device shown in the embodiments of the present application may be used.

[0120] Combine the following Figure 7 A streaming media system to which the embodiments of the present application are applicable is described. Figure 7 A schematic diagram of a streaming media system workflow applicable to an embodiment of the present application.

[0121] The streaming media system includes a content creation module to generate the required content data, such as video or audio. The streaming media system also includes a video encoding module to encode the generated content through an encoder. The streaming media system also includes a video stream transmission module to transmit the encoded video in the form of a code stream. Optionally, the format of the video stream can be converted into a code stream format of a common transmission protocol for OTT (over-the-top) devices, for example, the protocol includes but is not limited to Real Time Streaming Protocol (RTSP), Hyper Text Transfer Protocol (HTTP), HTTP Live streaming protocol (HLS), MPEG HTTP dynamic adaptive streaming media (MPEG Dynamic Adaptive Streaming over HTTP, MPEG-DASH), Real-time Transport Protocol (RTP), Real-time Messaging Protocol (RTMP) or any combination thereof. Optionally, the video stream can be stored to store the original format of the video stream and / or the converted multiple code stream formats for easy use. Furthermore, the streaming media system also includes a video stream encapsulation module for encapsulating the video stream to generate an encapsulated video stream, which can be called a video streaming media package. Exemplarily, the video streaming media package can be generated according to the transcoded video stream or the stored video stream. Furthermore, the streaming media system also includes a content distribution network (CDN), which is used to distribute the video streaming media package to multiple OTT devices, such as mobile phones, computers, tablets, home projectors, etc.

[0122] It should be noted that video encoding, video streaming transmission, video stream transcoding, video stream storage, video streaming package generation and content distribution network can all be implemented on cloud servers.

[0123] Combine the following Figure 8 A schematic diagram describing an exemplary end-to-end process of the present application is provided.

[0124] like Figure 8As shown, the original video file (also called a master file) is obtained through material production (such as shooting a video, making a computer graphics (CG) video, etc.), editing, color grading and other procedures. Then, based on the original video file, the corresponding dynamic metadata is obtained. After encoding the original video file and the dynamic metadata, a compressed video is obtained. The compressed video is distributed / transmitted to the terminal device (for example, a computer, a set-top box, a mobile phone, a tablet computer, etc.). The terminal device decodes the compressed video to obtain a decompressed video, and then displays it to the user through a display device (for example, a monitor, a TV, etc.).

[0125] Combine the following Fig. 9 An exemplary streaming media system architecture of the present application is described. The streaming media system architecture includes: a client device, a content distribution network, and a cloud server.

[0126] The user on the client device sends a play or playback request to the cloud platform.

[0127] The cloud platform makes a decision, responds to the client device, and sends the address of the client-requested content on the CDN to the client device.

[0128] The client device then requests the CDN to play the content based on the address, and the CDN provides the content to the client device, ultimately completing the request of the client device.

[0129] Combine the following Fig.10 Describe the system architecture applicable to the embodiments of the present application. Fig.10 This is a schematic diagram of a possible system architecture applicable to an embodiment of the present application. The system architecture of the embodiment of the present application includes: a front-end device, a transmission link, and a terminal display device.

[0130] Among them, the front-end device is used to capture or produce HDR / SDR content (for example, HDR / SDR video or image).

[0131] In a possible embodiment, the front-end device can also be used to extract corresponding metadata from the HDR content, and the metadata may include global mapping information, local mapping information, and dynamic metadata and static metadata corresponding to the HDR content. The front-end device can send the HDR content and metadata to the terminal display device through a transmission link. Specifically, the HDR content and metadata can be transmitted in the form of one data packet, or in two data packets respectively, which is not specifically limited in the embodiments of the present application.

[0132] Optionally, the terminal display device can be used to receive metadata and HDR content, and extract the global mapping information, local mapping information and the terminal display device information contained in the corresponding metadata according to the HDR content, obtain a mapping curve to perform global tone mapping and local tone mapping on the HDR content, convert it into display content adapted for the HDR display device or SDR device in the terminal display device, and display it. It should be understood that in different embodiments, the terminal display device may include a display device with a display capability of a lower dynamic range or a higher dynamic range than the HDR content generated by the front-end device, and this application is not limited thereto.

[0133] Optionally, the front-end device and the terminal display device in the present application may be independent and different physical devices. For example, the front-end device may be a video acquisition device or a video production device, wherein the video acquisition device may be a video camera, a camera, an image plotter, etc. The terminal display device may be a device with a video playback function, such as virtual reality (VR) glasses, a mobile phone, a tablet, a television, a projector, etc.

[0134] Optionally, the transmission link between the front-end device and the terminal display device can be a wireless connection or a wired connection, wherein the wireless connection can adopt technologies including, for example, long term evolution (LTE), fifth generation (5G) mobile communications, and future mobile communications. Wireless connections can also include technologies such as wireless fidelity (WiFi), Bluetooth, and near field communication (NFC). Wired connections can include Ethernet connections, local area network connections, etc. There is no specific limitation on this.

[0135] The present application may also integrate the functions of the front-end device and the functions of the terminal display device on the same physical device, for example, a terminal device such as a mobile phone or a tablet with a video shooting function. The present application may also integrate some functions of the front-end device and some functions of the terminal display device on the same physical device. There is no specific limitation on this.

[0136] Fig.11 Schematic diagram of an image processing method provided according to an embodiment of the present application. Fig.11 As shown, the image processing method provided in the embodiment of the present application can be applied at the decoding end.

[0137] The decoding module (also referred to as a decoder) can obtain the base layer data, the enhancement layer data and the metadata from the received bitstream. Then, the decoding module can synthesize the base layer data and the enhancement layer data to obtain an HDR image.

[0138] The graphics processing module can process the HDR image and send the processed HDR image and basic layer data to the display module. The graphics processing module can also process the basic layer data and send the processed HDR image and basic layer data to the display module.

[0139] The display module may present an HDR image based on the received data (eg, a processed HDR image, or a processed HDR image and processed base layer data, etc.).

[0140] For the sake of ease of description, the HDR image obtained by the decoding module based on the basic layer data and the enhanced layer data can be called HDR image 1, the HDR image obtained after the graphics processing module processes the HDR image 1 can be called HDR image 2, and the HDR image displayed by the display module can be called HDR image 3.

[0141] The base layer data, which may also be referred to as a base layer image, a base image or a basic image, may be an SDR image or an HDR image with a lower dynamic range.

[0142] The enhancement layer data, which may also be referred to as enhancement data, enhancement layer image or enhancement image, may include some image detail information. Thus, the image detail information contained in the enhancement layer data is used to supplement the base layer data, thereby synthesizing an HDR image with better contrast (i.e., the HDR image 1 mentioned above).

[0143] The dynamic range of the base layer data is smaller than the dynamic range of the HDR image (ie, HDR image 1) determined based on the base layer data and the enhancement layer data.

[0144] For example, in some embodiments, the base layer data may be an SDR image, and HDR image 1 is an HDR image.

[0145] For another example, in some other embodiments, the base layer data may be an HDR image, but the dynamic range of the HDR image is smaller than the dynamic range of the HDR image 1 .

[0146] For example, assuming the dynamic range of the base layer data is DR x1 To DR x2 , the dynamic range of HDR image 1 is DR y1 To DR y2 In some embodiments, DR y1 Less than DR x1 , DR y2 Greater than DR x2 In other embodiments, DR y1 Less than DR x1 , DR y2 Equal to DR x2In other embodiments, DR y1 Equal to DR x1 , DR y2 Greater than DR x2 In other words, the base layer data can be considered to be an image having a smaller dynamic range than the HDR image 1.

[0147] Fig.12 FIG. 1 is a schematic flow chart of an image processing method provided according to an embodiment of the present application. Fig.12 The method shown can be performed by an electronic device or a component (such as a chip, SoC, etc.) in the electronic device. The electronic device can be as follows Figure 5 The target device 14, such as Figure 6 The terminal device 2106 shown or Fig.10 For ease of description, the following Fig.12 The method embodiment shown is described by taking execution by an electronic device as an example.

[0148] 1201. Obtain first basic layer data, first enhanced layer data, and first metadata.

[0149] The first base layer data is the base layer data corresponding to the first HDR image, and the first enhancement layer data is the enhancement layer data corresponding to the first HDR image. In other words, the first base layer data and the first enhancement layer data are the base layer data and enhancement layer data corresponding to the same HDR image.

[0150] The first base layer data, the first enhancement layer data and the first metadata may be obtained from a bitstream.

[0151] For example, if the code stream is obtained by encoding using HEVC or VVC, the electronic device can obtain the first metadata from the SEI of HEVC or VVC. For another example, in some embodiments, if the code stream is obtained by encoding using AVC / H.264, the electronic device can obtain the first metadata from the NAL unit or the reserved packet unit. For another example, in some embodiments, the electronic device can obtain the first metadata from the APP extension information encapsulated in the JPEG file interchange format. For another example, in some embodiments, the electronic device can obtain the first metadata from the data segment encapsulated by the moving picture experts group (MPEG)-Part 14 (MPEG-4part 14, MP4).

[0152] The electronic device can decode the code stream to obtain the first base layer data, the first enhancement layer data, and the first metadata. The electronic device can decode the received code stream through a decoder (such as HEVC, JPEG, HEIF, etc.) to obtain the first base layer data, the first enhancement layer data, and the first metadata.

[0153] The embodiment of the present application does not limit the bitstream format of the first basic layer data and the first enhancement layer data. In terms of color space, it can be YUV, RGB, Lab, HSV, etc. In terms of data bit width, it can be 8 bits, 10 bits, or 12 bits, etc. In terms of value domain, it can be PQ, HLG, gamma, or log, etc.

[0154] 1202. Convert target data according to the first metadata to obtain target converted data.

[0155] The target data may be the first base layer data and / or the first enhancement layer data. That is, in some embodiments, the electronic device may convert the first base layer data according to the first metadata. In other embodiments, the electronic device may convert the first enhancement layer data according to the first metadata. In other embodiments, the electronic device may determine to convert the first base layer data and the first enhancement layer data according to the first metadata.

[0156] In some embodiments, the conversion of the target data may include one or more of the following: color space conversion, color gamut conversion, or color compensation.

[0157] In some embodiments, the first metadata may include conversion position information. The conversion position information is used to indicate the location where the conversion occurs. For example, the location where the conversion occurs may be before determining the second HDR image and / or after determining the second HDR image. If the location where the conversion occurs occurs before determining the second HDR image, the target data may be converted. If the location where the conversion occurs is after determining the second HDR image, the second HDR image may be converted.

[0158] In other embodiments, the location where the conversion occurs may be pre-set or pre-negotiated. In this case, the corresponding target data may be converted directly according to the location where the conversion occurs that is pre-set or pre-negotiated. For example, if the pre-set or pre-negotiated location is before the second HDR image is determined, then the target data may be converted before the second HDR image is determined; if the pre-set or pre-negotiated location is after the second HDR image is determined, then the determined second HDR image may be converted after the second HDR image is determined.

[0159] In some embodiments, the target data is determined according to the conversion position information. For example, if the location where the conversion occurs is after the second HDR image is determined, the target data includes the second HDR data. For another example, if the location where the conversion occurs is before the second HDR image is determined, the target data includes the first base layer data and / or the first enhancement layer data.

[0160] In some embodiments, the first metadata may include conversion judgment information, and the conversion judgment information is used to indicate whether to convert the target data. Exemplarily, according to the content of the indication, the conversion judgment information can be divided into first conversion judgment information or second conversion judgment information, and the first conversion judgment information is used to indicate whether to convert the target data, and the second conversion judgment information is used to indicate not to convert the target data. For example, if the conversion indication carried in the first metadata is the first conversion judgment information, then the target data can be converted, and then a second HDR image can be obtained based on the converted data; if the conversion judgment information carried in the first metadata is the second conversion judgment information, then there is no need to convert the target data.

[0161] In other embodiments, whether to convert the target data may also be predetermined, pre-negotiated, or determined based on the target data. For example, in some embodiments, it may be determined by default or in advance that the first basic layer data is always converted without converting the first enhanced layer data (in this case, the target data is the first basic layer data). For example, in other embodiments, it may be determined whether the target data needs to be converted based on the target data. For example, assume that the target data is the first basic layer data and the color space needs to be converted. If it is determined that the color space of the first basic layer data is different from the target color space (e.g., RGB color space), the first basic layer data needs to be converted to obtain the converted first basic layer data; if it is determined that the color space of the first basic layer data is the same as the target color space, the first basic layer data does not need to be converted.

[0162] In some embodiments, the first metadata may further include conversion object information. The conversion object information is used to indicate the target data that needs to be converted. For example, in some embodiments, the conversion object information may be used to indicate that the target data that needs to be converted is the first basic layer data. As another example, in some embodiments, the conversion object information may be used to indicate that the target data that needs to be converted is the first enhanced layer data. As another example, in some embodiments, the conversion object information may be used to indicate that the target data that needs to be converted is the first basic layer data and the first enhanced layer data. As another example, in some embodiments, the conversion object information may be used to indicate that the target data that needs to be converted is the second HDR image data.

[0163] In some other embodiments, the target data may be preset or pre-negotiated. In this case, the target data may be directly converted. For example, in some embodiments, the preset or pre-negotiated target data is the first basic layer data. In this case, the first basic layer data may be directly converted.

[0164] In some embodiments, the first metadata may further include conversion scheme information. The conversion scheme information is used to indicate a conversion scheme for the target data. The conversion scheme includes one or more of the following: color space conversion, color gamut conversion, or color compensation.

[0165] In other embodiments, the conversion scheme may be preset or pre-negotiated. In this case, the target data may be directly converted according to the preset or pre-negotiated conversion scheme.

[0166] In some embodiments, the first metadata may carry conversion indication information. In some embodiments, the conversion indication information may include one or more of the following information: conversion location information, conversion judgment information, conversion object information, and conversion scheme information. In some embodiments, the conversion indication information may include conversion location information, conversion judgment information, conversion object information, or conversion scheme information. That is, the first metadata may carry these four information. The electronic device may determine the location where the conversion occurs, whether conversion is required, and the target data based on the four information carried by the first metadata. In other embodiments, the conversion indication information may include part of the four information. For example, in some embodiments, the conversion indication information may include the conversion location information in the above four information. In this case, the electronic device may determine the location where the conversion needs to occur based on the conversion location information. Then, the electronic device may determine the target data, the conversion scheme, and whether the target data needs to be converted based on the result of the preset or pre-negotiation. For example, in other embodiments, the conversion indication information may include conversion location information and conversion scheme information. In this case, the electronic device may determine the target data based on the preset or pre-agreed, and determine the conversion location and conversion scheme based on the conversion location information and the conversion scheme information. In addition, the electronic device may determine that the target data needs to be converted when receiving the conversion indication information. In other words, in this case, it can be considered that the conversion indication information indicates that the target data needs to be converted. For another example, in some embodiments, the conversion indication information may include conversion position information, conversion scheme information, and conversion object information. In this case, the electronic device can determine the conversion position, conversion object, and target data based on the conversion indication information. Similarly, the electronic device can determine that the target data needs to be converted when receiving the conversion indication information. In other words, in this case, it can be considered that the conversion indication information indicates that the target data needs to be converted.

[0167] In some embodiments, the target data is converted according to the first metadata, and then the target data is converted to obtain a target conversion result. Exemplarily, the first metadata may include the above-mentioned conversion indication information. Exemplarily, the first metadata may include one or more of the above-mentioned conversion position information, conversion judgment information, conversion object information, and conversion scheme information.

[0168] In some embodiments, part of the above information may be implicitly indicated by other information. For example, in some embodiments, if the conversion position information in the first metadata indicates that the conversion position is before the second HDR image is determined, it may indicate that the target data needs to be converted. For example, it is assumed that 2 bits in the first metadata can be used to carry the conversion position information. If the values ​​of these two bits are 01, it indicates that the conversion position is before the second HDR image is determined; if the values ​​of these two bits are 10, it indicates that the conversion position is after the second HDR image is determined (i.e., the second HDR image needs to be converted); if the values ​​of these two bits are 00 or 11, it indicates that the target data or the second HDR image does not need to be converted. For another example, as described above, the target data may be the first basic layer data and / or the first enhanced layer data. Then if the conversion judgment information indicates that the target data needs to be converted, it can be determined that the location where the conversion occurs is before the second HDR image is determined. In some embodiments, the conversion judgment information can also be used to determine whether the second HDR image needs to be converted. For example, if the conversion judgment information indicates that the target data does not need to be converted, it can be determined that the second HDR image needs to be converted. Of course, in other embodiments, whether the second HDR image needs to be converted can also be determined by other information. For example, the conversion position information, the conversion object information, or a separate indication information for indicating whether to convert the second HDR image can be used. For another example, in some embodiments, if the target data indicated by the conversion object information is the first basic layer data, then it means that the first basic layer data needs to be converted; if the target data indicated by the conversion object information is the first enhanced layer data, then it means that the first enhanced layer data needs to be converted; if the conversion object information indicates the first basic data and the first enhanced layer data, then it means that the first basic layer data and the first enhanced layer data need to be converted. For example, 2 bits in the first metadata can be used to carry the conversion object information. If the values ​​of these two bits are 01, it means that the target data is the first basic layer data; if the values ​​of these two bits are 10, it means that the target data is the first enhanced layer data; if the values ​​of these two bits are 11, it means that the target data is the first basic layer data and the first enhanced layer data; if the values ​​of these two bits are 00, it means that the first basic layer data and the first enhanced data do not need to be converted.

[0169] It is understandable that, in addition to the above information, the first metadata may also carry other information, such as data format, region division information, region traversal order information, image features, encoding compression format or curve parameters, and one or more metadata information units. The first metadata information unit may also include coordinate information, image features, curve parameters, and other data.

[0170] As described above, the target data may be first basic layer data and / or first enhanced layer data. The following describes how to convert the target data.

[0171] Convert the first basic layer data:

[0172] Color space conversion:

[0173] Exemplarily, the first basic layer data is a first color space. Through color space conversion, the first basic layer data can be converted from the first color space to a second color space. The first color space is different from the second color space.

[0174] For example, in some embodiments, the first basic layer data can be converted into a color space by K() to obtain the converted first basic layer data (i.e., target conversion data). K() can be a default matrix or a conversion matrix indicated by the first metadata. The first metadata can carry the parameters of the conversion matrix. The color space described in this application does not specify a specific occurrence, including but not limited to RGB, Lab, YUV, HSV, etc.

[0175] For example, in some other embodiments, the color space of the first basic layer can be converted to a specified color space. For ease of description, the specified color space can be referred to as a target color space. The target color space can be one of RGB, Lab, YUV, HSV, etc. For example, in some embodiments, assuming that the target color space is an RGB color space, the electronic device can perform color space conversion on the first basic layer data to obtain the target conversion data, and the color space of the target conversion data is an RGB color space.

[0176] In some embodiments, the target color space may be pre-set or pre-negotiated.

[0177] In some other embodiments, the target color space may also be determined according to the first enhancement layer data. The target color space is the color space of the first enhancement layer data.

[0178] In some other embodiments, the first metadata may also carry an indication information, where the indication information is used to indicate the target color space gamut conversion:

[0179] Exemplarily, the first basic layer data is a first color gamut space. Through color gamut conversion, the first basic layer data can be converted from the first color gamut space to a second color gamut space. The first color gamut space is different from the second color gamut space.

[0180] For example, in some embodiments, the first basic layer data can be converted to a color gamut through T() to obtain the converted first basic layer data (i.e., target conversion data). T() can be a default matrix or a conversion matrix indicated by the first metadata. The first metadata can carry the parameters of the conversion matrix. The color gamut space that does not specify a specific occurrence described in this application includes but is not limited to BT.2020, BT.709, DCI-P3, sRGB, etc.

[0181] For example, in some other embodiments, the color gamut space of the first basic layer may be converted to a specified color gamut space. For ease of description, the specified color gamut space may be referred to as a target color gamut space. The target color gamut space may be one of BT.2020, BT.709, DCI-P3, sRGB, etc. For example, in some embodiments, assuming that the target color gamut space is an sRGB color gamut space, the electronic device may perform a color gamut space conversion on the first basic layer data to obtain the target conversion data, and the color gamut space of the target conversion data is an sRGB color gamut space.

[0182] In some embodiments, the color gamut space may be pre-set or pre-negotiated.

[0183] In some other embodiments, the color gamut space may also be determined according to the first enhancement layer data. The color gamut space is the color gamut space of the first enhancement layer data.

[0184] In some other embodiments, the conversion scheme information may further include indication information, where the indication information is used to indicate the color gamut space.

[0185] Color compensation:

[0186] For example, in some embodiments, the first basic layer data may be color compensated by a compensation method C() to obtain converted first basic layer data (ie, target converted data).

[0187] In some embodiments, the compensation method C() may be a color compensation matrix. The color compensation matrix may be preset or indicated by the first metadata.

[0188] In some embodiments, the compensation method C() may be a conversion function. For example, the conversion function may be a logarithmic function, an exponential function, etc. The conversion function may be preset (or pre-negotiated) or may be indicated by the first metadata. For example, in some embodiments, the conversion function may be preset, and the parameters of the conversion function (e.g., the exponent) may be indicated by the first metadata. For another example, in some embodiments, the parameters of the conversion function may be preset, and the conversion function is indicated by the first metadata. For another example, in some other embodiments, the conversion function and the parameters are both preset or indicated by the first metadata.

[0189] In some embodiments, the compensation method C() may be a compensation value and a corresponding compensation method. For example, the data of one or more channels in the base layer data are added, subtracted, multiplied, and / or divided by the compensation value. The compensation value and the corresponding compensation method may be preset (or pre-negotiated) or indicated by the first metadata. For example, in some embodiments, the compensation method may be preset, and the first metadata may carry the compensation value. For another example, in some embodiments, the compensation value is preset, and the compensation method is indicated by the first metadata. For another example, in some embodiments, the compensation value and the compensation method may both be preset or indicated by the first metadata.

[0190] It is understandable that if the first basic layer data needs to be converted multiple times, the conversion object of the first conversion is the first basic layer data, and the object of each conversion except the first conversion is the conversion result of the previous one. For example, suppose that the first basic layer data needs to be converted into a color space and a color gamut. In this case, the first basic layer data can be converted into a color space first to obtain the first basic layer data with the converted color space; then the first basic layer data with the converted color space can be converted to obtain the final conversion result (i.e., the target conversion data).

[0191] Convert the first enhancement layer data:

[0192] Color space conversion:

[0193] The method for performing color space conversion on the first enhancement layer data is similar to the method for performing color space conversion on the first basic layer data. For example, a default matrix or a conversion matrix indicated by the first metadata may be used to perform color space conversion on the first enhancement layer data to obtain the converted first enhancement layer data. As another example, the color space of the first enhancement layer data may be converted to a specified color space. The specified color space may be pre-set or pre-negotiated, or may be determined based on the first basic layer data, or may be indicated by the first metadata. The method for performing color space conversion on the first enhancement layer data may refer to the above-mentioned method for performing color space conversion on the first basic layer data, and for the sake of brevity, it will not be repeated here.

[0194] Color space conversion:

[0195] Similarly, the color gamut conversion method of the first enhancement layer data is similar to the color gamut conversion method of the first basic layer data. For example, a default matrix or a conversion matrix indicated by the first metadata can be used to perform color gamut conversion on the first enhancement layer data to obtain the converted first enhancement layer data. For another example, the color gamut of the first enhancement layer data can be converted to a specified color gamut space. The specified color gamut space can be pre-set or pre-negotiated, or it can be determined based on the first basic layer data, or it can be indicated by the first metadata. The method for performing color gamut conversion on the first enhancement layer data can refer to the above-mentioned method for performing color space conversion on the first basic layer data, and for the sake of brevity, it will not be repeated here.

[0196] Color compensation

[0197] Similarly, the compensation method C() can be used to perform color compensation on the first enhancement layer data to obtain the converted first enhancement layer data. The compensation method C() can be a color compensation matrix, a conversion function, or a compensation value and a corresponding compensation method. The color compensation matrix, the conversion function, the compensation value and the corresponding compensation method can be preset (or pre-negotiated) or indicated by the first metadata. The specific method for color compensation of the first enhancement layer data can refer to the above-mentioned method for color compensation of the first basic layer data. For the sake of brevity, it will not be repeated here.

[0198] Similarly, multiple conversions may be performed on the first enhancement layer data. The specific conversion method may refer to the conversion method of the first basic layer data, and for the sake of brevity, it will not be described here.

[0199] In some embodiments, the first enhancement layer data may be subjected to channel conversion. The first enhancement layer data may be data obtained after reorganization. For ease of description, the first enhancement layer data before reorganization may be referred to as original enhancement layer data.

[0200] In some embodiments, the reorganization method may be to replace some of the original channels in the original enhanced data with the difference or sum of some of the channels. For example, if the original channels are RGB, the reorganized channels may be R, R+P1×G, R+P2×B. In other words, the channels of the first enhanced layer data are R, R+P1×G, R+P2×B. The electronic device may restore the channels of the first enhanced data to the original channels by performing channel conversion on the first enhanced data. P1 and P2 may be constants or values ​​indicated in the first metadata.

[0201] In some other embodiments, the reorganization method may be to replace some of the original channels in the original enhanced data with the product or quotient of some of the channels. For example, if the original channels are RGB, the reorganized channels may be R, R×P3×G, R×P4×B or R, R / (P3×G), R / (P4×B). The electronic device may restore the channels of the first enhanced data to the original channels by performing channel conversion on the first enhanced data. P3 and P4 are constants or values ​​indicated in the first metadata.

[0202] Therefore, the channels of the first enhancement layer data can be restored to the original channels according to the channel reorganization information. The channel reorganization information may exemplarily include the parameters P1, P2 or P3, P4 in the above embodiments or the values ​​indicated in the first metadata.

[0203] In summary, in some embodiments, some or all of the parameters used in the conversion process may be preset (or pre-negotiated). In other embodiments, the first metadata may also include conversion parameters, and the target data is converted using the conversion parameters. For example, the conversion parameters may include one or more of a conversion matrix, a color compensation matrix, a conversion function, a compensation value and a compensation method, channel reorganization information, and the like.

[0204] 1203 : Determine a second HDR image according to the target conversion data.

[0205] For example, if the target data includes first base layer data, the target conversion data includes the converted first base layer data. In this case, the second HDR image can be determined based on the first enhancement layer data and the converted first base layer data.

[0206] For another example, if the target data includes first enhancement layer data, the target conversion data includes the converted first enhancement layer data. In this case, the second HDR image can be determined according to the first base layer data and the converted first enhancement layer data.

[0207] For another example, if the target data includes first enhancement layer data and first base layer data, the target conversion data includes the converted first enhancement layer data and the converted first base layer data. In this case, the second HDR image can be determined based on the converted first enhancement layer data and the converted first base layer data.

[0208] For ease of description, the terms "second base layer data" and "second enhancement layer data" are introduced. At least one of the second base layer data and the second enhancement layer data is the target conversion data. In other words, in some embodiments, the second base layer data is the converted first base layer data, and the second enhancement layer data is the same as the first enhancement layer data. In other embodiments, the second base layer data is the same as the first base layer data, and the second enhancement layer data is the converted first enhancement layer data. In other embodiments, the second base layer data is the converted first base layer data, and the second enhancement layer data is the converted first enhancement layer data. In this case, the second HDR image can be determined based on the second base layer data and the second enhancement layer data.

[0209] For ease of description, Base[i] may be used to represent the value of the i-th pixel in the second base layer data, Enhance[i] may be used to represent the value of the i-th pixel in the second enhancement layer data, and newPicture[i] may be used to represent the value of the i-th pixel in the second HDR image. In some embodiments, newPicture[i], Base[i], and Enhance[i] satisfy the following relationship:

[0210] newPicture[i]=Base[i]×f(Enhance[i]), (Formula 1.1)

[0211] Here, f() may be a default conversion function or a conversion function indicated by the first metadata.

[0212] In some other embodiments, newPicture[i], Base[i] and Enhance[i] satisfy the following relationship:

[0213] newPicture[i]=Base[i]+f(Enhance[i]), (Formula 1.2)

[0214] Here, f() may be a default conversion function or a conversion function indicated by the first metadata.

[0215] In some embodiments, the second base layer data and the second enhancement layer data may also be processed, and then the processed data may be used to determine the second HDR image. For ease of description, the third base layer data is used to represent the processed second base layer data, and the third enhancement layer data is used to represent the processed second enhancement layer data. BaseAfter[i] is used to represent the value of the i-th pixel in the third base layer data, EnhanceAfter[i] is used to represent the value of the i-th pixel in the third enhancement layer data, and newPicture[i] is used to represent the value of the i-th pixel of the second HDR image. In some embodiments, newPicture[i], BaseAfter[i] and EnhanceAfter[i] satisfy the following relationship:

[0216] newPicture[i]=BaseAfter[i]×f(EnhanceAfter[i]), (Formula 2.1)

[0217] Here, f() may be a default conversion function or a conversion function indicated by the first metadata.

[0218] In some other embodiments, newPicture[i], Base[i] and Enhance[i] satisfy the following relationship:

[0219] newPicture[i]=BaseAfter[i]+f(EnhanceAfter[i]), (Formula 2.2)

[0220] Here, f() may be a default conversion function or a conversion function indicated by the first metadata.

[0221] In some embodiments, the second base layer data or the second enhancement layer data may be processed, and then the processed data may be used to determine the second HDR image. That is, in some embodiments, the second base layer data is processed to obtain the third base layer data, and then the second HDR image is determined based on the third base layer data and the second enhancement layer data. In this case, EnhanceAfter[i] in Formula 2.1 and Formula 2.2 may be replaced with Enhance[i]. In other embodiments, the second enhancement layer data may be processed to obtain the third enhancement layer data, and then the second HDR image may be determined based on the second base layer data and the third enhancement layer data. In this case, BaseAfter[i] in Formula 2.1 and Formula 2.2 may be replaced with Base[i].

[0222] The method of processing the second basic layer data to obtain the third basic layer data is the same as the method of processing the second enhanced layer data to obtain the third enhanced layer data. For simplicity, the following takes the enhanced layer as an example to introduce how to obtain the third enhanced layer processing. The method for determining the third basic layer data can refer to the method for determining the third enhanced layer data.

[0223] In some embodiments, the second enhancement layer data is normalized data. In this case, the first metadata may include a first parameter THH and a second parameter THL. In this case, the third enhancement layer data may be determined according to the following formula:

[0224] EnhanceAfter[i]=Enhance[i]×THH+(A-Enhance[i])×THL, (Formula 3.1)

[0225] Where A is the maximum value in the second enhancement layer data. When the second enhancement layer data is normalized to between 0 and 1.0, A is 1.0.

[0226] In some other embodiments, the second enhancement layer data may be encoded data. For example, the second enhancement layer data may be non-linear encoded data such as PQ / HLG / LOG / Gamma. In this case, the second enhancement layer data may be converted to a linear domain to obtain third enhancement layer data. If the second enhancement layer data is also normalized before being encoded to obtain non-linear data, the first metadata may include a first parameter THH and a second parameter THL. Then, after converting the second enhancement layer data to a linear domain, the third enhancement data may also be determined by formula 3.1.

[0227] In some other embodiments, the first metadata may also include a second parameter THL. In this case, the second parameter THL, EnhanceAfter[i] and Enhance[i] satisfy the following relationship:

[0228] EnhanceAfter[i]=Enhance[i]+THL, (Formula 3.2).

[0229] In some other embodiments, the first metadata may also include a first parameter THH. In this case, the first parameter THH, EnhanceAfter[i] and Enhance[i] satisfy the following relationship:

[0230] EnhanceAfter[i]=THH+A-Enhance[i], (Formula 3.3)

[0231] Where A is the maximum value in the second enhancement layer data. When the second enhancement layer data is normalized to between 0 and 1.0, A is 1.0.

[0232] In some other embodiments, the first metadata may be used to indicate a mapping function TMB(). In this case, the third enhancement layer data may be determined according to the mapping function TMB() indicated by the first metadata.

[0233] In some embodiments, the mapping function TMB(), EnhanceAfter[i] and Enhance[i] satisfy the following relationship:

[0234] EnhanceAfterβ[i]=TMB(Enhanceβ[i]), (Formula 3.4).

[0235] In some other embodiments, the mapping function TMB(), EnhanceAfter[i] and Enhance[i] satisfy the following relationship:

[0236] EnhanceAfter[i]=TMB()×Enhanceβ[i], (Formula 3.5).

[0237] During the encoding process, a mapping relationship can be determined according to the histogram of the intermediate enhanced data, and then the intermediate enhanced data can be mapped to another numerical range interval according to the mapping relationship. The first interval is still used to represent the numerical range interval of the intermediate enhanced data, and the second interval is used to represent the numerical range interval after mapping. The numerical range of the second enhanced layer data is within the second interval. Through the above technical solution, the second enhanced layer data can be remapped to the first interval. TMB() is the inverse function of the mapping relationship. In some embodiments, TMB() can be directly carried in the first metadata. In other embodiments, the first metadata can carry the mapping relationship and / or the parameters of the mapping relationship. TMB() can be inferred based on the mapping relationship and / or the parameters of the mapping relationship.

[0238] In some embodiments, after determining the second HDR image, the second HDR image may be further processed to obtain a third HDR image. For example, in some embodiments, the second HDR image may be subjected to color gamut conversion and / or color compensation to obtain a third HDR image. The implementation of color gamut conversion and / or color compensation may refer to the description of the first base layer data or the first enhancement layer data, which will not be described in detail here.

[0239] Unless otherwise specified, the embodiments of the present application do not limit the third enhancement layer data and the third base layer data to any domain, which may be a linear domain or a PQ domain or a log domain, etc. The present application also does not limit the color space of the third enhancement layer data and the third base layer data, which may be a color space such as YUV, RGB, Lab, or HSV.

[0240] Fig.13 FIG. 1 is a schematic flow chart of an image processing method provided according to an embodiment of the present application. Fig.13 As shown, after acquiring the first basic layer data, the first enhanced layer data and the first metadata, the electronic device may first determine whether the target data needs to be converted.

[0241] In the case where it is determined that the target data needs to be converted, the conversion position is determined. If the conversion position is before the second HDR image is determined, the target data may be the first basic layer data and / or the first enhanced layer data. After the target data is determined, the target data may be converted to obtain target conversion data. Then the second HDR image is determined according to the target conversion data.

[0242] If the conversion position is after the second HDR image is determined, then the second HDR image may be determined according to the first enhancement layer data and the first base layer data, and then the second HDR image may be converted.

[0243] If it is determined that the target data does not need to be converted, an HDR image may be determined based on the first enhancement layer data and the first base layer data.

[0244] Fig.14 FIG. 1 is a schematic flow chart of another image processing method provided according to an embodiment of the present application. Fig.14 The method shown can be performed by an electronic device or a component (such as a chip, SoC, etc.) in the electronic device. The electronic device can be as follows Figure 5 The source device 12 in Figure 6 The capture device 2102 shown or Fig.10 For ease of description, the following embodiments assume that Fig.14 The method shown is performed by an electronic device.

[0245] 1401. Determine, based on an acquired first HDR image, first basic layer data, first enhanced layer data, and first metadata corresponding to the first HDR image, wherein the first metadata is used to convert target data, the target data including the first basic layer data and / or the first enhanced layer data.

[0246] The embodiment of the present application does not limit the location of the first metadata. For example, if the bitstream is encoded using HEVC or VVC, then the first hospital data can be located in the SEI of HEVC or VVC. For another example, if the bitstream is encoded using AVC / H.264, then the first metadata can be located in a NAL unit or a reserved packet unit. For another example, the first metadata can also be located in the APP extension information encapsulated in the JPEG file interchange format. For another example, in some embodiments, the first metadata can also be located in a data segment encapsulated by the Moving Picture Experts Group (MPEG)-Part 14 (MPEG-4part 14, MP4).

[0247] The embodiments of the present application do not limit the encoder, for example, HEVC, JPEG, HEIF, etc. can be used.

[0248] The embodiment of the present application does not limit the bitstream format of the first basic layer data and the first enhancement layer data. In terms of color space, it can be YUV, RGB, Lab, HSV, etc. In terms of data bit width, it can be 8 bits, 10 bits, or 12 bits, etc. In terms of value domain, it can be PQ, HLG, gamma, or log, etc.

[0249] The conversion performed on the target data may include one or more of the following: color space conversion, color gamut conversion, or color compensation.

[0250] The first metadata may carry one or more of the following information: conversion position information, conversion judgment information, conversion object information, and conversion scheme information.

[0251] The specific contents of the conversion position information, conversion judgment information, conversion object information and conversion scheme information can be referred to the above embodiments, and will not be described again here for the sake of brevity.

[0252] In some embodiments, the first metadata is used to determine a transformation to be performed on the target data.

[0253] In some embodiments, the first metadata is used to determine target data and transform the target data.

[0254] 1402. Encode the first base layer data, the first enhancement layer data, and the first metadata to obtain a code stream.

[0255] Fig.15 is a schematic structural block diagram of an electronic device provided according to an embodiment of the present application. Fig.15The electronic device 1500 shown includes an acquisition unit 1501 and a processing unit 1502 .

[0256] The acquisition unit 1501 is used to acquire first basic layer data, first enhanced layer data and first metadata, wherein the first basic layer data is the basic layer data of the first HDR image, and the first enhanced layer data is the enhanced layer data of the first HDR image.

[0257] The processing unit 1502 is configured to convert the target data according to the first metadata to obtain target converted data, where the target data is the first basic layer data and / or the first enhanced layer data.

[0258] The processing unit 1502 is further configured to determine a second HDR image according to the target conversion data.

[0259] The specific functions and beneficial effects of the acquisition unit 1501 and the processing unit 1302 can be referred to the above method embodiment, and for the sake of brevity, they will not be repeated here.

[0260] In some embodiments, the acquisition unit 1501 and the processing unit 1502 may be implemented by a processor.

[0261] In some embodiments, the electronic device may be a decoder.

[0262] In some embodiments, the electronic device may be Figure 5 The target device 14, such as Figure 6 The terminal device 2106 shown or Fig.10 The terminal display device shown, etc.

[0263] Fig.16 is a schematic structural block diagram of an electronic device provided according to an embodiment of the present application. Fig.16 The electronic device 1600 shown includes a processing unit 1601 and an encoding unit 1602 .

[0264] The processing unit 1601 is used to determine first basic layer data, first enhanced layer data and first metadata according to the acquired first HDR image, wherein the first metadata is used to convert target data, and the target data includes the first basic layer data and / or the first enhanced layer data.

[0265] The encoding unit 1602 is configured to encode the first base layer data, the first enhancement layer data, and the first metadata to obtain a code stream.

[0266] The specific functions and beneficial effects of the processing unit 1601 and the encoding unit 1602 can be referred to the above method embodiment, and for the sake of brevity, they will not be repeated here.

[0267] In some embodiments, the processing unit 1601 and the encoding unit 1602 may be implemented by a processor.

[0268] In some embodiments, the electronic device may be an encoder.

[0269] In some embodiments, the electronic device may be Figure 5 The source device 12 in Figure 6 The capture device 2102 shown or Fig.10 The front-end equipment shown, etc.

[0270] The present application also provides an electronic device. The electronic device can be used to implement the above Fig.14 An embodiment of the image processing method (which may be referred to as a coding method) shown. The electronic device includes a processor and a memory. The processor is used to execute a computer program or instruction stored in the memory, or read data / signaling stored in the memory to execute the methods in the above method embodiments. The memory may be integrated with the processor, or may be separately provided. The memory may also be used to store a code stream determined by the processor. Optionally, the processor is one or more. Optionally, the memory is one or more. Optionally, the electronic device may also include a transceiver (or a communication interface), which is used to receive and / or send signals. For example, the transceiver may be used to send a code stream determined by the processor to another electronic device.

[0271] The present application also provides an electronic device. The electronic device can be used to implement the above-mentioned Fig.12 or Fig.13 An embodiment of the image processing method (which may be referred to as a decoding method) shown in the figure. The electronic device includes a processor and a memory. The processor is used to execute a computer program or instruction stored in the memory, or read data / signaling stored in the memory to execute the method in each method embodiment above. The memory may be integrated with the processor, or may be separately provided. Optionally, there are one or more processors. Optionally, there are one or more memories. Optionally, the electronic device may further include a transceiver (or a communication interface), which is used to receive and / or send signals. For example, the transceiver may be used to receive a code stream.

[0272] It should be understood that the processor mentioned in the embodiments of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0273] It should also be understood that the memory mentioned in the embodiments of the present application may be a volatile memory and / or a non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM). For example, RAM can be used as an external cache. By way of example and not limitation, RAM includes the following forms: static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0274] It should be noted that when the processor is a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, the memory (storage module) can be integrated into the processor.

[0275] It should also be noted that the memory described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0276] The embodiment of the present application also provides a chip system. The chip system (or also referred to as a processing system) includes a logic circuit and an input / output interface.

[0277] Among them, the logic circuit can be a processing circuit in the chip system. The logic circuit can be coupled to the storage unit and call the instructions in the storage unit so that the chip system can implement the methods and functions of each embodiment of the present application. The input / output interface can be an input / output circuit in the chip system, outputting information processed by the chip system, or inputting data or signaling information to be processed into the chip system for processing.

[0278] As a solution, the chip system is used to implement the embodiments of each of the above image processing methods. For example, the chip system is used to implement the processing-related operations performed by the source device, the capture device or the front-end device in the above method embodiments.

[0279] As a solution, the chip system is used to implement the embodiments of the above image processing methods. For example, the chip system is used to implement the processing-related operations performed by the target device, terminal device or terminal display device in the above method embodiments.

[0280] An embodiment of the present application also provides a computer-readable storage medium on which are stored computer instructions for implementing the methods executed by an encoding end device (eg, a source device, a capture device, or a front-end device) in the above-mentioned method embodiments.

[0281] An embodiment of the present application also provides a computer-readable storage medium on which are stored computer instructions for implementing the methods executed by a decoding end device (eg, a target device, a terminal device, or a terminal display device) in the above-mentioned method embodiments.

[0282] An embodiment of the present application also provides a computer program product, comprising instructions, which, when executed by a computer, implement the methods performed by the encoding end device in the above-mentioned method embodiments.

[0283] The embodiment of the present application also provides a computer program product, which includes instructions, and when the instructions are executed by a computer, the methods performed by the decoding end device in the above-mentioned method embodiments are implemented.

[0284] An embodiment of the present application also provides a communication system, including the aforementioned encoding end device and decoding end device.

[0285] The embodiment of the present application also provides a code stream, which is determined by the above-mentioned image processing method (encoding method).

[0286] The embodiment of the present application also provides a memory, which is used to store the code stream determined by the above encoding method.

[0287] The present application also provides a computer device, which includes a memory, and the memory is used to store a code stream determined by the above encoding method.

[0288] The explanation of the relevant contents and beneficial effects of any of the above-mentioned devices can be referred to the corresponding method embodiments provided above, which will not be repeated here.

[0289] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0290] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. For example, the computer can be a personal computer, a server, or a network device, etc. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media integrations. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)). For example, the aforementioned available medium includes, but is not limited to, various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0291] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for image processing, characterized in that: include: Acquire first base layer data, first enhancement layer data, and first metadata, wherein the first base layer data is base layer data corresponding to the first HDR image, and the first enhancement layer data is enhancement layer data corresponding to the first HDR image; Converting target data according to the first metadata to obtain target converted data, where the target data is the first basic layer data and / or the first enhanced layer data; A second HDR image is determined according to the target conversion data.

2. The method according to claim 1, characterized in that The first metadata includes conversion position information, where the conversion position information is used to indicate a position where the conversion occurs. The converting the target data according to the first metadata includes: The target data is converted according to the conversion position information.

3. The method according to claim 1 or 2, characterized in that: The first metadata includes first conversion judgment information, and the first conversion judgment information is used to indicate that the target data needs to be converted; The converting the target data according to the first metadata includes: The target data is converted according to the first conversion judgment information.

4. The method according to any one of claims 1 to 3, characterized in that The conversion includes one or more of the following: Color space conversion, color gamut conversion, or color compensation.

5. The method according to claim 4, characterized in that In a case where the target data is the first basic layer data and the conversion includes color space conversion, converting the target data according to the first metadata comprises: The color space conversion is performed on the first basic layer data to obtain the target conversion data, and the color space of the target conversion data is an RGB color space.

6. The method according to claim 4 or 5, characterized in that: In a case where the target data is the first basic layer data and the conversion includes the color gamut conversion, converting the target data according to the first metadata includes: The target conversion data is obtained by performing the color gamut conversion on the first basic layer data through T(), where T() is a default matrix or a conversion matrix indicated by the first metadata.

7. The method according to claim 4, characterized in that In a case where the target data includes the first enhancement layer data, the conversion further includes channel conversion, and the channel conversion is used to restore channels of the first enhancement layer data to original channels.

8. The method according to claims 1 to 7, characterized in that The first metadata also includes conversion parameters used when performing the conversion.

9. The method according to claim 8, characterized in that In case the conversion includes the color compensation, the conversion parameters include: Color compensation matrix; a conversion function; or Compensation value and compensation method.

10. The method according to any one of claims 1 to 9, characterized in that The method further includes: performing color gamut conversion and / or color compensation on the second HDR image.

11. A method for image processing, characterized in that: The method comprises: Determine, according to the acquired first HDR image, first base layer data, first enhancement layer data, and first metadata corresponding to the first HDR image, wherein the first metadata is used to convert target data, and the target data is the first base layer data and / or the first enhancement layer data; The first basic layer data, the first enhancement layer data, and the first metadata are encoded to obtain a code stream.

12. The method according to claim 11, characterized in that The first metadata includes conversion position information, where the conversion position information is used to indicate a position where the conversion occurs.

13. The method according to claim 11 or 12, characterized in that: The first metadata includes first conversion judgment information, where the first conversion judgment information is used to indicate that the target data needs to be converted.

14. The method according to any one of claims 11 to 13, characterized in that: The conversion includes one or more of the following: Color space conversion, color gamut conversion, or color compensation.

15. The method according to claim 14, characterized in that The first metadata also includes conversion parameters used when performing the conversion.

16. The method according to claim 15, characterized in that In case the conversion comprises the color gamut conversion, the conversion parameters comprise a conversion matrix.

17. The method according to claim 15, characterized in that In case the conversion includes the color compensation, the conversion parameters include: Color compensation matrix; a conversion function; or Compensation value and compensation method.

18. An electronic device, characterized in that: The electronic device is used to implement the method as claimed in any one of claims 1 to 10, and / or is used to implement the method as claimed in any one of claims 11 to 17.

19. A computer device, characterized in that: include: A processor, the processor being configured to be coupled to a memory, read and execute instructions and / or program codes in the memory to execute the method according to any one of claims 1 to 10, and / or to execute the method according to any one of claims 11 to 17.

20. A chip system, characterized in that: include: A logic circuit, the logic circuit is used to be coupled to an input / output interface, and transmit data through the input / output interface to execute the method according to any one of claims 1 to 10, and / or to execute the method according to any one of claims 11 to 17.

21. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program code, and when the computer program code is executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 10 and / or to execute the method according to any one of claims 11 to 17.

22. A code stream, characterized in that: The code stream is determined according to the method according to any one of claims 11 to 17.

Images