Image encoding method, image decoding method and related device

Abstract

The present application discloses an image encoding method, applied to the technical field of data processing. The image encoding method comprises: encoding a target image into a bitstream; and encoding a first identifier into the bitstream, wherein the first identifier indicates an opto-electronic transfer function that matches the target image, and the target image comprises a high dynamic range (HDR) image and / or a standard dynamic range (SDR) image. That is, according to the present application, when encoding a target image, a first identifier indicating an opto-electronic transfer function that matches the target image is also encoded into a bitstream, to meet the display requirements of both SDR and HDR images at a decoding end, so that when decoding the first identifier, a decoder can use the corresponding opto-electronic transfer function to render and display a reconstructed image.

Description

Image encoding and decoding methods and related devices Technical Field

[0001] This application relates to the field of data processing technology, and in particular to an image encoding and decoding method and related apparatus. Background Technology

[0002] JPEG AI is a machine learning-based image encoding and decoding standard. The encoding and decoding scheme based on JPEG AI is based on a color component separation autoencoder structure. It achieves image encoding and decoding by encoding and decoding the main components and secondary components of the image separately.

[0003] The image encoding process based on JPEG AI is as follows: First, the RGB (a color space where R represents red, G represents green, and B represents blue) format image is converted to a YUV (a color space where Y represents luminance, and U and V represent chrominance) format image. Then, the Y component image and the UV component image are downsampled separately to reduce data processing volume. Subsequently, the downsampled Y component image and UV component image are encoded separately to obtain the Y component bitstream and the UV component bitstream, respectively. The image decoding process based on JPEG AI is as follows: First, the Y component bitstream and the UV component bitstream are parsed separately to obtain the reconstructed Y component image and the reconstructed UV component image. Then, the reconstructed Y component image and the reconstructed UV component image are upsampled separately to obtain the YUV format reconstructed image. Finally, the YUV format reconstructed image is converted to the RGB format reconstructed image through color space inverse conversion.

[0004] However, when displaying RGB format images after decoding, JPEG AI uses a photoelectric transfer function based on the "Gamma" function by default. This photoelectric conversion function works well for displaying standard dynamic range (SDR) images on traditional display devices (illuminance around 100 cd / m2), but it cannot accurately display HDR images on display devices for high dynamic range (HDR) images.

[0005] Summary of the Invention

[0006] This application provides an image encoding / decoding method and related apparatus that can achieve accurate display of HDR images.

[0007] In a first aspect, an image encoding method is provided, comprising: encoding a target image into a bitstream; and encoding a first identifier into the bitstream, wherein the first identifier indicates a photoelectric conversion function matching the target image, and the target image includes a high dynamic range (HDR) image and / or a standard dynamic range (SDR) image.

[0008] Optionally, the bitstream includes an image header, in which the first identifier is encoded.

[0009] Optionally, the method further includes: encoding a second identifier into the bitstream, the second identifier indicating the chromaticity coordinates of each primary color of the target image; encoding a third identifier into the bitstream, the third identifier indicating the pixel mapping range of the target image; and encoding a fourth identifier into the bitstream, the fourth identifier indicating the coefficients of the transformation matrix used for color space conversion of the target image.

[0010] Optionally, before encoding the target image into the bitstream, the method further includes: performing color space conversion on the target image based on the second identifier, the third identifier, and the fourth identifier.

[0011] Optionally, the bitstream includes an image header, in which the second identifier, the third identifier, and the fourth identifier are encoded.

[0012] Optionally, the method further includes: encoding a fifth identifier into the bitstream, the fifth identifier indicating the offset of the sampling positions of the principal and secondary components of the target image.

[0013] Optionally, the target image, after color space conversion, includes an image of the principal component and an image of the secondary component; before encoding the target image into the bitstream, the method further includes: downsampling the image of the principal component and the image of the secondary component based on the fifth identifier; encoding the target image into the bitstream includes: encoding the downsampled image of the principal component and the downsampled image of the secondary component into the bitstream.

[0014] Optionally, the bitstream includes an image header, and the fifth identifier is encoded into the image header.

[0015] Optionally, the method further includes: encoding static metadata of the target image into the bitstream, wherein the static metadata includes source device information and image display information, wherein the source device information indicates the color information of the main monitor that generates the target image, and the image display information indicates the display brightness of the target image.

[0016] Optionally, the source device information includes at least one of the chromaticity coordinates of the three primary colors of the main monitor, the chromaticity coordinates of standard white light, the maximum display brightness, and the minimum display brightness.

[0017] Optionally, the image display information includes at least one of the maximum display brightness and the maximum average display brightness of the target image.

[0018] Optionally, the bitstream includes an image header carrying the static metadata; or, the bitstream includes a metadata header carrying the static metadata; or, the bitstream includes a metadata sub-bitstream corresponding to the metadata header carrying the static metadata.

[0019] Optionally, the method further includes: encoding dynamic metadata of the target image into the bitstream, wherein the dynamic metadata includes color gamut volume change data of the target image.

[0020] Optionally, the bitstream includes an image header carrying the dynamic metadata; or, the bitstream includes a metadata header carrying the dynamic metadata; or, the bitstream includes a metadata sub-bitstream corresponding to the metadata header carrying the dynamic metadata.

[0021] In summary, when encoding the target image, the first identifier indicating the photoelectric conversion function that matches the target image can also be encoded into the bitstream to simultaneously meet the display requirements of SDR and HDR images at the decoding end. This allows the decoder to render and display the reconstructed image using the corresponding photoelectric conversion function after decoding the first identifier.

[0022] In a second aspect, an image decoding method is provided, comprising: parsing a first identifier from a bitstream, the first identifier indicating a photoelectric conversion function matching a target image, the target image including a high dynamic range (HDR) image and / or a standard dynamic range (SDR) image; decoding the bitstream to obtain a reconstructed image of the target image; and performing photoelectric conversion on the reconstructed image based on the first identifier.

[0023] Optionally, the bitstream includes an image header, in which the first identifier is carried.

[0024] Optionally, the method further includes: parsing a second identifier from the bitstream, the second identifier indicating the chromaticity coordinates of each primary color of the target image; parsing a third identifier from the bitstream, the third identifier indicating the mapping range of the target image; and parsing a fourth identifier from the bitstream, the fourth identifier indicating the coefficients of the transformation matrix used for color space conversion of the target image.

[0025] Optionally, before performing photoelectric conversion on the reconstructed image based on the first identifier, the method further includes performing color space inverse conversion on the reconstructed image based on the second identifier, the third identifier, and the fourth identifier.

[0026] Optionally, the bitstream includes an image header, in which the second identifier, the third identifier, and the fourth identifier are carried.

[0027] Optionally, the method further includes: parsing a fifth identifier from the bitstream, the fifth identifier indicating the offset of the sampling positions of the principal and secondary components of the target image.

[0028] Optionally, parsing the bitstream to obtain the reconstructed image of the target image includes: parsing the bitstream to obtain the reconstructed image of the principal component and the reconstructed image of the secondary component of the target image; and upsampling the reconstructed image of the principal component and the reconstructed image of the secondary component based on the fifth identifier to obtain the reconstructed image.

[0029] Optionally, the bitstream includes an image header, and the fifth identifier is carried in the image header.

[0030] Optionally, the method further includes: parsing static metadata of the target image from the bitstream, the static metadata including source device information and image display information, the source device information indicating the color information of the main monitor that generated the target image, and the image display information indicating the display brightness of the target image.

[0031] Optionally, the method further includes: performing tone mapping on the reconstructed image based on the static metadata.

[0032] Optionally, the source device information includes at least one of the chromaticity coordinates of the three primary colors of the main monitor, the chromaticity coordinates of standard white light, the maximum display brightness, and the minimum display brightness.

[0033] Optionally, the image display information includes at least one of the maximum display brightness and the maximum average display brightness of the target image.

[0034] Optionally, the bitstream includes an image header carrying the static metadata; or, the bitstream includes a metadata header carrying the static metadata; or, the bitstream includes a metadata sub-bitstream corresponding to the metadata header carrying the static metadata.

[0035] Optionally, the method further includes: parsing dynamic metadata of the target image from the bitstream, the dynamic metadata including color gamut volume change data of the target image.

[0036] Optionally, the method further includes: performing tone mapping on the reconstructed image based on the static metadata and the dynamic metadata.

[0037] Optionally, the bitstream includes an image header carrying the dynamic metadata; or, the bitstream includes a metadata header carrying the dynamic metadata; or, the bitstream includes a metadata sub-bitstream corresponding to the metadata header carrying the dynamic metadata.

[0038] In summary, when decoding the bitstream, the photoelectric conversion function matching the target image can be determined based on the decoded first identifier. This photoelectric conversion function can then be used to accurately render and display the reconstructed image of the target image, thus simultaneously meeting the display requirements of both SDR and HDR images at the decoding end and ensuring the accuracy of the displayed target image.

[0039] Thirdly, an encoding device is provided, which has the function of implementing the image encoding method behavior described in the first aspect above. The encoding device includes at least one module for implementing the image encoding method provided in the first aspect above.

[0040] Fourthly, a decoding device is provided, which has the function of implementing the image decoding method behavior described in the second aspect above. The decoding device includes at least one module for implementing the image decoding method provided in the second aspect above.

[0041] Fifthly, an encoding device is provided, the encoding device comprising: a processor coupled to a memory for storing programs or instructions, wherein when the programs or instructions are executed by the processor, the encoding device performs the image encoding method described in the first aspect.

[0042] In a sixth aspect, a decoding device is provided, the decoding device comprising: a processor coupled to a memory for storing programs or instructions, wherein when the program or instructions are executed by the processor, the decoding device performs the image decoding method described in the second aspect above.

[0043] In a seventh aspect, a coding and decoding system is provided, the coding and decoding system comprising the encoding device described in the fifth aspect and / or the decoding device described in the sixth aspect.

[0044] Eighthly, a computer-readable storage medium is provided, including program code that, when executed on a computer, causes the computer to perform the method described in the first aspect.

[0045] A ninth aspect provides a computer-readable storage medium including program code that, when executed on a computer, causes the computer to perform the method described in the second aspect above.

[0046] In a tenth aspect, a computer program product is provided, including instructions that, when executed on a computer, cause the computer to perform the method described in the first aspect.

[0047] Eleventhly, a computer program product is provided, including instructions that, when executed on a computer, cause the computer to perform the method described in the second aspect above.

[0048] In a twelfth aspect, a computer-readable storage medium is provided, on which a bitstream obtained according to the method described in the first aspect is stored.

[0049] In a thirteenth aspect, a device for storing a bitstream is provided, comprising at least one storage medium and a communication interface; the communication interface is used to receive or transmit the bitstream; the at least one storage medium is used to store the bitstream; the bitstream is encoded by an encoder according to the image encoding method described in the first aspect above.

[0050] In a fourteenth aspect, a method for storing a bitstream is provided, comprising: receiving the bitstream through a communication interface; and storing the bitstream in one or more storage media, wherein the bitstream is encoded by an encoder according to the image encoding method described in the first aspect above.

[0051] In a fifteenth aspect, a system for distributing a bitstream is provided, comprising at least one storage medium and a video streaming device; the at least one storage medium is used to store the bitstream, which is encoded by an encoder according to the image encoding method described in the first aspect; the video streaming device is used to send the bitstream in the at least one storage medium to the decoder in response to a request from the decoder.

[0052] In a sixteenth aspect, a method for distributing a bitstream is provided, comprising: receiving a first request; in response to the first request, selecting a bitstream from at least one storage medium; and sending the bitstream to a destination device; wherein the at least one storage medium is used to store the bitstream, the bitstream being encoded by an encoder according to the image encoding method described in the first aspect above.

[0053] In a seventeenth aspect, a system for processing bitstreams is provided, comprising an image source device, an encoder, one or more storage media, and a destination device; the image source device is used to provide image data; the encoder is used to acquire the image data from the image source device through an interface and encode the image data to obtain one or more bitstreams, the bitstreams being encoded by the encoder according to the image encoding method described in the first aspect; the encoder is used to store the one or more bitstreams in one or more storage media; or the encoder is used to encapsulate the one or more bitstreams to obtain a transmission bitstream; the encoder is used to transmit the transmission bitstream to the destination device via a communication link or communication network; the destination device is used to decapsulate the transmission bitstream to obtain the one or more bitstreams; and the destination device is used to decode the one or more bitstreams to obtain decoded data.

[0054] The technical effects achieved by the second to seventeenth aspects mentioned above are similar to those achieved by the corresponding technical means in the first aspect, and will not be repeated here. Attached Figure Description

[0055] Figure 1 is a schematic diagram of an implementation environment provided in an embodiment of this application;

[0056] Figure 2 is a schematic diagram of an AI coding model provided in an embodiment of this application;

[0057] Figure 3 is a schematic diagram of an AI decoding model provided in an embodiment of this application;

[0058] Figure 4 is a schematic diagram of a JPEG-AI encoding model provided in an embodiment of this application;

[0059] Figure 5 is a schematic diagram of a JPEG-AI decoding model provided in an embodiment of this application;

[0060] Figure 6 is a flowchart illustrating an image encoding method provided in an embodiment of this application;

[0061] Figure 7 is a schematic diagram of a bitstream structure provided in an embodiment of this application;

[0062] Figure 8 is a schematic diagram of a colorimetric sampling position provided in an embodiment of this application;

[0063] Figure 9 is a schematic diagram of another bitstream structure provided in an embodiment of this application;

[0064] Figure 10 is a flowchart illustrating an image decoding method provided in an embodiment of this application;

[0065] Figure 11 is a schematic diagram of the structure of an encoding device provided in an embodiment of this application;

[0066] Figure 12 is a schematic diagram of the structure of a decoding device provided in an embodiment of this application. Detailed Implementation

[0067] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0068] To facilitate understanding, before providing a detailed explanation of the image encoding and image decoding methods provided in the embodiments of this application, the terms, implementation environment, and application scenarios involved in the embodiments of this application will be introduced first.

[0069] First, the terms used in the embodiments of this application will be explained.

[0070] 1. High Dynamic Range (HDR)

[0071] Dynamic range is used in many fields to represent the ratio of the maximum to the minimum value of a variable. In digital images, dynamic range represents the ratio between the maximum and minimum grayscale values ​​within the displayable range of an image. Currently, in most color digital images, each of the R, G, and B channels uses one byte (8 bits) for storage. This means that each channel represents a range of 0 to 255 grayscale levels; this 0 to 255 is the image's dynamic range. In the real world, a dynamic range of 10⁻³ to 10⁶ for the same scene is called high dynamic range (HDR); conversely, the dynamic range of ordinary images is low dynamic range (LDR). The imaging process of a digital camera is essentially a mapping from the high dynamic range of the real world to the low dynamic range of the photograph.

[0072] 2. Standard dynamic range photoelectric conversion function

[0073] Standard dynamic range (SDR) images correspond to high dynamic range (HDR) images. Traditional 8-bit images in formats like JPEG can be considered SDR images. Before the advent of cameras capable of capturing HDR images, traditional cameras could only record light information within a certain range by controlling exposure values. Since the maximum illuminance of a display device cannot match the brightness of the real world, and we view images through display devices, a photoelectric transfer function is needed. Early display devices were CRT monitors, and their photoelectric transfer function was the Gamma function. This photoelectric transfer function based on the Gamma function was defined in the BT.1886 standard published by the International Telecommunication Union (ITU-R) Radiocommunications Sector. After tone mapping using this conversion function and quantization into 8 bits, the resulting image is the traditional SDR image. In other words, SDR images and the aforementioned photoelectric transfer function perform well on traditional display devices (illuminance around 100 cd / m²).

[0074] 3. HDR photoelectric transfer function

[0075] However, with the upgrading of display devices, the illuminance range of display devices continues to increase. Existing consumer-grade HDR displays have an illuminance of 600 cd / m2, while high-end HDR displays can reach 2000 cd / m2, far exceeding the illuminance of SDR display devices. The photoelectric conversion function in the BT.1886 standard cannot well express the display performance of HDR display devices. Therefore, an improved photoelectric transfer function is needed to adapt to the upgrading of display devices. The idea of ​​the photoelectric transfer function comes from the mapping function in the tone mapping (TM) algorithm. Appropriately adjusting the mapping function results in the photoelectric transfer function (OETF).

[0076] Currently, there are three common photoelectric conversion functions: perception quantization (PQ) photoelectric conversion function, hybrid log-gamma (HLG) photoelectric conversion function, and scene luminance fidelity (SLF) photoelectric conversion function. These three photoelectric conversion functions are the conversion functions specified by the audio video coding standard (AVS).

[0077] Unlike the traditional "Gamma" transfer function, the PQ photoelectric transfer function is a perceptual quantization transfer function proposed based on the human eye's brightness perception model. The PQ photoelectric transfer function represents the conversion relationship between linear signal values ​​of image pixels and nonlinear signal values ​​in the PQ domain. The HLG photoelectric transfer function is an improvement on the traditional Gamma curve. Specifically, the HLG photoelectric transfer function uses the traditional Gamma curve in the lower segment and supplements it with a log curve in the higher segment. The HLG photoelectric transfer function represents the conversion relationship between linear signal values ​​of image pixels and nonlinear signal values ​​in the HLG domain, while satisfying the optical characteristics of the human eye.

[0078] 4. Dynamic range mapping in display

[0079] Dynamic range mapping (DLM) methods are primarily used for adapting front-end HDR signals to back-end HDR display devices. For example, if the front-end captures a 4000 nit light signal, while the back-end HDR display device (such as a television) only has an HDR display capability of 500 nits, then mapping the 4000 nit signal to the 500 nit device involves a high-to-low tone-mapping process. Another scenario involves a front-end capturing a 100 nit SDR signal, while the display receives a 2000 nit television signal. Improving the display of the 100 nit signal on the 2000 nit device involves a different low-to-high tone-mapping process.

[0080] Dynamic range mapping methods can be divided into static and dynamic methods. Static mapping uses a single data point to perform an overall tone mapping process based on the same video content or hard drive content; the processing curve is usually the same. The advantage of this method is that it carries less information and has a simpler processing flow. The disadvantage is that using the same curve for tone mapping in every scene can lead to information loss in some scenes. For example, if the curve focuses on protecting bright areas, some details will be lost in extremely dark scenes, affecting the viewing experience. Dynamic mapping, on the other hand, dynamically adjusts the curve based on specific regions, each scene, or each frame. The advantage of this method is that it allows for different curve processing for specific regions, scenes, or frames, resulting in better processing results. However, it requires each frame or scene to carry relevant scene information, resulting in a larger amount of information being processed.

[0081] 5. ITU-T T.35 Terminal Identifier Code Allocation

[0082] ITU-T T.35 (hereinafter referred to as T.35) is the ITU-T standard T.35, "Procedure for the Allocation of Non-Standard Extension Codes Defined by the ITU." It has three components: a country code, a terminal provider code, and a terminal provider-oriented code. The country code is defined in Annexes A and B of ITU-T T.35, and the terminal provider code is assigned by the ITU management authority. Each terminal provider assigned a provider code by the management authority manages its own terminal provider-oriented code.

[0083] T.35 can be used in various contexts, such as SEI messages in video codecs. The data carried in T.35 messages may include HDR dynamic metadata, HDR static metadata, HDR display mapping information, etc. However, the information transmitted via T.35 is not limited to video; it can be information of any nature.

[0084] 6. Metadata

[0085] Recording video, scene, or frame contains key information about the image, such as average, maximum, and minimum values ​​within the scene. HDR static metadata, also known as fixed metadata, uses the same metadata throughout the entire video to control the color and detail of each frame. Conversely, HDR dynamic metadata, also known as variable metadata, uses different metadata for each frame of the video to control color and detail.

[0086] Secondly, the implementation environment involved in the embodiments of this application will be introduced.

[0087] Please refer to Figure 1, which is a schematic diagram of an implementation environment provided in an embodiment of this application. This implementation environment includes a source device 10, a destination device 20, a link 30, and a storage device 40. The source device 10 can generate encoded images. Therefore, the source device 10 can also be referred to as an image encoding device or encoding end. The destination device 20 can decode the encoded images generated by the source device 10. Therefore, the destination device 20 can also be referred to as an image decoding device or decoding end. The link 30 can receive the encoded images generated by the source device 10 and can transmit the encoded images to the destination device 20. The storage device 40 can receive the encoded images generated by the source device 10 and can store the encoded images. Under such conditions, the destination device 20 can directly obtain the encoded images from the storage device 40. Alternatively, the storage device 40 can correspond to a file server or another intermediate storage device that can store the encoded images generated by the source device 10. Under such conditions, the destination device 20 can stream or download the encoded images stored in the storage device 40.

[0088] Both source device 10 and destination device 20 may include one or more processors and memory coupled to the one or more processors. This memory may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or any other media that can be used to store desired program code in the form of computer-accessible instructions or data structures. For example, both source device 10 and destination device 20 may include mobile phones, smartphones, personal digital assistants (PDAs), wearable devices, pocket PCs (PPCs), tablets, smart car systems, smart TVs, smart speakers, desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, handsets such as so-called "smart" phones, televisions, cameras, display devices, digital media players, video game consoles, in-vehicle computers, or the like.

[0089] Link 30 may include one or more media or devices capable of transmitting encoded images from source device 10 to destination device 20. In one possible implementation, link 30 may include one or more communication media enabling source device 10 to directly transmit encoded images to destination device 20 in real time. In this embodiment, source device 10 may modulate the encoded image based on a communication standard, such as a wireless communication protocol, and transmit the modulated image to destination device 20. The one or more communication media may include wireless and / or wired communication media, such as radio frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media may form part of a packet-based network, such as a local area network, wide area network, or global network (e.g., the Internet). The one or more communication media may include routers, switches, base stations, or other devices facilitating communication from source device 10 to destination device 20, etc., which are not specifically limited in this embodiment.

[0090] In one possible implementation, storage device 40 can store the received encoded image sent by source device 10, and destination device 20 can directly retrieve the encoded image from storage device 40. Under such conditions, storage device 40 can include any of a variety of distributed or locally accessed data storage media. For example, any of these distributed or locally accessed data storage media can be a hard disk drive, Blu-ray disc, digital versatile disc (DVD), compact disc read-only memory (CD-ROM), flash memory, volatile or non-volatile memory, or any other suitable digital storage media for storing encoded images.

[0091] In one possible implementation, storage device 40 may correspond to a file server or another intermediate storage device capable of storing the encoded images generated by source device 10, and destination device 20 may stream or download the images stored by storage device 40. The file server may be any type of server capable of storing and sending encoded images to destination device 20. In one possible implementation, the file server may include a web server, a file transfer protocol (FTP) server, a network attached storage (NAS) device, or a local disk drive, etc. Destination device 20 may acquire the encoded images via any standard data connection (including an Internet connection). Any standard data connection may include a wireless channel (e.g., Wi-Fi connection), a wired connection (e.g., digital subscriber line (DSL), cable modem, etc.), or a combination of both suitable for acquiring encoded images stored on a file server. The transmission of encoded images from storage device 40 may be streaming, downloading, or a combination of both.

[0092] The implementation environment shown in Figure 1 is only one possible implementation method, and the technology of this application embodiment can be applied not only to the source device 10 that can encode images and the destination device 20 that can decode encoded images shown in Figure 1, but also to other devices that can encode images and decode encoded images. This application embodiment does not specifically limit this.

[0093] In the implementation environment shown in Figure 1, source device 10 includes a data source 120, an encoder 100, and an output interface 140. In some embodiments, output interface 140 may include a modem / demodulator and / or a transmitter, wherein the transmitter may also be referred to as a transmitter. Data source 120 may include an image capture device (e.g., a camera, etc.), an archive containing previously captured images, a feed interface for receiving images from an image content provider, and / or a computer graphics system for generating images, or a combination of these sources of images.

[0094] Data source 120 can send images to encoder 100, which can encode the received images sent by data source 120 to obtain encoded images. The encoder can then send the encoded images to an output interface. In some embodiments, source device 10 directly sends the encoded images to destination device 20 via output interface 140. In other embodiments, the encoded images can also be stored on storage device 40 for later retrieval by destination device 20 for decoding and / or display.

[0095] In the implementation environment shown in Figure 1, the destination device 20 includes an input interface 240, a decoder 200, and a display device 220. In some embodiments, the input interface 240 includes a receiver and / or a modem. The input interface 240 may receive encoded images via link 30 and / or from storage device 40, and then send them to the decoder 200, which may decode the received encoded images to obtain decoded images. The decoder may send the decoded images to the display device 220. The display device 220 may be integrated with the destination device 20 or may be external to the destination device 20. Generally, the display device 220 displays the decoded images. The display device 220 may be any type of display device, for example, a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or other types of display devices.

[0096] Although not shown in Figure 1, in some respects, encoder 100 and decoder 200 may be integrated with each other and may include appropriate multiplexer-demultiplexer (MUX-DEMUX) units or other hardware and software for encoding both audio and video in a common data stream or separate data streams. In some embodiments, the MUX-DEMUX unit may conform to the ITU H.223 multiplexer protocol, or other protocols such as User Datagram Protocol (UDP), if applicable.

[0097] Encoder 100 and decoder 200 may each be any of the following circuits: one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, or any combination thereof. If the techniques of the embodiments of this application are implemented in part in software, the apparatus may store instructions for software in a suitable non-volatile computer-readable storage medium, and the instructions may be executed in hardware using one or more processors to implement the techniques of the embodiments of this application. Any of the foregoing (including hardware, software, combinations of hardware and software, etc.) may be considered as one or more processors. Each of encoder 100 and decoder 200 may be included in one or more encoders or decoders, either of which may be integrated as part of a combined encoder / decoder (encoder-decoder) in the respective apparatus.

[0098] In this application embodiment, encoder 100 may be generally referred to as an apparatus that “signals” or “sends” certain information to, for example, decoder 200. The terms “signals” or “sends” may generally refer to the transmission of syntax elements and / or other data for decoding a compressed image. This transmission may occur in real-time or nearly real-time. Alternatively, this communication may occur after a period of time, for example, during encoding when syntax elements are stored in a computer-readable storage medium in a encoded bitstream, and the decoding apparatus may then retrieve the syntax elements at any time after they have been stored in this medium.

[0099] It should be noted that the application scenarios and implementation environments described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, as the application scenarios and implementation environments evolve, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0100] Finally, the application scenarios involved in the embodiments of this application will be introduced.

[0101] Encoding and decoding technology refers to a technique that utilizes data characteristics such as spatial redundancy, visual redundancy, and statistical redundancy to represent the original signal data with fewer bits, either lossily or without loss. It can achieve the effective transmission and storage of information such as images, videos, and audio, and plays an important role in the current media era where the types and amounts of information transmitted or stored are increasing.

[0102] Taking image compression as an example, image compression is divided into lossy compression and lossless compression. Lossy compression achieves a higher compression ratio at the cost of some reduction in image quality, while lossless compression does not cause any loss of image details. Traditional image / video compression algorithms have undergone decades of development, resulting in mature compression standards such as High Efficiency Video Coding (HEVC) and Versatile Video Coding (VVC). Traditional image / video compression algorithms have advantages such as comprehensive support functions, good versatility, and good hardware support, and have been widely used. However, they have also encountered bottlenecks in improving coding efficiency.

[0103] With deep learning outperforming traditional algorithms in numerous computer vision tasks such as image recognition and object detection, more and more researchers are exploring image / video compression methods based on deep learning. Unlike traditional image algorithms that manually design and optimize image encoding, quantization, and entropy coding separately, AI image compression algorithms optimize each module (encoding network, entropy estimation network, decoding network, etc.) as a whole, resulting in better compression performance. However, deep learning is a resource-intensive algorithm, requiring significant computational resources and consuming substantial amounts of memory. Despite increasing computing resources, optimizing the training and inference of deep learning networks remains crucial for model deployment. In particular, as more and more models move from servers to resource-constrained devices such as smartphones and embedded devices, how to deploy complex models on resource-limited devices is a problem that current deep learning technology must solve.

[0104] Deep learning-based image compression methods typically involve AI compression models and JPEG-AI compression models. For ease of understanding, a brief introduction to the encoding and decoding processes of these two compression models will be provided below.

[0105] (1) AI compression model

[0106] Please refer to Figure 2, which is a schematic diagram of an AI encoding model. This AI encoding model, also known as an encoder, is applied at the encoding end. During encoding, the image to be encoded, x, is input into the encoding network to obtain a feature map y. The feature map y is then input into the super-encoding network to obtain a super-prior feature z. Entropy encoding is performed on the super-prior feature z according to a specified probability distribution to incorporate it into the bitstream. Additionally, the super-prior feature z is input into the super-decoding network to obtain a prior feature p and variance σ. Based on the prior feature p and the feature map y, the mean μ is obtained through the context network. Then, based on this probability distribution N(μ, σ), entropy encoding is performed on the feature map y to incorporate it into the bitstream.

[0107] Among them, the bit sequence obtained by entropy encoding the prior feature z is a partial bit sequence included in the bit stream. This partial bit sequence can be called the prior bit stream, denoted as bs. z The bit sequence obtained by entropy encoding the feature map y is a partial bit sequence included in the bitstream. This partial bit sequence can be called the image bitstream, denoted as bs. y .

[0108] In some embodiments, the context network includes a context model and a probability distribution estimation model. In this case, the feature map y can be input into the context model to obtain the context feature φ. Combining the prior feature p and the context feature φ, the mean μ is estimated by the probability distribution estimation model. Of course, this is just an example, and other methods can be used in practical applications. This application embodiment does not limit this. Moreover, the above description uses entropy encoding as an example. In practical applications, other encoding methods can also be used, and this application embodiment does not limit this either.

[0109] Please refer to Figure 3, which is a schematic diagram of an AI decoding model. This AI decoding model, also known as a decoder, is applied at the decoding end. During decoding, it processes the super-prior bitstream bs included in the bitstream according to a specified probability distribution. z Entropy decoding is performed to obtain the prior features. This prior feature Input is fed into the super-decoding network to obtain prior features. And variance σ. Edge information is determined from the decoded partial image features, combined with prior features. And edge information, the mean μ is obtained through the context network, and based on the probability distribution N(μ, σ), the image bitstream bs included in the bitstream is obtained. y Medium entropy decoding of feature maps The feature map Input to the decoding network to obtain the reconstructed image

[0110] In some embodiments, the context network includes a context model and a probability distribution estimation model, in which case, based on prior features... The probability distribution estimation model can estimate the mean μ corresponding to some image features. Based on the probability distribution N(μ, σ) of these image features, the image bitstream bs included in the bitstream can be calculated. y The image features are decoded using medium entropy decoding. For other image features, edge information is determined from the decoded image features, and this edge information is input into the context model to obtain context features. Combining prior features and context features The mean μ corresponding to other image features is estimated using a probability distribution estimation model. Of course, this is just one example; other methods can be used in practical applications, and this application does not limit this approach. Furthermore, the above description uses entropy decoding as an example; other decoding methods corresponding to the encoding method can also be used in practical applications, and this application also does not limit this approach.

[0111] It should be noted that if the probability distribution estimation network described above uses a Gaussian model (such as a single Gaussian model or a Gaussian mixture model) for modeling, the estimated probability distribution includes the mean and variance. If the probability distribution estimation network uses a Laplace distribution model for modeling, the estimated probability distribution includes the location parameter and the scale parameter. If the probability distribution estimation network uses a logistic distribution model for modeling, the estimated probability distribution includes the mean and the scale parameter. The above explanation uses a Gaussian model as an example.

[0112] Furthermore, the aforementioned super-decoding network includes a super-decoding prediction network and a super-scale decoding network. The super-decoding prediction network is used to determine prior features, and the super-scale decoding network is used to determine the variance. That is, the super-prior features are input into the super-decoding prediction network to obtain the prior features, and the super-prior features are input into the super-scale decoding network to obtain the variance.

[0113] (2) JPEG-AI compression model

[0114] The JPEG-AI compression model is based on a color component separation autoencoder structure. It achieves image encoding and decoding by separately encoding and decoding each of the separated components of the image. Furthermore, the basic structure of the JPEG-AI compression model is the same as the AI ​​compression model mentioned above; however, the image input components of the JPEG-AI compression model are YUV components, and the Y and UV components are encoded and decoded separately. Specifically, during encoding, the Y component is encoded alone using the architecture shown in Figure 2, while the UV components are encoded by combining the data from the Y component. During decoding, the Y component is decoded alone using the architecture shown in Figure 3, while the UV components are decoded by combining the data from the Y component.

[0115] Please refer to Figure 4, which is a schematic diagram of a JPEG-AI encoding model. This JPEG-AI encoding model, also known as an encoder, is applied at the encoding end. During encoding, the image x in RGB (a color mode where R represents red, G represents green, and B represents blue) format is converted to YUV (a color encoding method where Y represents luminance, i.e., grayscale value, and U and V represent chrominance) format, thus obtaining the image x of the Y component. Y Image x of UV components UV The image of the Y component x Y The input is fed into the encoding network to obtain the feature map y of the Y component. Y The feature map y of the Y component Y The input is fed into a supercoding network to obtain the super-prior features z of the Y component. Y Based on the specified probability distribution, the prior features z of the Y component Y Entropy encoding is performed to obtain the prior features z of the Y component. Y Encode into the bitstream. Then, use the prior features z of the Y component. Y The input is fed into the super-decoding network to obtain the prior features p of the Y component. Y and variance σ Y Based on prior features p Y and variance σ Y The mean μ is obtained through the context network. Y Furthermore, based on the probability distribution N(μ) of the Y component Y , σ Y ), and the feature map y of the Y component Y Entropy encoding is performed to encode the feature map y of the Y component. Y Encode into the bitstream.

[0116] Additionally, the image of the Y component x Y Auxiliary image x' converted to UV components after downsampling UV The image of UV components x UVAnd auxiliary image x' of UV components UV Input to the encoding network to obtain feature maps y of the UV components UV The feature map y of the UV components UV Input is fed into a supercoding network to obtain the super-prior features z of the UV components. UV Based on the specified probability distribution, the prior features z of the UV components UV Entropy encoding is performed to encode the prior features z of the UV components. UV Encode into the bitstream. Incorporate the prior features z of the UV components. UV Input is fed into the super-decoding network to obtain prior features p of the UV components. UV and variance σ UV Prior features p based on UV components UV and feature map y UV The mean μ of the UV components is obtained through the context network. UV Furthermore, based on the probability distribution N(μ) of the UV components UV , σ UV ), and the characteristic map y of the UV component UV Entropy encoding is performed to encode the feature map y of the UV components. UV Encode into the bitstream.

[0117] Among them, the prior features z of the Y component Y The bit sequence obtained by entropy coding is a partial bit sequence included in the bit stream. This partial bit sequence can be called the super-prior bit stream bs of the Y component. zY The characteristic map y of the Y component Y The bit sequence obtained by entropy coding is a partial bit sequence included in the bitstream; this partial bit sequence can be called the image bitstream bs of the Y component. yY The prior features z of the UV components UV The bit sequence obtained by entropy coding is a partial bit sequence included in the bitstream. This partial bit sequence can be called the super-prior bitstream bs of the UV components. zUV Characteristic map y of UV components UV The bit sequence obtained by entropy coding is a partial bit sequence included in the bitstream; this partial bit sequence can be called the image bitstream bs of the UV components. yUV .

[0118] Please refer to Figure 5, which is a schematic diagram of a JPEG-AI decoding model. This JPEG-AI decoding model, also known as a decoder, is applied at the decoding end. During decoding, it processes the Y component of the bitstream according to a specified probability distribution, representing the super-prior bitstream bs. zY Entropy decoding is performed to obtain the prior features of the Y component. The prior features of the Y component The input is fed into the super-decoding network to obtain the prior features of the Y component. and variance σ Y Edge information is determined from the decoded partial image features, combined with prior features of the Y component. The mean μ of the Y component is obtained through the context network, along with edge information. Y The probability distribution N(μ) based on the Y component Y , σ Y The feature map of the Y component is entropy decoded from the image bitstream containing the Y component in the bitstream. Feature map of Y component The image is input into a decoding network to obtain the reconstructed Y component.

[0119] In addition, the feature map y of the Y component Y Auxiliary features y' converted to UV components after downsampling UV Then, based on the specified probability distribution, the super-prior bitstream bs of the UV components included in the bitstream is... zUV Entropy decoding is performed to obtain the prior features of the UV components. Super-prior features of UV components The input is fed into the super-decoding network to obtain prior features of the UV components. and variance σ UV Edge information is determined from the features of the decoded partial image, combined with prior features of the UV components. The mean μ of the UV components is obtained through the context network, along with edge information. UV The probability distribution N(μ) based on UV components UV , σ UV The feature maps of the UV components are entropy decoded from the image bitstream containing the UV components in the bitstream. Feature map of UV components Auxiliary features y′ of UV components UV The image is input into a decoding network to obtain the reconstructed UV components.

[0120] Subsequently, the reconstructed Y component image is processed through an Inter-Channel Correlation Information (ICCI) network. Images of UV components The process involves processing to obtain a reconstructed image in YUV format. Finally, a color space inverse conversion is performed to convert the YUV format reconstructed image to RGB format, resulting in the final reconstructed image.

[0121] The above describes the general process of encoding and decoding in the JPEG-AI compression model. For more details, please refer to the explanation of the AI ​​compression model above. It will not be repeated here.

[0122] In other words, before encoding an image, the JPEG-AI compression model requires color space conversion of the target image and downsampling of its component images. After decoding and reconstructing the reconstructed component images of the target image, it upsamples these component images and performs an inverse color space conversion. However, the coefficients of the transformation matrix used in the JPEG AI decoding model during the inverse color space conversion are suitable for SDR images, thus failing to reconstruct accurate HDR images. Furthermore, when upsampling the reconstructed component images, the offset of the chroma information relative to the luminance information sampling position differs from that of the downsampling at the decoding end, also preventing the reconstruction of accurate HDR images. Additionally, for the reconstructed images, the display devices at the decoding end typically use the photoelectric conversion function corresponding to SDR for tone mapping, which cannot accurately map and display the reconstructed HDR images.

[0123] Based on this, in order to realize the encoding, decoding, transmission and display of HDR images, this application provides an image encoding method and an image decoding method, which incorporates image information and HDR metadata information during encoding to guide the accurate reconstruction and display of HDR images, so that the decoding end can combine these image information to reconstruct an accurate image; moreover, combined with the HDR metadata information, the HDR image can be effectively displayed.

[0124] Referring to Figure 6, this application provides an image encoding method, which is applied to an encoder and includes the following steps.

[0125] Step 601: Encode the target image into a bitstream.

[0126] The specific implementation method of encoding the target image into the bitstream can be found in the relevant explanation of the encoding and decoding process of the JPEG AI compression model mentioned above, and will not be repeated here.

[0127] Step 602: Encode a first identifier into the bitstream, wherein the first identifier indicates a photoelectric conversion function that matches the target image, which includes an HDR image and / or an SDR image.

[0128] The photoelectric conversion function is used to convert digital signals into analog signals that can drive the physical output of the display so that the image can be displayed correctly on the display. When the target image is an HDR image, the first identifier indicates the photoelectric conversion function corresponding to the HDR image, such as the PQ photoelectric conversion function, HLG photoelectric conversion function, etc.; when the target image is an SDR image, the first identifier indicates the photoelectric conversion function corresponding to the SDR image, such as the photoelectric transfer function based on the "Gamma" function.

[0129] In some embodiments, when the first identifier is encoded into the bitstream, the first identifier can be encoded into the bitstream in the form of an 8-bit unsigned integer, that is, the first identifier corresponds to 0-255 values; of course, the first identifier can also be encoded into the bitstream in other ways, and this application embodiment does not limit this.

[0130] When the first identifier is encoded into the bitstream in the form of an 8-bit unsigned integer, Table 1 below provides an exemplary correspondence to determine the photoelectric conversion function indicated by the first identifier based on this correspondence.

[0131] Table 1

[0132] As shown in Table 1 above, when the target image is an SDR image, its first identifier can be 13, and its corresponding photoelectric conversion function is the photoelectric conversion function defined by the IEC 61966-2-1 standard. This photoelectric function supports photoelectric conversion of SDR images. When the target image is an HDR image, the first identifier in the encoded bitstream is 16 or 18. 16 indicates the PQ photoelectric conversion function, and 18 indicates the SLG photoelectric conversion function. The specific function expressions can be found in the middle column of Table 1. o The linear signal value before transformation is normalized to [0, 1], L c V represents the linear signal value before conversion, with a range of [0, 12], and V represents the nonlinear signal value after conversion, with a range of [0, 1]. The corresponding photoelectric conversion functions are also recorded and explained in relevant standards, and can be found in the standard documents listed in the right column of Table 1.

[0133] It should be noted that Table 1 above is merely an example. In specific applications, the photoelectric conversion function corresponding to the identified values ​​in the table above can also be adjusted according to the relevant standards for image / video encoding and decoding. This application does not impose any restrictions on this.

[0134] Referring to Figure 7, which is a schematic diagram of the bitstream structure defined by JPEG AI, the bitstream structure includes a start of codestream marker (SOC), a picture header marker (PIH) and its corresponding picture header, a tools header marker (TOH) and its corresponding (encoding / decoding) tool header, a start of quality map marker (SOQ) and its corresponding Q-stream, multiple image bitstreams, and an end of codestream marker (EOC).

[0135] The multiple image streams include a coded stream of super-prior features (Z-stream), a coded stream of primary component residuals (RY-stream), and a coded stream of secondary component residuals (RUV-stream). Each image stream corresponds to a start marker. The coded stream of super-prior features follows the start of the z-stream marker (SOZ), the coded stream of primary component residuals follows the start of the primary component residual stream marker (SORP), and the coded stream of secondary component residuals follows the start of the secondary component residual stream marker (SORS).

[0136] Based on this bitstream structure, step 602 can be implemented by: encoding the first identifier into the aforementioned picture header.

[0137] Therefore, regardless of whether the target image being encoded is an SDR image or an HDR image, a photoelectric conversion function matching the target image can be determined. During encoding, a first identifier indicating the photoelectric conversion function is also encoded into the bitstream, so that the decoding end can display the reconstructed image corresponding to the target image based on the first identifier and using the matching photoelectric conversion function.

[0138] In some embodiments, before encoding the target image, if the target image is in RGB format, a color space conversion is required to transform the RGB image into a YUV image. However, JPEG AI's color space conversion only supports two modes: the preset BT.709 conversion matrix and a user-defined conversion matrix. BT.709 is the default color space for SDR images. However, other ITU-R RGB to YUV conversion standards, such as ITU-R BT.2020 and ITU-R BT.2100, specify HDR television standards and inherit and use the color space specified in the BT.2020 standard. Therefore, when encoding and decoding HDR images, both the encoder and decoder need to support BT.2020 color space conversion (which is consistent with BT.2100 color space).

[0139] Based on this, when the decoding end encodes the target image, it can also encode the information related to the color space conversion matrix used by the encoding end to perform color space conversion on the target image into the bitstream, so as to guide the decoding end to convert the YUV format reconstructed image into the RGB format reconstructed image based on the information related to the color space conversion matrix.

[0140] In one possible implementation, the image encoding method provided in this application further includes: encoding a second identifier into the bitstream, the second identifier indicating the chromaticity coordinates of each primary color of the target image; encoding a third identifier into the bitstream, the third identifier indicating the pixel mapping range of the target image; and encoding a fourth identifier into the bitstream, the fourth identifier indicating the coefficients of the transformation matrix used for color space conversion of the target image.

[0141] In this case, the encoder performs color space conversion on the target image based on the second, third, and fourth identifiers before encoding the target image into the bitstream.

[0142] In other words, the encoder uses the corresponding color space conversion matrix to convert the color space of the target image according to its type (i.e., HDR image or SDR image), and then encodes the converted target image into the bitstream; at the same time, it encodes the bitstream with identifiers related to the color space conversion matrix, namely the second identifier, the third identifier, and the fourth identifier.

[0143] Similarly, when the bitstream includes an image header, the aforementioned second, third, and fourth identifiers can also be encoded into that image header.

[0144] In some embodiments, the second identifier used to indicate the chromaticity coordinates of each primary color of the target image can also be encoded into the bitstream in the form of an 8-bit unsigned integer, corresponding to 0 to 255 values. Taking the chromaticity of CIE 1931 as specified by ISO / CIE 11664-1, which includes x and y coordinates, as an example, Table 2 below shows the method for determining the corresponding chromaticity coordinates based on the second identifier. That is, Table 2 provides an exemplary correspondence to find the chromaticity coordinates corresponding to red, green, blue, and standard white light based on the second identifier.

[0145] Table 2

[0146] As shown in Table 2 above, when the target image is an HDR image, the second identifier in the encoded bitstream is 9, the green coordinate corresponding to chroma x is 0.170, the green coordinate corresponding to chroma y is 0.797; the blue coordinate corresponding to chroma x is 0.131, the blue coordinate corresponding to chroma y is 0.046; the red coordinate corresponding to chroma x is 0.708, the red coordinate corresponding to chroma y is 0.292; the standard white light coordinate corresponding to chroma x is 0.3127, and the standard white light coordinate corresponding to chroma y is 0.3290. The above information regarding HDR images and chroma primary colors is covered in both the ITU-R BT.2020-2 and ITU-R BT.2100-2 standards, and can also be found in the ITU-R BT.2020-2 and ITU-R BT.2100-2 standard documents.

[0147] In some embodiments, the third identifier used to indicate the pixel mapping range of the target image can be 0 or 1. When the input signal is an RGB image, the third identifier is 1 to indicate that all pixels in the image are mapped within the full range; when the input signal is a luminance and chrominance image (such as a YUV format image), if the black level and range of the luminance and chrominance signals are not specified, the third identifier can be 0 to indicate that all pixels in the image can be mapped within a limited range.

[0148] Optionally, for YUV format images, the identifier value of the third identifier mentioned above can also be recorded as 1 to perform full-range mapping of all pixels in the image. This application embodiment does not limit this.

[0149] In some embodiments, when the fourth identifier, which indicates the coefficients of the transformation matrix used for color space conversion of the target image, is encoded into the bitstream, the fourth identifier may also be encoded into the bitstream as an unsigned integer, corresponding to 0 to 255 values. Taking the chromaticity of CIE 1931 as specified in ISO / CIE 11664-1, which includes x and y coordinates, as an example, Table 3 below gives an exemplary correspondence to determine the fourth identifier and the coefficients of the transformation matrix based on this correspondence.

[0150] Table 3

[0151] As shown in Table 3 above, when the target image is an HDR image, the fourth identifier in the encoded bitstream is 9, and the corresponding matrix coefficient is K. R K is 0.2627. B The value is 0.0593. Furthermore, this matrix coefficient is mentioned in both ITU-R BT.2020-2 and ITU-R BT.2100-2 standards, and can be found in the ITU-R BT.2020-2 and ITU-R BT.2100-2 standard documents.

[0152] In this embodiment, the chromaticity coordinates of each primary color of the target image, the pixel mapping range of the target image, and the coefficients of the transformation matrix used for color space conversion of the target image are considered together to determine the transformation matrix used when performing color space conversion on the target image. Therefore, when the encoder performs color space conversion on the target image, the relevant information indicating the above-mentioned transformation matrix, namely the second identifier, the third identifier, and the fourth identifier, can also be encoded into the bitstream so that the decoder can perform color space conversion processing on the reconstructed YUV format HDR image based on the second identifier, the third identifier, and the fourth identifier.

[0153] In some embodiments, after the encoder converts the RGB format target image into a YUV format target image through color space conversion, in order to reduce the amount of data processing, the Y component image and the UV component image can be downsampled separately. However, under different sampling modes, the sampling positions in the Y component image and the UV component image will be offset. When the decoder upsamples the reconstructed Y component image and the reconstructed UV component image, if the sampling position offset is different from that at the encoder, it will also affect the accuracy of the reconstructed image.

[0154] Based on this, the encoder can also incorporate a fifth identifier into the bitstream, which indicates the offset of the sampling positions of the principal and secondary components of the target image. In other words, after color space conversion, the target image includes images of the principal and secondary components. Before incorporating the target image into the bitstream, the encoder can downsample the images of the principal and secondary components based on the fifth identifier; then, the downsampled images of the principal and secondary components are incorporated into the bitstream.

[0155] For YUV format images, the principal component can be the Y component, and the secondary component can be the UV component.

[0156] Taking the 420 sampling mode as an example, refer to the schematic diagram of the chromaticity sampling position shown in Figure 8. The luminance (Y) sample area in the upper left corner of the image is represented by a black solid line box, and the chromaticity (UV) sample area in the upper left corner of the image is represented by a black solid line box. The center point of the luminance sample area is the luminance sampling point, and the center point of the chromaticity sample area is the chromaticity sampling point. Figure 8 shows six offset cases (a)-(f).

[0157] For the six offset scenarios listed in Figure 8, Table 4 below shows the offset between the chroma sampling position and the luminance sampling position in the 420 sampling mode. That is, Table 4 lists the correspondence between the fifth identifier and the chroma sampling position offset scenarios, and, corresponding to the schematic diagram in Figure 8, lists the horizontal and vertical offset values ​​of the chroma sampling position relative to the luminance sampling position in each offset scenario.

[0158] Table 4

[0159] It should be understood that when the value of the fifth identifier is 0, corresponding to Figure 8(a), the sampling position of the chroma sampling point relative to the luminance sampling point has a horizontal offset of 0 and a vertical offset of 0.5; when the value of the fifth identifier is 1, corresponding to Figure 8(b), the sampling position of the chroma sampling point relative to the luminance sampling point has a horizontal offset of 0.5 and a vertical offset of 0.5; when the value of the fifth identifier is 2, corresponding to Figure 8(c), the sampling position of the chroma sampling point relative to the luminance sampling point has a horizontal offset of 0 and a vertical offset of 0.5. The offset value is 0; when the fifth identifier is 3, corresponding to Figure 8(d), the sampling position of the chroma sampling point is offset by 0.5 in the horizontal direction and 0 in the vertical direction compared to the luminance sampling point; when the fifth identifier is 4, corresponding to Figure 8(e), the sampling position of the chroma sampling point is offset by 0 in the horizontal direction and 1 in the vertical direction compared to the luminance sampling point; when the fifth identifier is 5, corresponding to Figure 8(f), the sampling position of the chroma sampling point is offset by 0.5 in the horizontal direction and 1 in the vertical direction compared to the luminance sampling point.

[0160] As shown in Figure 8 and Table 4 above, when the fifth identifier is 1, the chroma sampling point is located in the middle row and column of the luminance sampling point. In this case, if the pixel coordinates of the luminance sampling point Y are (x, y), then the pixel coordinates of the corresponding chroma sampling points U and V will be (x / 2, y / 2). When the fifth identifier is 2, the chroma sampling point is located at the upper left corner of the luminance sampling point. In this case, if the pixel coordinates of the luminance sampling point Y are (x, y), then the pixel coordinates of the corresponding chroma sampling points U and V will be (floor(x / 2), floor(y / 2)).

[0161] It should be noted that in this 420 sampling mode, the sampling density of luminance samples is lower than that of chrominance to reduce the need for data transmission and storage. Furthermore, in the 420 sampling mode, the sampling density of luminance in the horizontal direction is full resolution, while the sampling density of chrominance is only half that of luminance; in the vertical direction, both luminance and chrominance are sampled at half full resolution.

[0162] It should be understood that the above example is only used to illustrate the 420 sampling mode. For other sampling modes such as the 422 sampling mode, the same method can be used to encode the identifier indicating the offset of the sampling position of the main component and the secondary component in the bitstream. This application embodiment does not limit this.

[0163] Similarly, if the bitstream includes an image header, the aforementioned fifth identifier can also be encoded into that image header.

[0164] In this embodiment of the application, after color space conversion of the target image, if the images of the principal component and the secondary component are downsampled, the fifth identifier indicating the offset of the sampling positions of the principal component and the secondary component of the target image can also be encoded into the bitstream, so that the decoder can perform upsampling processing on the reconstructed images of the Y component and the reconstructed images of the UV component based on the fifth identifier.

[0165] In some embodiments, to accurately render and display the target image, the image encoding method provided in this application further includes: encoding static metadata of the target image into a bitstream, wherein the static metadata includes source device information and image display information. The source device information indicates the color information of the main monitor that generates the target image, and the image display information indicates the display brightness of the target image.

[0166] As an example, the source device information includes at least one of the chromaticity coordinates of the three primary colors of the main monitor, the chromaticity coordinates of standard white light, the maximum display brightness, and the minimum display brightness. The chromaticity coordinates of the three primary colors include the x-coordinates and y-coordinates of the three primary colors, and the chromaticity coordinates of the standard white light include the x-coordinates and y-coordinates of the standard white light.

[0167] Optionally, each item in the source device information can be represented in the form of a 16-bit unsigned integer. This application does not impose any limitations on this.

[0168] As an example, image display information includes at least one of the target image's maximum display brightness and maximum average display brightness.

[0169] The maximum display brightness refers to the maximum brightness value that a single pixel in the target image can achieve. This maximum display brightness can be represented by a 6-bit unsigned integer, in units of 1 cd / m², with a value range of 1 cd / m² to 65535 cd / m². When the target image is any frame in an image sequence or video, the average brightness of all pixels in each frame of the image sequence or video is first calculated. Then, the maximum value among the average brightness values ​​corresponding to each frame is the aforementioned maximum average display brightness.

[0170] Optionally, when there is only one independent image frame, the above maximum average display brightness can be a default value, and this application embodiment does not limit this.

[0171] As shown in Figure 7, the bitstream includes an image header, which carries static metadata. That is, the first to fifth identifiers mentioned above, as well as the static metadata, can all be encoded into this image header.

[0172] In some embodiments, when the target image is a frame in an image sequence or video, in order to accurately render and display the target image, the image encoding method provided in this application embodiment further includes: encoding the dynamic metadata of the target image into the bitstream, wherein the dynamic metadata includes the color gamut volume change data of the target image.

[0173] Among them, gamut volume change data is used to describe the changes in content and scene in the image over time or frame number. Gamut volume change data includes color change information and brightness change information of the target image.

[0174] As shown in Figure 7, the bitstream includes an image header, which carries static metadata. That is, the first to fifth identifiers mentioned above, as well as static and / or dynamic metadata, can all be encoded into the image header.

[0175] In some embodiments of this application, when the static metadata and / or dynamic metadata of the target image are encoded into the bitstream, as shown in FIG9, a metadata header may also be added to the bitstream to carry the static metadata and / or dynamic metadata of the target image through the metadata header.

[0176] In some embodiments, when the bitstream includes a metadata sub-bitstream corresponding to the metadata header, the static and dynamic metadata of the target image can also be carried through the metadata sub-bitstream.

[0177] It should be understood that the embodiments of this application only limit the data information of the target image that needs to be encoded into the bitstream, such as the first identifier to the fifth identifier, as well as static metadata and / or dynamic metadata. It does not limit the specific location of these data information when they are encoded into the bitstream. That is, the image header, metadata header and metadata sub-bitstream in the above example can all carry relevant data information. Of course, all the above data information can also be carried only in the image header, metadata header or metadata sub-bitstream. The embodiments of this application do not limit this.

[0178] Based on the above embodiments, in order to carry the first to fifth identifiers, as well as the static and dynamic metadata of the target image, in the image header of the bitstream, this application embodiment adds four syntax elements to the image header of the bitstream to carry image information and metadata information related to the target image. Table 5 provides an exemplary syntax element for the image header.

[0179] Table 5

[0180] The `picture_header_size` parameter defines the spatial size of the image header, which contains relevant image information. For example, `img_width` specifies the width of the encoded image; `img_height` specifies the height of the encoded image; `picture_format` specifies the image format, defining whether the encoded image is a 4:4:4 sampling YUV image, a 4:2:2 sampling YUV image, a 4:2:0 sampling YUV image, or an RGB image; `bit_depth` contains quantization information; and `model_header()` contains the syntax related to the encoding model. The specific meanings of these parameters can be found in the definitions of advanced syntax in the JPEG AI standard's encoded file header; these details will not be elaborated upon here.

[0181] In the above image header, coding_independent_code_points() is a new syntax added in this application for carrying image characteristics, mastering_display_color_volume() is a new syntax added in this application for carrying source device information, content_light_level_information() is a new syntax added in this application for carrying image display information, and dynamic_metadata() is a new syntax added in this application for carrying dynamic metadata of the target image.

[0182] It should be noted that in the syntax element table shown in the embodiments of this application, the Descriptor column on the right is the data type and bit information of the variable defined by the corresponding syntax. For example, U(16) refers to a 16-bit unsigned integer.

[0183] As an example, the syntax elements of the image feature syntax coding_independent_code_points() are shown in Table 6.

[0184] Table 6

[0185] In Table 6, colour_primaries is the byte corresponding to the second identifier indicating the chromaticity coordinates of each primary color of the target image; matrix_coefficients is the byte corresponding to the fourth identifier indicating the coefficients of the transformation matrix used for color space conversion of the target image; image_full_range_flag is the byte corresponding to the third identifier indicating the pixel mapping range of the target image; transfer_characteristics is the byte corresponding to the first identifier indicating the photoelectric conversion function that matches the target image; and chroma420_sample_loc_type is the byte corresponding to the fifth identifier indicating the offset of the sampling positions of the principal and secondary components of the target image.

[0186] As an example, the syntax elements of the source device information mastering_display_color_volume() are shown in Table 7.

[0187] Table 7

[0188] In Table 7, mastering_display_colour_primaries_x[i] and mastering_display_colour_primaries_y[i] are the bytes corresponding to the chromaticity coordinates of the three primary colors of the main monitor, mastering_display_white_point_chromaticity_x and mastering_display_white_point_chromaticity_y are the bytes corresponding to the chromaticity coordinates of the standard white light of the main monitor, mastering_display_maximum_luminance is the byte corresponding to the maximum display brightness of the main monitor, and mastering_display_minimum_luminance is the minimum display brightness of the main monitor.

[0189] As an example, the syntax elements of the image display information content_light_level_information() are shown in Table 8.

[0190] Table 8

[0191] In Table 8, maximum_content_light_level is the byte corresponding to the maximum display brightness of the target image, and maximum_frame_average_light_level is the byte corresponding to the maximum average display brightness of the target image.

[0192] As an example, the syntax elements of dynamic_metadata() are shown in Table 9.

[0193] Table 9

[0194] In Table 9 above, itu_t_t35_country_code is the byte with a value specified as the country code by ITU-T T.35 Annex A, itu_t_t35_country_code_extension_byte should be the byte with a value specified as the country code by ITU-T T.35 Annex B, itu_t_t35_payload_byte should be the byte containing data registered in accordance with ITU-T T.35, and the ITU-T T.35 terminal provider code and the terminal provider-oriented code (terminal provider-directed code) should be included in the first or more bytes of itu_t_t35_payload_byte, with the format specified by the competent authority that issues the terminal provider code. Any remaining itu_t_t35_payload_byte data should be data with the syntax and semantics specified by the entity identified by the ITU-T T.35 country code and terminal provider code.

[0195] It should be understood that Table 5 above is an example, in which all the data information added in the embodiments of this application is carried in the image header. In specific applications, Table 5 above can also be modified to carry only some data in the image header. The embodiments of this application do not limit this.

[0196] Based on the above embodiments, this application embodiment can also carry static metadata and dynamic metadata of the target image in the metadata header of the bitstream. As an example, Table 10 below gives a syntax element of a metadata header.

[0197] Table 10

[0198] MDH stands for Metadata Header Marker, and the metadata header is formed after the metadata header is marked. Metadata_header_size is the size of the metadata header.

[0199] As an example, the syntax elements of static metadata are shown in Table 11 below.

[0200] Table 11

[0201] For details regarding the syntax elements of mastering_display_color_volume(), please refer to Table 7 above. For details regarding the syntax elements of content_light_level_information(), please refer to Table 8 above. These will not be elaborated upon here.

[0202] In summary, when encoding the target image, the encoder can also encode the first identifier, which indicates the photoelectric conversion function that matches the target image, into the bitstream. This satisfies the display requirements of both SDR and HDR images at the decoding end, allowing the decoder to render and display the reconstructed image using the corresponding photoelectric conversion function after decoding the first identifier.

[0203] Next, the image decoding method provided in the embodiments of this application will be described.

[0204] Please refer to Figure 10. This application provides an image decoding method, which is applied to a decoder and includes the following steps.

[0205] Step 1001: Parse a first identifier from the bitstream, the first identifier indicating a photoelectric conversion function that matches a target image, the target image including HDR images and / or SDR images.

[0206] The bitstream includes an image header, in which a first identifier is carried.

[0207] Optionally, the bitstream includes an image header, in which a second identifier, a third identifier, and a fourth identifier are carried. In this case, the second identifier, the third identifier, and the fourth identifier can also be parsed from the bitstream.

[0208] The second identifier indicates the chromaticity coordinates of each primary color of the target image; the third identifier indicates the mapping range of the target image; and the fourth identifier indicates the coefficients of the transformation matrix used for color space conversion of the target image.

[0209] Optionally, the bitstream includes an image header, in which the fifth identifier is carried. In this case, the fifth identifier can also be parsed from the bitstream.

[0210] Optionally, the bitstream includes an image header, and the static metadata and / or dynamic metadata of the target image are also included in the image header. In this case, the static metadata and / or dynamic metadata of the target image can be parsed from the bitstream.

[0211] Step 1002: Decode the bitstream to obtain the reconstructed image of the target image.

[0212] When the fifth identifier is parsed from the bitstream, step 1002 can be implemented as follows: parse the bitstream to obtain the reconstructed image of the principal component and the reconstructed image of the secondary component of the target image; then, based on the fifth identifier, upsample the reconstructed image of the principal component and the reconstructed image of the secondary component to obtain the reconstructed image.

[0213] Optionally, if the second identifier, the third identifier, and the fourth identifier are parsed from the bitstream, before performing step 1003 above, the decoder determines the transformation matrix used by the encoding end to perform color space conversion on the target image by looking up Tables 1-3 above, based on the second identifier, the third identifier, and the fourth identifier, and then performs inverse color space conversion on the reconstructed image based on the transformation matrix.

[0214] Step 1003: Based on the first identifier, perform photoelectric conversion on the reconstructed image.

[0215] In some embodiments, when static metadata of the target image is parsed from the bitstream, the reconstructed image can be tone-mapped based on the static metadata to display the reconstructed image.

[0216] In some embodiments, by parsing the dynamic metadata of the target image from the bitstream, the reconstructed image can be tone-mapped based on the dynamic metadata in order to display the reconstructed image.

[0217] It should be noted that when the above decoder executes the image decoding method, the specific meaning and execution order of the relevant technical features can be referred to the above embodiment of the encoder executing the image encoding method, and will not be repeated here.

[0218] In summary, when decoding the bitstream, the decoder can determine the photoelectric conversion function that matches the target image based on the decoded first identifier. Then, it can use this photoelectric conversion function to accurately render and display the reconstructed image of the target image, so as to simultaneously meet the display requirements of SDR and HDR images at the decoding end and ensure the accuracy of displaying the target image.

[0219] Figure 11 is a schematic diagram of an encoding device provided in an embodiment of this application. The encoding device can be the encoder shown in Figure 1. Referring to Figure 11, the encoding device 1100 includes: an image encoding module 1101 and an information encoding module 402.

[0220] Image encoding module 1101 is used to encode the target image into a bitstream;

[0221] Information encoding module 1102 is used to encode a first identifier into the bitstream, wherein the first identifier indicates a photoelectric conversion function that matches the target image, the target image including a high dynamic range (HDR) image and / or a standard dynamic range (SDR) image.

[0222] Optionally, the bitstream includes an image header, in which the first identifier is encoded.

[0223] Optionally, the information encoding module 1102 is further configured to:

[0224] The second identifier is encoded into the bitstream, and the second identifier indicates the chromaticity coordinates of each primary color of the target image;

[0225] A third identifier is encoded into the bitstream, the third identifier indicating the pixel mapping range of the target image;

[0226] A fourth identifier is encoded into the bitstream, the fourth identifier indicating the coefficients of the transformation matrix used for color space conversion of the target image.

[0227] Optionally, the device further includes:

[0228] The image conversion module is used to perform color space conversion on the target image based on the second identifier, the third identifier, and the fourth identifier.

[0229] Optionally, the bitstream includes an image header, in which the second identifier, the third identifier, and the fourth identifier are encoded.

[0230] Optionally, the information encoding module 1102 is further configured to:

[0231] A fifth identifier is encoded into the bitstream, the fifth identifier indicating the offset of the sampling positions of the primary and secondary components of the target image.

[0232] Optionally, the target image, after color space conversion, includes an image of the principal component and an image of the secondary component; the apparatus further includes:

[0233] The downsampling module is used to downsample the images of the main component and the secondary component based on the fifth identifier;

[0234] The image encoding module 1101 is specifically used for:

[0235] The downsampled images of the primary component and the downsampled images of the secondary component are encoded into the bitstream.

[0236] Optionally, the bitstream includes an image header, and the fifth identifier is encoded into the image header.

[0237] Optionally, the information encoding module 1102 is further configured to:

[0238] The static metadata of the target image is encoded into the bitstream. The static metadata includes source device information and image display information. The source device information indicates the color information of the main monitor that generated the target image, and the image display information indicates the display brightness of the target image.

[0239] Optionally, the source device information includes at least one of the chromaticity coordinates of the three primary colors of the main monitor, the chromaticity coordinates of standard white light, the maximum display brightness, and the minimum display brightness.

[0240] Optionally, the image display information includes at least one of the maximum display brightness and the maximum average display brightness of the target image.

[0241] Optionally, the bitstream includes an image header carrying the static metadata; or, the bitstream includes a metadata header carrying the static metadata; or, the bitstream includes a metadata sub-bitstream corresponding to the metadata header carrying the static metadata.

[0242] Optionally, the information encoding module 1102 is further configured to:

[0243] The dynamic metadata of the target image is encoded into the bitstream, and the dynamic metadata includes the color gamut volume change data of the target image.

[0244] Optionally, the bitstream includes an image header carrying the dynamic metadata; or, the bitstream includes a metadata header carrying the dynamic metadata; or, the bitstream includes a metadata sub-bitstream corresponding to the metadata header carrying the dynamic metadata.

[0245] In this embodiment of the application, when the encoding device encodes the target image, it can also encode the first identifier, which indicates the photoelectric conversion function that matches the target image, into the bitstream, so as to simultaneously meet the display requirements of SDR and HDR images at the decoding end. This allows the decoder to render and display the reconstructed image by using the corresponding photoelectric conversion function after decoding the first identifier.

[0246] It should be noted that the encoding device provided in the above embodiments is only illustrated by the division of the above functional modules when performing image encoding. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the encoding device and the image encoding method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0247] Figure 12 is a schematic diagram of a decoding device provided in an embodiment of this application. The decoding device can be the decoder shown in Figure 1. Referring to Figure 12, the device includes: an information decoding module 1201, an image decoding module 1202, and an image conversion module 1203.

[0248] Information decoding module 1201 is used to parse a first identifier from the bit stream, the first identifier indicating a photoelectric conversion function matching a target image, the target image including a high dynamic range (HDR) image and / or a standard dynamic range (SDR) image;

[0249] Image decoding module 1202 is used to decode the bitstream to obtain a reconstructed image of the target image;

[0250] Image conversion module 1203 is used to perform photoelectric conversion on the reconstructed image based on the first identifier.

[0251] Optionally, the bitstream includes an image header, in which the first identifier is carried.

[0252] Optionally, the information decoding module 1201 is further configured to:

[0253] The second identifier is parsed from the bitstream, the second identifier indicating the chromaticity coordinates of each primary color of the target image;

[0254] A third identifier is parsed from the bitstream, the third identifier indicating the mapping range of the target image;

[0255] The fourth identifier is parsed from the bitstream, the fourth identifier indicating the coefficients of the transformation matrix used for color space conversion of the target image.

[0256] Optionally, the device further includes:

[0257] The image inverse conversion module is used to perform color space inverse conversion on the reconstructed image based on the second identifier, the third identifier, and the fourth identifier.

[0258] Optionally, the bitstream includes an image header, in which the second identifier, the third identifier, and the fourth identifier are carried.

[0259] Optionally, the information decoding module 1201 is further configured to:

[0260] The fifth identifier is parsed from the bitstream, which indicates the offset of the sampling positions of the principal and secondary components of the target image.

[0261] Optionally, the image decoding module 1202 is specifically used for:

[0262] The bitstream is parsed to obtain the reconstructed images of the principal components and the secondary components of the target image;

[0263] Based on the fifth identifier, the reconstructed image of the principal component and the reconstructed image of the secondary component are upsampled to obtain the reconstructed image.

[0264] Optionally, the bitstream includes an image header, and the fifth identifier is carried in the image header.

[0265] Optionally, the information decoding module 1201 is further configured to:

[0266] The static metadata of the target image is parsed from the bitstream. The static metadata includes source device information and image display information. The source device information indicates the color information of the main monitor that generated the target image, and the image display information indicates the display brightness of the target image.

[0267] Optionally, the device further includes:

[0268] A tone mapping module is used to perform tone mapping on the reconstructed image based on the static metadata.

[0269] Optionally, the source device information includes at least one of the chromaticity coordinates of the three primary colors of the main monitor, the chromaticity coordinates of standard white light, the maximum display brightness, and the minimum display brightness.

[0270] Optionally, the image display information includes at least one of the maximum display brightness and the maximum average display brightness of the target image.

[0271] Optionally, the bitstream includes an image header carrying the static metadata; or, the bitstream includes a metadata header carrying the static metadata; or, the bitstream includes a metadata sub-bitstream corresponding to the metadata header carrying the static metadata.

[0272] Optionally, the information decoding module 1201 is further configured to:

[0273] The dynamic metadata of the target image is parsed from the bitstream, and the dynamic metadata includes the color gamut volume change data of the target image.

[0274] Optionally, the tone mapping module is further configured to:

[0275] Based on the static metadata and the dynamic metadata, tone mapping is performed on the reconstructed image.

[0276] Optionally, the bitstream includes an image header carrying the dynamic metadata; or, the bitstream includes a metadata header carrying the dynamic metadata; or, the bitstream includes a metadata sub-bitstream corresponding to the metadata header carrying the dynamic metadata.

[0277] In this embodiment of the application, when decoding the bitstream, the decoding device can determine the photoelectric conversion function that matches the target image based on the decoded first identifier, and then use the photoelectric conversion function to accurately render and display the reconstructed image of the target image, so as to simultaneously meet the display requirements of SDR images and HDR images at the decoding end and ensure the accuracy when displaying the target image.

[0278] It should be noted that the decoding device provided in the above embodiments is only illustrated by the division of the above functional modules when performing image decoding. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the decoding device and the image decoding method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0279] This application embodiment also provides an encoding device, the encoding device including: a processor, the processor being coupled to a memory, the memory being used to store programs or instructions, and when the program or instructions are executed by the processor, the encoding device performs the above-described image encoding method.

[0280] This application embodiment also provides a decoding device, the decoding device including: a processor, the processor being coupled to a memory, the memory being used to store programs or instructions, and when the program or instructions are executed by the processor, the decoding device performs the above-described image decoding method.

[0281] This application also provides an encoding / decoding system, which includes the encoding device and / or the decoding device.

[0282] This application also provides a computer-readable storage medium including program code, which, when run on a computer, causes the computer to perform the image encoding method described above.

[0283] This application also provides a computer-readable storage medium including program code, which, when run on a computer, causes the computer to perform the image decoding method described above.

[0284] This application also provides a computer program product, including instructions that, when executed on a computer, cause the computer to perform the image encoding method described above.

[0285] This application also provides a computer program product, including instructions that, when executed on a computer, cause the computer to perform the image decoding method described above.

[0286] This application also provides a computer-readable storage medium storing a bitstream obtained according to the image encoding method described above.

[0287] This application also provides a device for storing a bitstream, including at least one storage medium and a communication interface; the communication interface is used to receive or send the bitstream; the at least one storage medium is used to store the bitstream; the bitstream is encoded by an encoder according to the image encoding method described above.

[0288] This application embodiment also provides a method for storing a bitstream, comprising: receiving a bitstream through a communication interface; storing the bitstream in one or more storage media, wherein the bitstream is encoded by an encoder according to the above-described image encoding method.

[0289] This application also provides a system for distributing bitstreams, including at least one storage medium and a video streaming device; the at least one storage medium is used to store the bitstream, which is encoded by an encoder according to the image encoding method described above; the video streaming device is used to send the bitstream in the at least one storage medium to the decoder in response to a request from the decoder.

[0290] This application embodiment also provides a method for distributing a bitstream, comprising: receiving a first request; in response to the first request, selecting a bitstream from at least one storage medium; sending the bitstream to a destination device; wherein the at least one storage medium is used to store the bitstream, and the bitstream is encoded by an encoder according to the image encoding method described above.

[0291] This application also provides a system for processing bitstreams, including an image source device, an encoder, one or more storage media, and a destination device; the image source device is used to provide image data; the encoder is used to acquire the image data from the image source device through an interface and encode the image data to obtain one or more bitstreams, wherein the bitstreams are encoded by the encoder according to the image encoding method described above; the encoder is used to store the one or more bitstreams in one or more storage media; or the encoder is used to encapsulate the one or more bitstreams to obtain a transmission bitstream; the encoder is used to transmit the transmission bitstream to the destination device through a communication link or communication network; the destination device is used to decapsulate the transmission bitstream to obtain the one or more bitstreams; the destination device is used to decode the one or more bitstreams to obtain decoded data.

[0292] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital versatile disc (DVD)), or a semiconductor medium (e.g., solid-state disk (SSD)). It is worth noting that the computer-readable storage medium mentioned in the embodiments of this application can be a non-volatile storage medium; in other words, it can be a non-transient storage medium.

[0293] It should be understood that "multiple" as mentioned herein refers to two or more. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. In addition, to facilitate a clear description of the technical solutions of the embodiments of this application, the terms "first," "second," etc., are used in the embodiments of this application to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first," "second," etc., do not limit the quantity or execution order, and the terms "first," "second," etc., do not necessarily imply that they are different.

[0294] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals involved in the embodiments of this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0295] The above descriptions are embodiments provided in this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An image coding method characterized by, The method includes: Encode the target image into a bitstream; A first identifier is encoded into the bitstream, wherein the first identifier indicates a photoelectric conversion function that matches the target image, the target image including a high dynamic range (HDR) image and / or a standard dynamic range (SDR) image.

2. The method of claim 1, wherein, The bitstream includes an image header, and the first identifier is encoded into the image header.

3. The method of claim 1 or 2, wherein, The method further includes: The second identifier is encoded into the bitstream, and the second identifier indicates the chromaticity coordinates of each primary color of the target image; A third identifier is encoded into the bitstream, the third identifier indicating the pixel mapping range of the target image; A fourth identifier is encoded into the bitstream, the fourth identifier indicating the coefficients of the transformation matrix used for color space conversion of the target image.

4. The method of claim 3, wherein, Before encoding the target image into the bitstream, the method further includes: Based on the second identifier, the third identifier, and the fourth identifier, the target image is subjected to color space conversion.

5. The method of claim 3 or 4, wherein, The bitstream includes an image header, and the second identifier, the third identifier, and the fourth identifier are encoded into the image header.

6. The method according to any one of claims 1 to 5, wherein, The method further includes: A fifth identifier is encoded into the bitstream, the fifth identifier indicating the offset of the sampling positions of the primary and secondary components of the target image.

7. The method of claim 6, wherein, The target image, after color space conversion, comprises a principal component image and a secondary component image; before encoding the target image into the bitstream, the method further includes: Based on the fifth identifier, the images of the principal component and the secondary component are downsampled; The step of encoding the target image into a bitstream includes: The downsampled images of the primary component and the downsampled images of the secondary component are encoded into the bitstream.

8. The method of claim 6 or 7, wherein, The bitstream includes an image header, and the fifth identifier is encoded into the image header.

9. The method according to any one of claims 1 to 8, wherein, The method further includes: The static metadata of the target image is encoded into the bitstream. The static metadata includes source device information and image display information. The source device information indicates the color information of the main monitor that generated the target image, and the image display information indicates the display brightness of the target image.

10. The method of claim 9, wherein, The source device information includes at least one of the chromaticity coordinates of the three primary colors of the main monitor, the chromaticity coordinates of standard white light, the maximum display brightness, and the minimum display brightness.

11. The method of claim 9 or 10, wherein, The image display information includes at least one of the maximum display brightness and the maximum average display brightness of the target image.

12. The method according to any one of claims 9 to 11, characterized in that, The bitstream includes an image header that carries the static metadata; or, the bitstream includes a metadata header that carries the static metadata; or, the bitstream includes a metadata sub-bitstream corresponding to the metadata header that carries the static metadata.

13. The method according to any one of claims 9 to 12, wherein, The method further includes: The dynamic metadata of the target image is encoded into the bitstream, and the dynamic metadata includes the color gamut volume change data of the target image.

14. The method of claim 13, wherein, The bitstream includes an image header that carries the dynamic metadata; or, the bitstream includes a metadata header that carries the dynamic metadata; or, the bitstream includes a metadata sub-bitstream corresponding to the metadata header that carries the dynamic metadata.

15. An image decoding method, characterized by, The method includes: Parse a first identifier from the bitstream, the first identifier indicating a photoelectric conversion function that matches a target image, the target image including a high dynamic range (HDR) image and / or a standard dynamic range (SDR) image; Decode the bitstream to obtain the reconstructed image of the target image; Based on the first identifier, the reconstructed image is subjected to photoelectric conversion.

16. The method of claim 15, wherein, The bitstream includes an image header, and the first identifier is carried in the image header.

17. The method of claim 15 or 16, wherein, The method further includes: The second identifier is parsed from the bitstream, the second identifier indicating the chromaticity coordinates of each primary color of the target image; A third identifier is parsed from the bitstream, the third identifier indicating the mapping range of the target image; The fourth identifier is parsed from the bitstream, the fourth identifier indicating the coefficients of the transformation matrix used for color space conversion of the target image.

18. The method of claim 17, wherein, Before performing photoelectric conversion on the reconstructed image based on the first identifier, the method further includes: Based on the second identifier, the third identifier, and the fourth identifier, the reconstructed image undergoes an inverse color space conversion.

19. The method of claim 17 or 18, wherein, The bitstream includes an image header, in which the second identifier, the third identifier, and the fourth identifier are carried.

20. The method of any one of claims 15-19, wherein, The method further includes: The fifth identifier is parsed from the bitstream, which indicates the offset of the sampling positions of the principal and secondary components of the target image.

21. The method of claim 20, wherein, The process of parsing the bitstream to obtain the reconstructed image of the target image includes: The bitstream is parsed to obtain the reconstructed images of the principal components and the secondary components of the target image; Based on the fifth identifier, the reconstructed image of the principal component and the reconstructed image of the secondary component are upsampled to obtain the reconstructed image.

22. The method of any one of claims 20 or 21, wherein, The bitstream includes an image header, and the fifth identifier is carried in the image header.

23. The method of any one of claims 15-22, wherein, The method further includes: The static metadata of the target image is parsed from the bitstream. The static metadata includes source device information and image display information. The source device information indicates the color information of the main monitor that generated the target image, and the image display information indicates the display brightness of the target image.

24. The method of claim 23, wherein, The method further includes: Based on the static metadata, tone mapping is performed on the reconstructed image.

25. The method of claim 23 or 24, wherein, The source device information includes at least one of the chromaticity coordinates of the three primary colors of the main monitor, the chromaticity coordinates of standard white light, the maximum display brightness, and the minimum display brightness.

26. The method of any one of claims 23-25, wherein, The image display information includes at least one of the maximum display brightness and the maximum average display brightness of the target image.

27. The method of any one of claims 23-26, wherein, The bitstream includes an image header that carries the static metadata; or, the bitstream includes a metadata header that carries the static metadata; or, the bitstream includes a metadata sub-bitstream corresponding to the metadata header that carries the static metadata.

28. The method of any one of claims 23-27, wherein, The method further includes: The dynamic metadata of the target image is parsed from the bitstream, and the dynamic metadata includes the color gamut volume change data of the target image.

29. The method of claim 28, wherein, The method further includes: Based on the static metadata and the dynamic metadata, tone mapping is performed on the reconstructed image.

30. The method of claim 29, wherein, The bitstream includes an image header that carries the dynamic metadata; or, the bitstream includes a metadata header that carries the dynamic metadata; or, the bitstream includes a metadata sub-bitstream corresponding to the metadata header that carries the dynamic metadata.

31. An encoding device, comprising: The encoding device includes: The image encoding module is used to encode the target image into a bitstream; An information encoding module is used to encode a first identifier into the bitstream, wherein the first identifier indicates a photoelectric conversion function that matches the target image, the target image including a high dynamic range (HDR) image and / or a standard dynamic range (SDR) image.

32. A decoding device, comprising: The decoding device includes: An information decoding module is used to parse a first identifier from the bitstream, the first identifier indicating a photoelectric conversion function that matches a target image, the target image including a high dynamic range (HDR) image and / or a standard dynamic range (SDR) image; An image decoding module is used to decode the bitstream to obtain a reconstructed image of the target image; An image conversion module is used to perform photoelectric conversion on the reconstructed image based on the first identifier.

33. An encoding device, comprising: The encoding device includes: a processor coupled to a memory for storing programs or instructions, which, when executed by the processor, cause the encoding device to perform the method as described in any one of claims 1 to 14.

34. A decoding device, comprising: The decoding device includes a processor coupled to a memory for storing programs or instructions that, when executed by the processor, cause the decoding device to perform the method as described in any one of claims 15 to 30.

35. A coding system characterized by The encoding / decoding system includes the encoding device as described in claim 33, and / or the decoding device as described in claim 34.

36. A computer-readable storage medium, characterized in that, Includes program code that, when run on a computer, causes the computer to perform the image encoding method as described in any one of claims 1 to 14.

37. A computer-readable storage medium, comprising: Includes program code that, when run on a computer, causes the computer to perform the image decoding method as described in any one of claims 15 to 30.

38. A computer program product, characterised in that, Includes instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1 to 14.

39. A computer program product, characterised in that, Includes instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 15 to 30.

40. A computer-readable storage medium, comprising: The computer-readable storage medium stores a bitstream obtained by the method of any one of claims 1-14, executed by one or more processors.

41. An apparatus for storing a bitstream, the apparatus comprising: Includes at least one storage medium and a communication interface; The communication interface is used to receive or send code streams; The at least one storage medium is used to store the bit stream; The bitstream is encoded by the encoder according to any one of the image encoding methods in claims 1 to 14.

42. A method of storing a bitstream, characterized by, include: Receive the bitstream through the communication interface; The bitstream is stored in one or more storage media, and the bitstream is an encoder. Encoded according to any one of the image encoding methods in claims 1 to 14.

43. A system for distributing a code stream, the system comprising: Includes at least one storage medium and a video streaming device; The at least one storage medium is used to store a bitstream, which is encoded by an encoder using any one of the image encoding methods according to claims 1 to 14; The video streaming device is configured to, in response to a request from the decoder, transmit the bitstream in the at least one storage medium to the decoder.

44. A method of distributing a code stream, the method comprising: include: Receive the first request; In response to the first request, a bitstream is selected from at least one storage medium; Send the target bitstream to the destination device; The at least one storage medium is used to store a bitstream, which is encoded by an encoder using any one of the image encoding methods according to claims 1 to 14.

45. A system for processing a bitstream, the system comprising: Includes an image source device, an encoder device, one or more storage media, and a destination device; The image source device is used to provide image data; The encoder device is used to acquire the image data of the image source device through an interface, and encode the image data to obtain one or more bitstreams, wherein the bitstreams are encoded by the encoder according to any one of the image encoding methods according to claims 1 to 14; The encoder device is used to store the one or more bit streams into one or more storage media; or The encoder device is used to encapsulate the one or more code streams to obtain a transmission code stream; The encoder device is used to transmit the transmitted code stream to the destination device via a communication link or communication network; The destination device is used to decapsulate the transmitted code stream to obtain the one or more code streams; The target device is used to decode the one or more bitstreams to obtain decoded data.

Images