Fallback map generation method, encoding method, apparatus, and device

Abstract

The present application relates to the technical field of image processing, and discloses a fallback map generation method, an encoding method, an apparatus, and a device. The fallback map generation method in embodiments of the present application comprises: respectively decoding reference map information and gain map information to obtain a first reference map and a first gain map; and performing a first operation on the basis of the first reference map and the first gain map, and generating a first fallback map on the basis of the first operation.

Description

Method, apparatus and device for generating a backup image, and method and apparatus for encoding

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Chinese Patent Application No. 202411102221.X, filed on August 12, 2024, the contents of which are incorporated herein by reference in its entirety. TECHNICAL FIELD

[0003] The present application belongs to the technical field of image processing, and specifically relates to a backup image generation method, an encoding method, an apparatus and a device. BACKGROUND

[0004] With the progress of image acquisition and display technology, high dynamic range imaging and display can be completed on display devices such as smart phones, tablets and computer displays. Although there are more and more devices supporting high dynamic range still image display, there are still a large number of electronic devices that only support standard dynamic range (SDR). Taking into account the high dynamic range image shooting can be displayed on high dynamic range (HDR) display devices and SDR display devices at the same time, and good display effect can be presented. The HDR acquisition, storage and display scheme based on gain map is a better scheme.

[0005] In the related art, a display device generates a backup image by applying a gain map to a reference image, and uses the backup image for display. However, how to apply the gain map to the reference image to generate the backup image is a technical problem to be solved. SUMMARY

[0006] Embodiments of the present application provide a backup image generation method, an encoding method, an apparatus and a device, which can solve the problem of how to apply a gain map to a reference image to generate a backup image.

[0007] In a first aspect, a backup image generation method is provided, which is executed by a decoding end device, and the method comprises:

[0008] Decoding reference image information and gain map information respectively to obtain a first reference image and a first gain map;

[0009] Performing a first operation based on the first reference image and the first gain map, and generating a first backup image based on the first operation;

[0010] In the case where the first gain map is a gain map of three channels, the gain map information comprises encoding information of the gain map of each channel of the three channels, or the gain map information comprises encoding information of an encapsulated image of the gain map of the three channels.

[0011] and / or,

[0012] The first operation satisfies at least one of the following:

[0013] Determine that the resolution of the first backup image is the same as the resolution of the first reference image;

[0014] Obtain a backup image resolution from metadata, and apply the first gain image based on the backup image resolution;

[0015] Obtain a backup image bit depth, and perform backup image conversion based on the backup image bit depth;

[0016] In a case where it is determined to perform color gamut space conversion, perform conversion processing on the first reference image;

[0017] Obtain a gain image bit depth, and apply the first gain image on the first reference image based on the gain image bit depth.

[0018] In a second aspect, an encoding method is provided, which is executed by an encoding end device, and the method comprises:

[0019] Generate a second gain image based on a second reference image and a second backup image;

[0020] Encode the second reference image and the second gain image respectively to obtain reference image information and gain image information;

[0021] The gain image information comprises a backup image bit depth; and / or,

[0022] In a case where the second gain image is a gain image of three channels, the gain image information comprises encoded information of the gain image of each channel of the three channels respectively; or, the gain image information comprises encoded information of an encapsulated image of the gain image of the three channels; and / or,

[0023] The gain image information comprises a gain image bit depth.

[0024] In a third aspect, a backup image generation apparatus is provided, comprising:

[0025] A processing module is configured to decode reference image information and gain image information respectively to obtain a first reference image and a first gain image;

[0026] The processing module is further configured to execute a first operation based on the first reference image and the first gain image, and generate a first backup image based on the first operation;

[0027] In a case where the first gain map is a gain map of three channels, the gain map information respectively comprises encoded information of a gain map of each of the three channels, or the gain map information comprises encoded information of a packaged image of the gain maps of the three channels.

[0028] and / or,

[0029] The first operation satisfies at least one of the following:

[0030] It is determined that the resolution of the first backup map is the same as the resolution of the first reference map;

[0031] The backup map resolution is obtained from the metadata, and the first gain map is applied based on the backup map resolution;

[0032] The backup map bit depth is obtained, and backup map conversion is performed based on the backup map bit depth;

[0033] In a case where it is determined to perform color gamut space conversion, the first reference map is converted;

[0034] The gain map bit depth is obtained, and the first gain map is applied on the first reference map based on the gain map bit depth.

[0035] In a fourth aspect, an encoding device is provided, comprising:

[0036] A processing module is configured to generate a second gain map based on a second reference map and a second backup map;

[0037] The processing module is further configured to respectively encode the second reference map and the second gain map to obtain reference map information and gain map information;

[0038] The gain map information comprises a backup map bit depth; and / or,

[0039] In a case where the second gain map is a gain map of three channels, the gain map information respectively comprises encoded information of a gain map of each of the three channels, or the gain map information comprises encoded information of a packaged image of the gain maps of the three channels; and / or,

[0040] The gain map information comprises a gain map bit depth.

[0041] In a fifth aspect, a backup map generation device is provided, which is configured to perform the steps of the method of the first aspect.

[0042] In a sixth aspect, an encoding device is provided, which is configured to perform the steps of the method of the second aspect.

[0043] In a seventh aspect, a decoding-side device is provided, which includes a processor and a memory storing programs or instructions executable on the processor, and the programs or instructions, when executed by the processor, implement steps of the method according to the first aspect.

[0044] In an eighth aspect, a decoding-side device is provided, which includes a processor and a communication interface, wherein,

[0045] The processor is configured to decode the reference map information and the gain map information respectively to obtain a first reference map and a first gain map.

[0046] The processor is further configured to perform a first operation based on the first reference map and the first gain map, and generate a first backup map based on the first operation.

[0047] In a case where the first gain map is a gain map of three channels, the gain map information includes encoded information of a gain map of each of the three channels, or the gain map information includes encoded information of a packed image of the gain maps of the three channels.

[0048] And / or,

[0049] The first operation satisfies at least one of the following conditions:

[0050] The resolution of the first backup map is determined to be the same as the resolution of the first reference map.

[0051] The backup map resolution is obtained from metadata, and the first gain map is applied based on the backup map resolution.

[0052] The backup map bit depth is obtained, and backup map conversion is performed based on the backup map bit depth.

[0053] In a case where it is determined to perform color gamut space conversion, the first reference map is converted.

[0054] The gain map bit depth is obtained, and the first gain map is applied on the first reference map based on the gain map bit depth.

[0055] In a ninth aspect, an encoding-side device is provided, which includes a processor and a memory storing programs or instructions executable on the processor, and the programs or instructions, when executed by the processor, implement steps of the method according to the second aspect.

[0056] In a tenth aspect, an encoding-side device is provided, which includes a processor and a communication interface, wherein,

[0057] The processor is configured to generate a second gain map based on a second reference map and a second backup map.

[0058] The processor is further configured to encode the second reference map and the second gain map respectively to obtain reference map information and gain map information.

[0059] The gain map information comprises a backup map bit depth; and / or,

[0060] When the second gain map is a gain map of three channels, the gain map information comprises encoded information of the gain map of each of the three channels; or, the gain map information comprises encoded information of a packaged image of the gain map of the three channels; and / or,

[0061] The gain map information comprises a gain map bit depth.

[0062] In a eleventh aspect, a readable storage medium is provided, and the readable storage medium stores a program or instructions, which are executed by a processor to implement the steps of the method in the first aspect or the steps of the method in the second aspect.

[0063] In a twelfth aspect, a wireless communication system is provided, and the wireless communication system comprises a decoding end device and an encoding end device, the decoding end device is configured to implement the steps of the method in the first aspect, and the encoding end device is configured to implement the steps of the method in the second aspect.

[0064] In a thirteenth aspect, a chip is provided, and the chip comprises a processor and a communication interface, the communication interface is coupled to the processor, and the processor is configured to run a program or instructions to implement the method in the first aspect or the method in the second aspect.

[0065] In a fourteenth aspect, a computer program / program product is provided, and the computer program / program product is stored in a storage medium, and the computer program / program product is executed by at least one processor to implement the method in the first aspect or the method in the second aspect.

[0066] In the embodiments of the present application, the reference picture information and the gain picture information are decoded respectively to obtain a first reference picture and a first gain picture; a first operation is performed based on the first reference picture and the first gain picture, and a first backup picture is generated based on the first operation, so that the backup picture can be generated by applying the gain picture to the reference picture. Further, in the case that the first gain picture is a gain picture of three channels, the gain picture information includes the encoding information of the gain picture of each channel in the three channels, or the gain picture information includes the encoding information of the encapsulated image of the gain picture of the three channels, so that the problem of how to realize the encoding and decoding of the three-channel gain picture can be solved; and / or, the resolution of the first backup picture is determined to be the same as the resolution of the first reference picture, or the resolution of the backup picture is obtained from the metadata, and the first gain picture is applied based on the resolution of the backup picture, so that the problem of how to determine the resolution of the backup picture can be solved; and / or, the bit depth of the backup picture is obtained, and backup picture conversion is performed based on the bit depth of the backup picture, so that the problem of how to determine the bit depth of the backup picture can be solved; and / or, in the case that it is determined to perform color gamut space conversion, the first reference picture is converted, so that the problem of how to determine the processing timing of the conversion processing of the reference picture can be solved; and / or, the bit depth of the gain picture is obtained, and the first gain picture is applied on the first reference picture based on the bit depth of the gain picture, so that the problem of how to determine the bit depth of the gain picture can be solved. BRIEF DESCRIPTION OF DRAWINGS

[0067] FIG. 1 is a block diagram of a wireless communication system to which embodiments of the present application can be applied;

[0068] FIG. 2a is a schematic diagram of generating an HDR picture from an SDR picture combined with a gain picture according to an embodiment of the present application;

[0069] FIG. 2b is a schematic diagram of generating an SDR picture from an HDR picture combined with a gain picture according to an embodiment of the present application;

[0070] FIG. 3 is a schematic diagram of a calculation process of a gain picture according to an embodiment of the present application;

[0071] FIG. 4 is a flowchart of a gain picture application process according to an embodiment of the present application;

[0072] FIG. 5 is a flowchart of a gain picture application process including linear conversion and color gamut conversion according to an embodiment of the present application;

[0073] FIG. 6 is a block diagram of color gamut conversion without linear conversion and gain picture application according to an embodiment of the present application;

[0074] FIG. 7 is a flowchart of a backup picture generation method according to an embodiment of the present application;

[0075] FIG. 8 is a flowchart of an encoding method according to an embodiment of the present application;

[0076] FIG. 9 is a structural schematic diagram of a backup graph generation apparatus according to an embodiment of the present application;

[0077] FIG. 10 is a structural schematic diagram of an encoding apparatus according to an embodiment of the present application;

[0078] FIG. 11 is a structural schematic diagram of a communication device according to an embodiment of the present application;

[0079] FIG. 12 is a structural schematic diagram of a terminal according to an embodiment of the present application;

[0080] FIG. 13 is a structural schematic diagram of a network-side device according to an embodiment of the present application;

[0081] FIG. 14 is another structural schematic diagram of a network-side device according to an embodiment of the present application. DETAILED DESCRIPTION

[0082] The technical solutions in the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all the other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present application.

[0083] The terms "first", "second", and the like in the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than that illustrated or described herein, and the objects distinguished by "first", "second" are usually a category and do not limit the number of objects, for example, the first object can be one or more. In addition, "or" in the present application means at least one of the connected objects. For example, the protection scope of "A or B" at least covers three schemes, namely, scheme one: including A and not including B; scheme two: including B and not including A; scheme three: including A and B. In addition, the terms "A and / or B", "at least one of A and B", "at least one of A or B" also at least cover the above three schemes, respectively. The character " / " generally represents that the objects before and after are in an "or" relationship.

[0084] The term "indication" in the present application can be a direct indication (or explicit indication) or an indirect indication (or implicit indication). The direct indication can be understood as that the sender explicitly informs the receiver of the specific information, the operation to be performed or the request result, etc. in the indication sent by the sender. The indirect indication can be understood as that the receiver determines the corresponding information according to the indication sent by the sender, or judges and determines the operation to be performed or the request result, etc. according to the judgment result.

[0085] It is worth noting that the technology described in the embodiments of the present application is not limited to Long Term Evolution (LTE) / LTE-Advanced (LTE-A) systems, but can also be used in other wireless communication systems, such as Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal Frequency Division Multiple Access (OFDMA), Single-carrier Frequency-Division Multiple Access (SC-FDMA) or other systems. The terms "system" and "network" are often used interchangeably in the embodiments of the present application, and the described technology can be used in the above-mentioned systems and radio technologies, as well as in other systems and radio technologies. The following description describes a New Radio (NR) system for example purposes, and NR terminology is used in most of the following description, but these technologies can also be applied to systems other than NR systems, such as 6th Generation (6G) communication systems. th

[0086] ​FIG. 1 shows a block diagram of a wireless communication system to which embodiments of the present application can be applied. The wireless communication system includes a terminal 11 and a network-side device 12. The terminal 11 can be a terminal-side device such as a mobile phone, a Tablet Personal Computer, a Laptop Computer, a notebook computer, a Personal Digital Assistant (PDA), a palmtop computer, a netbook, an Ultra-mobile Personal Computer (UMPC), a Mobile Internet Device (MID), an Augmented Reality (AR) device, a Virtual Reality (VR) device, a robot, a wearable device, a flight vehicle, a Vehicle User Equipment (VUE), a shipboard device, a Pedestrian User Equipment (PUE), a smart home (a home device with a wireless communication function such as a refrigerator, a television, a washing machine, or furniture), a game console, a Personal Computer (PC), a kiosk, or a self-service machine. The wearable device includes a smart watch, a smart bracelet, a smart earphone, smart glasses, smart jewelry (a smart bracelet, a smart necklace, a smart ring, a smart necklace, a smart anklet, a smart necklace, etc.), a smart wristband, smart clothes, etc. The vehicle-mounted device can also be referred to as a vehicle-mounted terminal, a vehicle-mounted controller, a vehicle-mounted module, a vehicle-mounted component, a vehicle-mounted chip, or a vehicle-mounted unit, etc. It should be noted that the specific type of the terminal 11 is not limited in the embodiments of the present application. The network-side device 12 can include an access network device or a core network device. The access network device can also be referred to as a Radio Access Network (RAN) device, a radio access network function, or a radio access network unit. The access network device can include a base station, a Wireless Local Area Network (WLAN) Access Point (AP), or a Wireless Fidelity (WiFi) node, etc.The base station can be referred to as a Node B (NB), an evolved Node B (eNB), a next generation Node B (gNB), a New Radio Node B (NR Node B), an access point, a relay station (RBS), a serving base station (SBS), a base transceiver station (BTS), a radio base station, a radio transceiver, a basic service set (BSS), an extended service set (ESS), a home Node B (HNB), a home evolved Node B, a transmit / receive point (TRP), or some other suitable terminology in the art, and is not limited to a particular technical terminology, provided that the same technical effect is achieved. It should be noted that in the embodiments of the present application, only the base station in the NR system is taken as an example for introduction, and the specific type of the base station is not limited.

[0087] The core network device can also be referred to as a core network node, a core network function, or a core network network element, etc., which includes but is not limited to at least one of the following: a mobility management entity (MME), an access and mobility management function (AMF), a session management function (SMF), a user plane function (UPF), a policy control function (PCF), a policy and charging rules function (PCRF), an edge application server discovery function (EASDF), a unified data management (UDM), a unified data repository (UDR), a home subscriber server (HSS), a centralized network configuration (CNC), a network repository function (NRF), a network exposure function (NEF), a local NEF (L-NEF), a binding support function (BSF), an application function (AF), a location management function (LMF), a gateway mobile location center (GMLC), a network data analytics function (NWDAF), etc. It should be noted that only the core network device in the NR system is taken as an example for introduction in the embodiments of the present application, and the specific type of the core network device is not limited. If the name of the core network device mentioned in the embodiments of the present application changes in the subsequent protocol version (for example, 6G), it is also within the protection scope of the present application.

[0088] Optionally, the core network device can be implemented by one or more function modules in one device, or can be implemented by multiple devices jointly, and the embodiments of the present application do not make a specific limitation in this regard. It can be understood that the function modules described above can be network elements in a hardware device, can be software function modules running on a dedicated hardware, or can be virtualized function modules instantiated on a platform (for example, a cloud platform).

[0089] For the convenience of understanding, some contents related to the embodiments of the present application are explained as follows:

[0090] 1. HDR high dynamic range imaging based on gain map

[0091] With the progress of image acquisition and display technology, high dynamic range imaging and display can be completed on display devices such as smart phones, tablets and computer displays. Although there are more and more devices supporting high dynamic range still image display, there are still a large number of electronic devices supporting only standard dynamic range (SDR). Taking into account the high dynamic range image shooting can be displayed on high dynamic range (HDR) display devices and SDR display devices at the same time, and good display effect is presented, the HDR acquisition, storage and display scheme based on gain map is a method commonly used in related technologies. The gain map is usually generated when the HDR image is acquired, and can be stored as a specific file format, such as an image format or additional information of a certain image format, and then encoded, packaged and transmitted, as an additional information. When the display device is applied, the gain map information is decoded. The gain map is usually generated based on the HDR image and the corresponding tone-mapped SDR image, and the use of the gain map can be SDR and gain map to generate the corresponding HDR image, or HDR and gain map to generate the corresponding SDR image, as shown in FIG. 2a and FIG. 2b. In these two formats, the gain map information needs to be stored. Therefore, the gain map information as additional information will increase the storage space of the image. The SDR image in FIG. 2a and the HDR image in FIG. 2b are usually referred to as reference images, and the images generated by applying the gain map to the reference images are referred to as backup images, such as the HDR image in FIG. 2a and the SDR image in FIG. 2b.

[0092] 2. Calculation, downsampling and normalization processing of gain map

[0093] The calculation of the gain map can refer to FIG. 3. The gain map is generated based on the luminance values of the backup map and the reference map, for example, the gain map can be the ratio of the corresponding luminance values (e.g., corresponding pixel values) of the backup map and the reference map. Before calculating the ratio of the two luminance values, the backup map and the reference map in a certain bit depth can be converted to a linear domain; the backup map and the reference map in a certain bit depth can also be converted to a corresponding Perceptual Quantization (PQ) domain or Hybrid Log-gamma (HLQ) domain or gamma domain based on perceptual quantization (PQ) or hybrid log-gamma (HLQ) or gamma. The numerical value of the reference map needs to be scaled and aligned in some cases, so that the floating point number corresponding to the 3 RGB color values of the reference diffuse white is 1.0. The normalization of the reference map or the integer-to-floating point number processing, and the floating point number-to-integer conversion require the bit depth information of the reference map.

[0094] The gain map is used as a multiplier when applied, where a value of "1" means that the backup map and the reference map keep the luminance values unchanged, a value less than 1 means that the backup map is darkened, and a value greater than 1 means that the backup map is brightened.

[0095] In an implementation, the gain map is the ratio of the corresponding luminance values of the linear domain backup map and the reference map, and the ratio value is Log2, that is: G = log2((L alternate +k alternate ) / (L baseline +k baseline )

[0096] where L alternate is the luminance value of the backup map, L baseline is the luminance value of the reference map, k alternate and k baseline are non-zero constants. The gain map can be single-channel or multi-channel. Single-channel gain map means that the same gain map is applied to R / G / B three channels. Multi-channel, for example three-channel gain map, means that each R, G, B channel has its own gain map.

[0097] In order to reduce the storage space of the gain map, the gain map can be down-sampled, for example, the length and width of the gain map are each down-sampled by 1 / 2. The gain map after down-sampling is normalized. In the gain map normalization and inverse normalization operations, the maximum value, the minimum value and the encoding bit depth information in the current gain map are required. The normalization operation is as follows: G 归一化 =(G-G 最小值 ) / (G 最大值 -G 最小值

[0098] ​Where G is the gain map value to be normalized, G 归一化 is the value after normalization of G 最小值 is the minimum value of the current gain map, G 最大值 is the maximum value of the current gain map. In practical applications, the gain map value after normalization will also be subjected to an optional gamma correction operation, that is:

[0099] G 归一化_gamma = G 归一化 gamma

[0100] If gamma is 1, it means no gamma correction is performed.

[0101] The gain map value after normalization will be encoded with a bit depth of N (N can be 8, 10, 12, etc.), which scales the normalized gain map value to the range of 0 to 2 N -1. If N is 8, the scaling range is 0 to 255.

[0102] 3. Application of gain map

[0103] The application process of the gain map is shown in FIG. 4. The reference map and the gain map are usually encoded as JPEG / HEIF / AVIF or other supported image formats at the acquisition end. When applied, the reference map and the gain map are first decoded to obtain the decoded reference map and gain map, respectively, as well as the corresponding metadata. The decoded reference map needs to be converted to the linear domain or to the PQ or HLG or gamma domain. After the reference map is converted from the nonlinear domain to the linear domain, it is usually subjected to normalization or scaling to convert the pixel value of the reference map from an integer to a floating-point number.

[0104] When the gain map is displayed, the bit depth information of the gain map is needed, and the decoded gain map value G with a bit depth of N needs to be converted to a floating-point number, for example, G float = G / (2 N -1). Based on the gamma value in the metadata, inverse gamma operation is performed, and based on the maximum and minimum values of the gain map in the metadata, inverse normalization operation is performed to obtain the gain map value in the log2 domain.

[0105] If the gain map is down-sampled before compression and encoding, the decoded gain map value G log2An upsampling operation is performed so that the resolution of the gain map is the same as the resolution of the reference image. Then, depending on the capabilities of the display device and the current display environment, the gain map is applied on the reference image, the gain map values are scaled, and a linear domain or a PQ or HLG or gamma domain backup image is generated. For each color component of the reference image, the backup image is generated by combining the reference image and the gain map in the linear RGB space.

[0106] The linear domain or the PQ or HLG or gamma domain backup image needs to be encoded to the display according to the supported bit depth N of the display, scaled to the range of 0 to 2 N -1. If N is 10, the scaling range is 0 to 1023.

[0107] In the related art, in the flowchart of FIG. 5, only the conversion of the reference image to the linear domain is given. This module can also be the conversion of the PQ or HLG or gamma domain.

[0108] 4. Color gamut space conversion

[0109] In the gain map metadata, there are three metadata related to the color gamut space, which are the color gamut space of the reference image, the color gamut space of the backup image, and the color gamut space of the gain map application. Among them, the color gamut spaces of the reference image and the backup image can use Exif colorSpace Tag, ICC profile, or coding-independent code points (CICP) metadata, etc. to indicate the color encoding and color space of the reference image and the backup image. The color gamut space metadata of the gain map application indicates the color space when the gain map is applied, which can be one of the color space of the reference image and the color space of the backup image.

[0110] FIG. 6 is a block diagram of gain map application containing possible color space conversion modules. The dashed box in FIG. 6 is two optional modules, only one of which will be executed. When the color gamut space of the gain map application is the color gamut space of the backup image, and the color gamut space of the reference image is different from the color gamut space of the backup image, the color gamut space of the reference image is converted to the color gamut space of the backup image before the gain map is applied. When the color gamut space of the gain map application is the color gamut space of the reference image, and the color gamut space of the reference image is different from the color gamut space of the backup image, the newly obtained linear color gamut space of the backup image is converted to the color space specified by the color gamut space metadata of the backup image after the gain map is applied.

[0111] 5. Gain map metadata encapsulation and gain map storage

[0112] In the related art, gain map metadata is stored in a binary metadata structure. The byte order of the data structure is big-endian, that is, the lowest address stores the most significant byte. For example, the data structure is as follows:

[0113] GainMapChannel specifies the metadata of each channel gain map. If it is a single channel, the metadata is applied to the RGB three channels. If it is a multi-channel, the R, G, and B three channels have their respective channel metadata.

[0114] In the JPEG encapsulation format, the gain map adopts the CIPA Multi-Picture Format (CIPA DC-007) to store the gain map and its metadata. The reference map contains the application marker 2 (APP2 marker) in Table 1, and the corresponding information in the GainMapVersion structure in the metadata (Metadata) is stored. The reference map is the first individual image stored in the CIPA Multi-Picture Format (CIPA DC-007) format.

[0115] Table 1

[0116] Wherein, Unique header label refers to a unique header label.

[0117] Null-termination character refers to a null-termination character.

[0118] Metadata refers to metadata.

[0119] When using the JPEG format, the gain map data is compressed by JPEG, stored after the reference map using the CIPA DC-007 Multi-Picture Format, and the MP type of the gain map complies with the definition of the gain map type in CIPA DC-007. The gain map contains the APP2 marker in Table 1. The CIPA DC-007 Multi-Picture Format supports storing multiple other individual images after the reference map.

[0120] The following describes a backup map generation method, an encoding method, an apparatus, and related equipment provided by embodiments of the present application in detail in combination with some embodiments and application scenarios thereof and the accompanying drawings.

[0121] Referring to FIG. 7, FIG. 7 is a flowchart of a backup map generation method provided by an embodiment of the present application, which can be executed by a decoding end device. As shown in FIG. 7, the backup map generation method comprises the following steps:

[0122] Step 101: decode the reference map information and the gain map information respectively to obtain a first reference map and a first gain map;

[0123] Step 102: perform a first operation based on the first reference map and the first gain map, and generate a first backup map based on the first operation;

[0124] In a case where the first gain map is a gain map of three channels, the gain map information comprises encoded information of a gain map of each of the three channels, or the gain map information comprises encoded information of a packed image of the gain maps of the three channels.

[0125] And / or,

[0126] The first operation satisfies at least one of the following conditions:

[0127] It is determined that the resolution of the first backup map is the same as the resolution of the first reference map;

[0128] A backup map resolution is obtained from metadata, and the first gain map is applied based on the backup map resolution;

[0129] A backup map bit depth is obtained, and backup map conversion is performed based on the backup map bit depth;

[0130] In a case where it is determined to perform color gamut space conversion, the first reference map is converted;

[0131] A gain map bit depth is obtained, and the first gain map is applied on the first reference map based on the gain map bit depth.

[0132] The gain map can also be translated as an enhancement map, and the gain map can also be expressed as an enhancement map.

[0133] In an implementation, when the first gain map is applied, it can be determined that the resolution of the first backup map is the same as the resolution of the first reference map.

[0134] The first reference map can also be expressed as a reference map, a decoded reference map, or a decoded reference map. The first gain map can also be expressed as a gain map, a decoded gain map, or a decoded gain map.

[0135] The backup image resolution can be obtained from the gain image metadata or from the baseline image metadata, and the like, which are not limited in the embodiment.

[0136] In an embodiment, the baseline image information can include the encoding information of the second baseline image (or referred to as source baseline image); or, the baseline image information can include the encoding information of the second baseline image and the encoding information of the baseline image metadata.

[0137] In an embodiment, the gain image information can include the encoding information of the second gain image (or referred to as source gain image); or, the gain image information can include the encoding information of the second gain image and the encoding information of the gain image metadata.

[0138] In an embodiment, the decoding of the baseline image information and the gain image information respectively to obtain the first baseline image and the first gain image includes:

[0139] The decoding of the baseline image information and the gain image information and the metadata thereof respectively to obtain the first baseline image and the first gain image and the corresponding metadata includes:

[0140] The first operation based on the first baseline image and the first gain image and the first backup image generated based on the first operation includes:

[0141] The first operation based on the first baseline image and the first gain image and the corresponding metadata and the first backup image generated based on the first operation.

[0142] In an embodiment, the decoding of the baseline image information and the gain image information respectively to obtain the first baseline image and the first gain image includes:

[0143] The decoding of the baseline image information, the gain image information, the encoding information of the gain image metadata and the encoding information of the baseline image metadata respectively to obtain the first baseline image, the first gain image, the gain image metadata and the baseline image metadata includes:

[0144] The first operation based on the first baseline image and the first gain image and the first backup image generated based on the first operation includes:

[0145] The first operation based on the first baseline image, the first gain image, the gain image metadata and the baseline image metadata and the first backup image generated based on the first operation.

[0146] In an embodiment, the first operation includes applying the gain image, and in the process of applying the gain image, it is determined that the resolution of the first backup image is the same as the resolution of the first baseline image.

[0147] In an implementation, the first operation includes applying the gain map, and the applying the gain map includes: obtaining a backup map resolution from the metadata, and applying the first gain map based on the backup map resolution.

[0148] In an implementation, the first operation includes: after generating a linear backup map by applying the gain map, obtaining a backup map bit depth, and performing backup map conversion based on the backup map bit depth.

[0149] In an implementation, the first operation includes: before applying the gain map, performing conversion processing on the first reference map in a case where it is determined to perform color gamut space conversion; or, after applying the gain map, performing conversion processing on the first reference map in a case where it is determined to perform color gamut space conversion.

[0150] In an implementation, the first operation includes applying the gain map, and the applying the gain map includes: obtaining a gain map bit depth, and applying the first gain map on the first reference map based on the gain map bit depth.

[0151] It should be noted that in the related art, in the process of applying the gain map, the length and width of the gain map and the reference map need to be compared. If the length and width of the gain map are smaller than the length and width of the reference map, the gain map needs to be up-sampled so that the length and width of the gain map and the reference map are the same, and then the gain map is applied. In the metadata definition, the resolution of the backup map is not limited. In this way, different implementations may have different results on the resolution of the backup map, which may cause the gain map to have poor universality in the application of the HDR image.

[0152] In the embodiments of the present application, it is determined that the resolution of the first backup map is the same as the resolution of the first reference map, which can solve the problem of how to determine the resolution of the backup map in the case of missing metadata.

[0153] As shown in FIG. 3, generating a gain map requires a source alternate image and a source reference image. The relationship between the source alternate image and the source reference image when generating the gain map is not defined in the related art. In an embodiment, the source alternate image and the source reference image have the same resolution and are in the same color space before generating the gain map. The reference image and the generated gain map are stored in a JPEG or HEIF, AVIF or other image format after being compressed and encoded. In the metadata of the image format, the resolution information such as the length and width of the reference image and the gain map is encoded. When the gain map is applied or the gain map HDR image is edited on the display side, the resolution information such as the length and width of the reference image and the gain map can be obtained from the metadata of the image format or the auxiliary information of the image format. The alternate image is not encoded with the reference image and the gain map, so the resolution information of the alternate image cannot be obtained when the gain map is edited or applied on the display side. In order to clarify the gain map application operation, the embodiments of the present application determine that the resolution of the alternate image is the same as the resolution of the decoded reference image when the gain map is applied.

[0154] For example, the source alternate image and the source reference image are used to generate the gain map at the shooting end (or when editing the HDR image). After the gain map is generated, the gain map can be subjected to down-sampling, normalization, encoding, decoding, up-sampling, and inverse normalization, and the reference image can be subjected to encoding and decoding. When the gain map is applied on the display side, the generated alternate image is an approximation of the source alternate image, not exactly the same, but a reconstructed alternate image. If the gain map is not down-sampled and a lossless encoding and decoding scheme is used, the source gain map can be completely reconstructed on the display side. Considering that the storage space required for lossless encoding is larger than that for lossy encoding, the gain map is usually subjected to lossy encoding and decoding processing.

[0155] It should be noted that when the gain map is applied, the linear domain alternate image needs to be sent to the display for display. When the alternate image is sent to the display, it needs to be converted from a linear domain floating point number to an RGB image with a bit depth of N. The bit depth N is affected by two factors, namely the bit depth supported by the display device and the bit depth of the alternate image when the gain map is generated. In the related art, the bit depth of the alternate image is not constrained and defined. For example, when the gain map is generated, the bit depth of the alternate image is 12 bits, but when the bit depth supported by the display device is only 10 bits, the alternate image obtained by applying the gain map needs to be converted to 10 bits. If the display end does not know that the source alternate image for generating the gain map is 12 bits, artifacts such as banding effects may occur during the conversion process.

[0156] In the embodiments of the present application, the backup picture bit depth is acquired, and backup picture conversion is performed based on the backup picture bit depth. For example, the backup picture bit depth can be acquired from gain map metadata, or the backup picture bit depth can be selected according to the maximum bit depth supported by the electronic device, thereby solving the problem of how to determine the backup picture bit depth.

[0157] It should be noted that in the related art, the gain map can be single-channel or three-channel. When the gain map uses three channels, the gain of each channel is usually important. When the three-channel gain map is compressed and encoded, if JPEG or HEVC encoding is used, the RGB data needs to be converted to YCbCr / YUV 4:2:0. After conversion, due to downsampling, the gain map in the G and B channels loses half of the information, which causes the quality of the reconstructed backup picture to decrease at the display end.

[0158] In the embodiments of the present application, in the case where the first gain map is a three-channel gain map, the gain map information includes the encoding information of the gain map of each channel of the three channels, or the gain map information includes the encoding information of the encapsulated image of the gain map of the three channels, thereby realizing the encoding and encapsulation method of the three-channel gain map.

[0159] It can be understood that the gain map can be single-channel, in which case the same gain map is applied to the three channels of RGB. The gain map can also be three-channel, which is suitable for processing content with rich colors and contrasts. The gain map is the ratio of the brightness of the backup picture and the reference picture, and is not a traditional grayscale or color image. In order to facilitate integration into the existing image encoding process, the gain map is compressed and decompressed using an image encoder, just like the reference picture. In a scenario suitable for a three-channel gain map, the gain values of the three channels are different and are applied to the three channels of RGB respectively. In the current JPEG or HEIF or AVIF image format, the RGB data can be converted to YCbCr 4:2:0 format or YCbCr 4:2:2 format and then compressed and encoded by an image encoder. If the gain map is three-channel, and encapsulated into an image according to the RGB color components, the gain map corresponding to the red R channel is encapsulated into the R channel of the RGB image format, the gain map corresponding to the green G channel is encapsulated into the G channel of the RGB image format, and the gain map corresponding to the blue B channel is encapsulated into the B channel of the RGB image format. After conversion from RGB to YCbCr, the gain information of the G channel is not lost, but the gain information of the R and B channels is lost by half. Although the YCbCr is upsampled to RGB after decoding, part of the gain information of the R and B channels is lost, which affects the quality of the generated backup picture.

[0160] In one embodiment, the three-channel gain map can be processed as three gray-scale images, for example, the encoding and decoding order is R, G, B channel. Each channel gain map is encoded and decoded as an independent gray-scale image. In this processing case, no additional down-sampling is performed on the single channel gain map, more information of the gain map is preserved, and the quality of the generated backup map is guaranteed.

[0161] In one embodiment, if the gain map is three-channel, and the three-channel gain map is processed as an RGB image, only one three-channel gain map is encapsulated and decoded. If the gain map is three-channel, and the three-channel gain map is three gray-scale images, the three-channel gain map is encapsulated and decoded separately, and the order is R, G, B or other agreed processing order. For JPEG encoding, in order to obtain high reconstruction gain map and backup map quality, three gray-scale images can be selected to encode and encapsulate the gain map. For HEVC encoding or other encoding that supports chroma component 4:4:4 format, no down-sampling is performed on the chroma component, and the three-channel gain map can still be fused into an RGB image for encoding and encapsulation. In order to obtain smaller gain map storage space, the three-channel gain map can be fused into an RGB image for encoding, and the chroma component sampling format is 4:2:0 or 4:2:2.

[0162] Compared with the scheme of encoding each channel as a gray-scale image, the scheme of encoding three channels as an RGB image occupies less storage space.

[0163] The decision of whether to use the scheme of encoding each channel as a gray-scale image or the scheme of encoding three channels as an RGB image can be made by considering storage space and image quality.

[0164] It should be noted that in the color gamut conversion part in the related art, as shown in FIG. 6, the operation between the reference image and the color gamut conversion module is not described in the block diagram and description. In the current module, the reference image is subjected to color gamut conversion after JPEG or HEIF or AVIF decoding. Then, based on the reference image, the gain map, the metadata, and the current display capability, the gain map is applied. The application of the gain map should be performed in the linear domain or the PG or HLG or gamma domain. The current description and block diagram do not have related operations.

[0165] In the embodiments of the present application, the first reference picture is converted when it is determined that color gamut space conversion is to be performed. For example, when the gain map application color gamut space is the spare picture color gamut space and the reference picture color gamut space is different from the spare picture color gamut space, the first reference picture is converted; the converted first reference picture is subjected to color gamut space conversion; the first spare picture is generated by applying the first gain map to the color gamut space converted first reference picture. Or, when the gain map application color gamut space is the reference picture color gamut space and the reference picture color gamut space is different from the spare picture color gamut space, the first reference picture is converted; the first spare picture is generated by applying the first gain map to the converted first reference picture. The problem that the reference picture needs to be linearly converted or subjected to PQ or HLG or gamma domain conversion when color gamut conversion is needed before or after the gain map is applied can be solved.

[0166] It should be noted that in the related art, the bit depth quantization of the gain map is defined in the gain map metadata. When the gain map is applied, there is no gain map bit depth quantization metadata in the decoded gain map metadata. When the gain map is subjected to floating point number operation, the bit depth information is missing.

[0167] In the embodiments of the present application, the gain map bit depth is acquired, and the first gain map is applied to the first reference picture based on the gain map bit depth.

[0168] In the embodiments of the present application, the reference map information and the gain map information are decoded respectively to obtain a first reference map and a first gain map; a first operation is performed based on the first reference map and the first gain map, and a first backup map is generated based on the first operation, so that the backup map can be generated by applying the gain map to the reference map. Further, in the case that the first gain map is a gain map of three channels, the gain map information includes the encoding information of the gain map of each channel in the three channels, or the gain map information includes the encoding information of the encapsulated image of the gain map of the three channels, so that the problem of how to realize the encoding and decoding of the three-channel gain map can be solved; and / or, the resolution of the first backup map is determined to be the same as the resolution of the first reference map, or the backup map resolution is obtained from the metadata, and the first gain map is applied based on the backup map resolution, so that the problem of how to determine the resolution of the backup map can be solved; and / or, the backup map bit depth is obtained, and backup map conversion is performed based on the backup map bit depth, so that the problem of how to determine the backup map bit depth can be solved; and / or, in the case of determining to perform color gamut space conversion, the first reference map is converted, so that the problem of how to determine the processing timing of the conversion processing of the reference map can be solved; and / or, the gain map bit depth is obtained, and the first gain map is applied on the first reference map based on the gain map bit depth, so that the problem of how to determine the gain map bit depth can be solved.

[0169] Optionally, the obtaining the backup map bit depth comprises:

[0170] The backup map bit depth is obtained from gain map metadata, and the gain map information includes the gain map metadata.

[0171] In this embodiment, the backup map bit depth is obtained from gain map metadata, and the gain map information includes the gain map metadata, so that the problem of how to determine the backup map bit depth can be solved.

[0172] Optionally, the backup map bit depth is stored in the gain map metadata in a binary structure.

[0173] The backup map bit depth in the gain map metadata can be backup map bit depth metadata.

[0174] In order to restore and generate a backup image with high precision when the gain map is applied, the bit depth of the source backup image when the gain map is generated needs to be known. How to obtain the bit depth of the source backup image is a problem to be solved. In the embodiments of the present application, the bit depth of the source backup image can be stored as a metadata in the gain map metadata. When the gain map is applied, the bit depth of the backup image can be obtained by decoding from the gain map metadata. Based on the bit depth N of the backup image, the linear backup image generated by applying the gain map can be converted to an RGB image with a bit depth of N before the backup image is displayed, for example, the linear backup image is converted from a floating point number to an integer with a bit depth of N. If the bit depth N is less than or equal to the bit depth supported by the display device, it can be directly displayed. If the bit depth N is greater than the bit depth M supported by the display device, the N-bit backup image can be converted to an M-bit image for display, and a specific algorithm can be introduced in the conversion process to reduce possible artifacts such as banding.

[0175] For example, the source backup image is 12 bits, and the display device only supports 10 bits. The linear domain backup image obtained by applying the gain map can be converted to a 12-bit RGB image, and then the 12-bit RGB image is converted to a 10-bit RGB image by a specific algorithm and sent to the display device for display. Another implementation method is to directly convert the linear domain backup image obtained by applying the gain map to a 10-bit RGB image and send it to the display device for display, but the probability of artifact is high.

[0176] The bit depth information of the backup image can be stored by a binary structure, as follows:

[0177] alternate_bit_depth indicates the bit depth of the backup image, for example, 000 is 8 bits, 001 is 10 bits, 010 is 12 bits, 011 is 14 bits, 100 is 16 bits, and other values are reserved bits. In other embodiments, the bit of alternate_bit_depth can be selected based on the number of alternate_bit_depth bit depths that need to be supported, for example, if there are only 4 choices, 2 bits can be used to represent unsigned int(2) alternate_bit_depth.

[0178] Optionally, the obtaining the backup image bit depth comprises:

[0179] In the case where the backup image bit depth is not included in the gain map metadata, the backup image bit depth is selected according to the maximum bit depth supported by the electronic device, and the gain map information comprises the gain map metadata;

[0180] The selected backup image bit depth is less than or equal to the maximum bit depth supported by the electronic device.

[0181] In addition, the electronic device can be a display device.

[0182] In an embodiment, the selecting the backup picture bit depth according to the maximum bit depth supported by the electronic device comprises: obtaining a reference picture bit depth, and selecting the backup picture bit depth based on the reference picture bit depth and the maximum bit depth supported by the electronic device.

[0183] The reference picture bit depth can be a bit depth of a first reference picture.

[0184] In an embodiment, if there is no backup picture bit depth information in the gain map metadata, when the reference picture is an SDR picture, the backup picture bit depth can be selected according to the following rules: selecting a value greater than the SDR bit depth and less than or equal to the bit depth supported by the display device. When the reference picture is an HDR picture, selecting a value less than the HDR bit depth and not greater than the bit depth supported by the display device.

[0185] In this embodiment, in the case where the backup picture bit depth is not included in the gain map metadata, the backup picture bit depth is selected according to the maximum bit depth supported by the electronic device, which can solve the problem of how to determine the backup picture bit depth. Further, the selected backup picture bit depth is less than or equal to the maximum bit depth supported by the electronic device, so that the selected backup picture bit depth can be applied to the electronic device.

[0186] Optionally, in the case where the first reference picture is a standard dynamic range (SDR) picture, the selected backup picture bit depth is greater than the bit depth of the first reference picture; or,

[0187] In the case where the first reference picture is a high dynamic range (HDR) picture, the selected backup picture bit depth is less than the bit depth of the first reference picture.

[0188] Optionally, the gain map information comprises gain map metadata, the gain map metadata comprises the gain map bit depth, and a data type of the gain map bit depth is binary metadata.

[0189] Or,

[0190] The gain map information comprises gain map auxiliary information in an image format, and the gain map auxiliary information comprises the gain map bit depth.

[0191] The gain map bit depth in the gain map metadata can be in the form of gain map bit depth metadata. The gain map bit depth metadata can be used to store the bit depth information of the gain map. In an embodiment, the bit depth of the gain map is at least 8 bits. The bit depth of the gain map can be different from the bit depth of the reference picture or the backup picture. The gain map bit depth metadata can be stored in a binary structure, as shown below:

[0192] gain_map_quantization is used to indicate the bit depth of the gain map, for example, 000 is 8bit, 001 is 10bit, 010 is 12bit, 011 is 14bit, 100 is 16bit, and other values are reserved bits. Embodiments of the present application give the binary encapsulation structure of the gain map bit depth metadata in the gain map metadata. In other embodiments, the bit of gain_map_quantization can be selected based on the number of supported gain_map_quantization bit depths, for example, if there are only 4 choices, 2 bits can be used to represent unsigned int(2) gain_map_quantization.

[0193] In an embodiment, the gain map bit depth is not stored separately by binary structure metadata, but is stored in the auxiliary information of the image format, such as the auxiliary information of the JPEG or HEIF or AVIF image format. The bit depth information of the gain map is obtained by decoding the auxiliary information of the JPEG or HEIF or AVIF image format. The bit depth information of the gain map is used when the gain map is displayed. The decoded gain map value G with bit depth N is converted to a floating point number G float N .

[0194] In this embodiment, the gain map information includes gain map metadata, and the gain map metadata includes the gain map bit depth, and the data type of the gain map bit depth is binary metadata; or, the gain map information includes gain map auxiliary information of an image format, and the gain map auxiliary information includes the gain map bit depth; in this way, the gain map bit depth can be obtained from the gain map metadata or the gain map auxiliary information, and the problem of how to determine the gain map bit depth can be solved.

[0195] Optionally, the first operation is performed based on the first reference map and the first gain map, and the first backup map is generated based on the first operation, including:

[0196] In the case where the color gamut space of the gain map application is the color gamut space of the backup map, and the color gamut space of the reference map is different from the color gamut space of the backup map, the first reference map is converted;

[0197] The converted first reference map is converted in color gamut space;

[0198] The first backup map is generated by applying the first gain map to the first reference map converted in color gamut space.

[0199] ​The gain map application color gamut space can be understood or replaced as a color gamut space to which the gain map is applied, or a color gamut space to which the gain map is applied. The gain map application color gamut space can refer to a color gamut space to which the first gain map is applied.

[0200] In addition, the backup map color gamut space can be understood or replaced as a color gamut space of the backup map. The backup map color gamut space can refer to a color gamut space of the first backup map.

[0201] In addition, the reference map color gamut space can be understood or replaced as a color gamut space of the reference map. The reference map color gamut space can refer to a color gamut space of the first reference map.

[0202] In an embodiment, based on the gain map metadata information, when color space conversion is needed, it is determined that the gain map is applied in the linear domain, or in the PQ or HLG or gamma domain.

[0203] For example, when the gain map application color gamut space is the backup map color gamut space, and the reference map color gamut space is different from the backup map color gamut space, the reference map and the gain map are converted from integer to floating point based on the reference map bit depth and the gain map bit depth, respectively, linear domain conversion is performed before the gain map is applied, or PQ or HLG or gamma domain conversion is performed, as shown in FIG. 5, color gamut space conversion is then performed, and then the gain map is applied to generate the final reconstructed backup map.

[0204] In addition, the linear domain conversion, such as gamma correction on the RGB image, is inverse gamma correction on the decoded RGB image.

[0205] In this embodiment, when the gain map application color gamut space is the backup map color gamut space, and the reference map color gamut space is different from the backup map color gamut space, the first reference map is converted; the first reference map after the conversion is converted in color gamut space; and the first gain map is applied to the first reference map after the color gamut space conversion to generate the first backup map, thereby solving the problem of how to determine the processing timing of the conversion processing of the reference map.

[0206] Optionally, the first operation is performed based on the first reference map and the first gain map, and the first backup map is generated based on the first operation, which includes:

[0207] In the case where the gain map application color gamut space is the reference map color gamut space, and the reference map color gamut space is different from the backup map color gamut space, the first reference map is converted;

[0208] The first gain map is applied to the first reference map after the conversion;

[0209] The first reference map after the application of the first gain map is converted in color gamut space to generate the first backup map.

[0210] The gain map application color gamut space, the backup map color gamut space, and the reference map color gamut space can refer to the descriptions of the gain map application color gamut space, the backup map color gamut space, and the reference map color gamut space described above, and details are not repeated here.

[0211] In an embodiment, based on the gain map metadata information, when color space conversion is needed, it is determined that the gain map is applied in a linear domain or in a PQ or HLG or gamma domain.

[0212] When the gain map application color gamut space is the reference map color gamut space and the reference map color gamut space is different from the backup map color gamut space, after the decoded reference map is obtained, the reference map and the gain map are converted from integers to floating points based on the reference map bit depth and the gain map bit depth, linear domain conversion or PQ or HLG or gamma domain conversion is performed, the gain map is applied, and then the reconstructed backup map color space generated by applying the gain map is converted to the color space specified by the backup map color space metadata to generate a final reconstructed backup map, as shown in FIG. 5.

[0213] In addition, the linear domain conversion, for example, gamma correction of an RGB image, is inverse gamma correction of the decoded RGB image.

[0214] In this embodiment, when the gain map application color gamut space is the reference map color gamut space and the reference map color gamut space is different from the backup map color gamut space, the first reference map is converted, the first gain map is applied to the converted first reference map, and the first reference map after the first gain map is applied is converted in the color gamut space to generate the first backup map, thereby solving the problem of how to determine the processing timing of the conversion processing of the reference map.

[0215] Optionally, the conversion processing includes at least one of the following:

[0216] integer-to-floating-point conversion;

[0217] linear conversion, perceptual quantization PQ domain conversion, hybrid logarithm HLG domain conversion, or gamma domain conversion.

[0218] The conversion processing of the first reference map can include converting the decoded first gain map from an integer to a floating point based on the bit depth of the first gain map, converting the decoded first reference map from an integer to a floating point based on the bit depth of the first reference map, and performing linear conversion, PQ domain conversion, HLG domain conversion, or gamma domain conversion on the first reference map.

[0219] Optionally, the first operation further satisfies any one of the following:

[0220] In a case where the gain chart application color gamut space is the backup chart color gamut space and the reference chart color gamut space is different from the backup chart color gamut space, it is determined to convert the reference chart color gamut space to the backup chart color gamut space;

[0221] In a case where the gain chart application color gamut space is the reference chart color gamut space and the reference chart color gamut space is different from the backup chart color gamut space, it is determined to convert the newly obtained reconstructed linear backup chart color gamut space to the color space specified by the backup chart color gamut space metadata.

[0222] In a case where the reference chart color gamut space is the same as the backup chart color gamut space, no color gamut space conversion is performed.

[0223] The gain chart application color gamut space, the backup chart color gamut space, and the reference chart color gamut space can refer to the descriptions of the gain chart application color gamut space, the backup chart color gamut space, and the reference chart color gamut space described above, and will not be described here again.

[0224] It should be noted that the processing flow chart with color gamut space conversion is shown in FIG. 6. According to the application color space information of the gain chart metadata, optional color space conversion is performed before or after the application of the gain chart.

[0225] In an embodiment, when the gain chart application color gamut space is the backup chart color gamut space and the reference chart color gamut space is different from the backup chart color gamut space, the reference chart color gamut space is converted to the backup chart color gamut space before the application of the gain chart.

[0226] In an embodiment, when the gain chart application color gamut space is the reference chart color gamut space and the reference chart color gamut space is different from the backup chart color gamut space, the newly obtained reconstructed linear backup chart color gamut space is converted to the color space specified by the backup chart color gamut space metadata after the application of the gain chart.

[0227] In an embodiment, when the reference chart color gamut space is the same as the backup chart color gamut space, no color gamut space conversion is performed before or after the application of the gain chart.

[0228] In addition, when color space conversion is required, it is determined whether the application of the gain chart is performed in a linear domain or a PQ or HLG or gamma domain.

[0229] In this embodiment, in the case that the gain map application color gamut space is the backup map color gamut space and the reference map color gamut space is different from the backup map color gamut space, it is determined to convert the reference map color gamut space to the backup map color gamut space; or, in the case that the gain map application color gamut space is the reference map color gamut space and the reference map color gamut space is different from the backup map color gamut space, it is determined to convert the obtained reconstructed linear backup map color gamut space to the color space indicated by the backup map color gamut space metadata; or, in the case that the reference map color gamut space is the same as the backup map color gamut space, no color gamut space conversion is performed. In this way, how to determine the color gamut space conversion and how to perform the color gamut space conversion can be realized.

[0230] Optionally, the encoding information of the gain map of each of the three channels includes: encoding information obtained after a gray-scale image of the gain map of each of the three channels is encoded respectively.

[0231] Or,

[0232] The encoding information of the encapsulated image of the gain map of the three channels includes: encoding information obtained after the gain map of the three channels is encapsulated and the encapsulated image is encoded.

[0233] In this case, there are two encoding and encapsulation methods:

[0234] A): the gray-scale images of the gain map of each of the three channels are encoded respectively, for example, the gain maps G_R, G_G, and G_B of the three channels are encoded and encapsulated separately;

[0235] B): the gain maps of the three channels are encapsulated and the encapsulated image is encoded, for example, the gain maps G_R, G_G, and G_B of the three channels are encapsulated into an image (similar to a color image including RGB three channels), in JPEG or HEVC encoding, RGB is converted into YCbCr format, and then the image is encoded and encapsulated as a three-channel image, wherein the luminance component corresponds to the gain map of the G channel, the chroma components correspond to the gain maps of the R and B channels, or the Y luminance channel corresponds to G_G, the chroma channel Cb corresponds to the gain map G_B, and the chroma channel Cr corresponds to the gain map G_R.

[0236] Compared with the three-channel gain map, the single-channel gain map refers to only one single-channel gain map G, and in the application of the gain map, the gain maps G of each channel of RGB are completely the same, and only one single-channel gain map G is encoded and decoded.

[0237] Referring to FIG. 8, FIG. 8 is a flowchart of an encoding method provided by an embodiment of the present application, which can be executed by an encoding terminal device. As shown in FIG. 8, the encoding method comprises the following steps:

[0238] Step 201: generating a second gain map based on the second reference map and the second backup map;

[0239] Step 202: encoding the second reference map and the second gain map respectively to obtain reference map information and gain map information;

[0240] The gain map information comprises a backup map bit depth; and / or,

[0241] In the case that the second gain map is a gain map of three channels, the gain map information comprises encoding information of the gain map of each of the three channels; or, the gain map information comprises encoding information of a packaged image of the gain map of the three channels; and / or,

[0242] The gain map information comprises a gain map bit depth.

[0243] The second reference map can also be referred to as a source reference map. The second backup map can also be referred to as a source backup map. The second gain map can also be referred to as a source gain map.

[0244] The encoding of the second reference map and the second gain map respectively to obtain reference map information and gain map information can comprise: encoding the second reference map to obtain reference map information; and encoding the second gain map to obtain gain map information. The reference map information can be used to obtain the first reference map by decoding. The gain map information can be used to obtain the first gain map by decoding.

[0245] Optionally, the gain map information comprises gain map metadata, and the backup map bit depth is stored in the gain map metadata in a binary structure.

[0246] Optionally, the gain map information comprises gain map metadata, and the gain map metadata comprises the gain map bit depth. The data type of the gain map bit depth is binary metadata.

[0247] Or,

[0248] The gain map information comprises gain map auxiliary information in an image format, and the gain map auxiliary information comprises the gain map bit depth.

[0249] Optionally, the encoding information of the gain map of each of the three channels comprises encoding information obtained by respectively encoding a grayscale image of the gain map of each of the three channels.

[0250] Or,

[0251] The gain map of the three channels encapsulates the encoding information of the encapsulated image, including: encapsulating the gain map of the three channels, and obtaining the encoding information after encoding the encapsulated image.

[0252] It should be noted that the embodiment is an implementation of the corresponding encoding end device in the embodiment shown in FIG. 7, and the specific implementation can refer to the related description of the embodiment shown in FIG. 7. To avoid repetition, the embodiment will not be described again.

[0253] The execution subject of the backup graph generation method provided in the embodiment of the application can be a backup graph generation device. In the embodiment of the application, the backup graph generation method is executed by the backup graph generation device as an example to illustrate the backup graph generation device provided in the embodiment of the application.

[0254] The backup graph generation device provided in the embodiment of the application can be a communication device or a component in the communication device, for example, a chip, as an example. The communication device can be a terminal, a network side device, a server, or the like. Exemplarily, the terminal can include but is not limited to the types of the terminal 11 listed above, the network side device can include but is not limited to the types of the network side device 12 listed above, and the embodiment of the application does not make specific limitations.

[0255] The backup graph generation device includes a receiving module, a sending module, and a processing module. The receiving module, the sending module, and the processing module can be implemented by software or by hardware. When implemented by hardware, the processing module can be implemented by a processor, which can include a general-purpose processor, a special-purpose processor, or the like, for example, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), an artificial intelligent (AI) processor, a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a network processor (NP), a field programmable gate array (FPGA), or other programmable logic devices, gate circuits, transistors, discrete hardware components, or the like. The receiving module and the sending module can be implemented by a communication interface, which can include one or more of a transceiver, a pin, a circuit, a bus, a radio frequency unit, or the like.

[0256] Specifically, referring to FIG. 9, when the backup map generation apparatus is a decoding end device or a component in the decoding end device, the backup map generation apparatus 300 includes:

[0257] a processing module 301, configured to respectively decode the reference map information and the gain map information to obtain a first reference map and a first gain map;

[0258] The processing module 301 is further configured to perform a first operation based on the first reference map and the first gain map, and generate a first backup map based on the first operation.

[0259] In a case where the first gain map is a gain map of three channels, the gain map information includes encoded information of a gain map of each of the three channels, or the gain map information includes encoded information of a packed image of the gain maps of the three channels.

[0260] And / or,

[0261] The first operation satisfies at least one of the following conditions:

[0262] The resolution of the first backup map is determined to be the same as the resolution of the first reference map.

[0263] The backup map resolution is obtained from metadata, and the first gain map is applied based on the backup map resolution.

[0264] The backup map bit depth is obtained, and backup map conversion is performed based on the backup map bit depth.

[0265] In a case where it is determined to perform color gamut space conversion, the first reference map is converted.

[0266] The gain map bit depth is obtained, and the first gain map is applied on the first reference map based on the gain map bit depth.

[0267] Optionally, the processing module is specifically configured to:

[0268] The backup map bit depth is obtained from gain map metadata, and the gain map information includes the gain map metadata.

[0269] Optionally, the backup map bit depth is stored in the gain map metadata in a binary structure.

[0270] Optionally, the processing module is specifically configured to:

[0271] In a case where the backup map bit depth is not included in the gain map metadata, the backup map bit depth is selected according to the maximum bit depth supported by the electronic device, and the gain map information includes the gain map metadata.

[0272] The selected backup image bit depth is less than or equal to a maximum bit depth supported by the electronic device.

[0273] Optionally, in a case where the first reference image is a standard dynamic range (SDR) image, the selected backup image bit depth is greater than a bit depth of the first reference image; or,

[0274] In a case where the first reference image is a high dynamic range (HDR) image, the selected backup image bit depth is less than a bit depth of the first reference image.

[0275] Optionally, the gain map information includes gain map metadata, and the gain map metadata includes the gain map bit depth, and a data type of the gain map bit depth is binary metadata.

[0276] Or,

[0277] The gain map information includes gain map auxiliary information in an image format, and the gain map auxiliary information includes the gain map bit depth.

[0278] Optionally, the processing module is specifically configured to:

[0279] In a case where a gain map application color gamut space is a backup image color gamut space, and a reference image color gamut space is different from the backup image color gamut space, performing conversion processing on the first reference image;

[0280] Performing color gamut space conversion on the first reference image after the conversion processing;

[0281] Applying the first gain map to the first reference image after the color gamut space conversion to generate a first backup image.

[0282] Optionally, the processing module is specifically configured to:

[0283] In a case where a gain map application color gamut space is a reference image color gamut space, and a reference image color gamut space is different from a backup image color gamut space, performing conversion processing on the first reference image;

[0284] Applying the first gain map to the first reference image after the conversion processing;

[0285] Performing color gamut space conversion on the first reference image after the first gain map is applied to generate a first backup image.

[0286] Optionally, the conversion processing includes at least one of the following:

[0287] Integer-to-floating-point conversion;

[0288] Linear conversion, perceptual quantization (PQ) domain conversion, hybrid logarithmic (HLG) domain conversion, or gamma domain conversion.

[0289] Optionally, the first operation further satisfies any one of the following conditions:

[0290] In a case where the gamut space of the gain map is the backup map gamut space and the reference map gamut space is different from the backup map gamut space, it is determined to convert the reference map gamut space to the backup map gamut space;

[0291] In a case where the gamut space of the gain map is the reference map gamut space and the reference map gamut space is different from the backup map gamut space, it is determined to convert the obtained reconstructed linear backup map gamut space to the color space indicated by the backup map gamut space metadata;

[0292] In a case where the reference map gamut space is the same as the backup map gamut space, no gamut space conversion is performed.

[0293] Optionally, the encoding information of the gain map of each of the three channels includes encoding information obtained by respectively encoding a grayscale image of the gain map of each of the three channels.

[0294] Or,

[0295] The encoding information of the encapsulated image of the gain map of the three channels includes encoding information obtained by encoding an image obtained by encapsulating the gain map of the three channels.

[0296] The backup map generation apparatus provided in the embodiments of the present application can implement each process implemented by the method embodiment of FIG. 7 and achieve the same technical effects. To avoid repetition, the same will not be described here.

[0297] The encoding method provided in the embodiments of the present application can be performed by an encoding apparatus. In the embodiments of the present application, the encoding apparatus is taken as an example to describe the encoding apparatus provided in the embodiments of the present application.

[0298] The encoding apparatus provided in the embodiments of the present application can be a communication device or a component in a communication device, for example, a chip. The communication device can be a terminal, a network side device or a server, etc. For example, the terminal can include but is not limited to the types of the terminal 11 listed above, the network side device can include but is not limited to the types of the network side device 12 listed above, and the embodiments of the present application are not limited in this regard.

[0299] The encoding device includes a receiving module, a sending module and a processing module. The receiving module, the sending module and the processing module can be implemented by software or by hardware. When implemented by hardware, the processing module can be implemented by a processor, which can include a general-purpose processor, a special-purpose processor, etc., such as a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), an artificial intelligent (AI) processor, a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a network processor (NP), a field programmable gate array (FPGA) or other programmable logic devices, a gate circuit, a transistor, a discrete hardware component, etc. The receiving module and the sending module can be implemented by a communication interface, which can include one or more of a transceiver, a pin, a circuit, a bus, a radio frequency unit, etc.

[0300] Referring to FIG. 10, when the encoding device is an encoding end device or a component in the encoding end device, the encoding device 400 includes:

[0301] The processing module 401 is configured to generate a second gain map based on the second reference map and the second backup map.

[0302] The processing module 401 is further configured to encode the second reference map and the second gain map respectively to obtain reference map information and gain map information.

[0303] The gain map information includes a backup map bit depth; and / or

[0304] When the second gain map is a gain map of three channels, the gain map information includes encoding information of the gain map of each channel of the three channels respectively; or the gain map information includes encoding information of a packaged image of the gain map of the three channels; and / or

[0305] The gain map information includes a gain map bit depth.

[0306] Optionally, the gain map information includes gain map metadata, and the backup map bit depth is stored in the gain map metadata in a binary structure.

[0307] Optionally, the gain map information comprises gain map metadata, and the gain map metadata comprises the gain map bit depth, and a data type of the gain map bit depth is binary metadata.

[0308] Or,

[0309] The gain map information comprises gain map auxiliary information in an image format, and the gain map auxiliary information comprises the gain map bit depth.

[0310] Optionally, the encoding information of the gain map of each of the three channels comprises: encoding information obtained by respectively encoding a grayscale image of the gain map of each of the three channels.

[0311] Or,

[0312] The encoding information of the encapsulated image of the gain map of the three channels comprises: encoding information obtained by encoding an image obtained by encapsulating the gain map of the three channels.

[0313] The backup map generation apparatus provided in the embodiments of the present application can implement each process implemented by the method embodiment of Figure 8 and achieve the same technical effects. To avoid repetition, details are not described herein.

[0314] As shown in Figure 11, the embodiments of the present application further provide a communication device 500, comprising a processor 501 and a memory 502, and the memory 502 stores programs or instructions executable on the processor 501. For example, when the communication device 500 is a terminal, the programs or instructions are executed by the processor 501 to implement each step of the backup map generation method embodiment described above and achieve the same technical effects. When the communication device 500 is a network side device, the programs or instructions are executed by the processor 501 to implement each step of the encoding method embodiment described above and achieve the same technical effects. To avoid repetition, details are not described herein.

[0315] The embodiments of the present application further provide a terminal comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is configured to run programs or instructions to implement the steps in the method embodiments shown in Figure 7 or Figure 8. The terminal embodiment corresponds to the encoding end device or decoding end device method embodiment described above, and each implementation process and implementation manner of the method embodiment described above can be applied to the terminal embodiment and achieve the same technical effects. The terminal can be the apparatus shown in Figure 9 or Figure 10. Specifically, Figure 12 is a hardware structure schematic diagram of a terminal implementing the embodiments of the present application.

[0316] The terminal 600 includes, but is not limited to, at least part of components such as a radio frequency unit 601, a network module 602, an audio output unit 603, an input unit 604, a sensor 605, a display unit 606, a user input unit 607, an interface unit 608, a memory 609, and a processor 610.

[0317] Those skilled in the art can understand that the terminal 600 can further include a power supply (such as a battery) for supplying power to each component, and the power supply can be logically connected to the processor 610 through a power management system, so as to realize functions such as management of charging, discharging, and power consumption management through the power management system. The terminal structure shown in FIG. 12 does not constitute a limitation on the terminal, and the terminal can include more or fewer components than those shown, or combine certain components, or different component arrangements, which are not described here.

[0318] It should be understood that in the embodiments of the present application, the input unit 604 can include a graphics processor 6041 and a microphone 6042. The graphics processor 6041 processes image data of a still picture or a video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 606 can include a display panel 6061, which can be configured in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit 607 includes at least one of a touch panel 6071 and other input devices 6072. The touch panel 6071 is also called a touch screen. The touch panel 6071 can include two parts of a touch detection device and a touch controller. The other input devices 6072 can include, but are not limited to, a physical keyboard, function keys (such as volume control keys, on-off keys, etc.), trackballs, mice, joysticks, which are not described here.

[0319] In the embodiments of the present application, after the radio frequency unit 601 receives downlink data from a network side device, the radio frequency unit 601 can transmit the downlink data to the processor 610 for processing. In addition, the radio frequency unit 601 can send uplink data to the network side device. Generally, the radio frequency unit 601 includes, but is not limited to, an antenna, an amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc.

[0320] The memory 609 can be used to store software programs or instructions and various data. The memory 609 can mainly include a first storage area storing programs or instructions and a second storage area storing data, wherein the first storage area can store an operating system, application programs or instructions required by at least one function (such as a sound playing function, an image playing function, etc.), and the like. In addition, the memory 609 can include a volatile memory or a non-volatile memory. The non-volatile memory can be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a Random Access Memory (RAM), a Static RAM (SRAM), a Dynamic RAM (DRAM), a Synchronous DRAM (SDRAM), a Double Data Rate SDRAM (DDR SDRAM), an Enhanced SDRAM (ESDRAM), a Synch link DRAM (SLDRAM), and a Direct Rambus RAM (DRRAM). The memory 609 in the embodiments of the present application includes but is not limited to these and any other suitable types of memory.

[0321] The processor 610 can include one or more processing units; optionally, the processor 610 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and an application program, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 610.

[0322] In the case where the terminal is a decoding end device:

[0323] The processor 610 is configured to decode the reference map information and the gain map information respectively to obtain a first reference map and a first gain map.

[0324] The processor 610 is further configured to perform a first operation based on the first reference map and the first gain map, and generate a first backup map based on the first operation.

[0325] In a case where the first gain map is a gain map of three channels, the gain map information respectively includes encoded information of a gain map of each of the three channels, or the gain map information includes encoded information of a packaged image of the gain maps of the three channels.

[0326] and / or,

[0327] The first operation satisfies at least one of the following:

[0328] Determine that the resolution of the first backup map is the same as the resolution of the first reference map;

[0329] Obtain a backup map resolution from metadata, and apply the first gain map based on the backup map resolution;

[0330] Obtain a backup map bit depth, and perform backup map conversion based on the backup map bit depth;

[0331] In a case where it is determined to perform color gamut space conversion, perform conversion processing on the first reference map;

[0332] Obtain a gain map bit depth, and apply the first gain map on the first reference map based on the gain map bit depth.

[0333] Optionally, the processor 610 is specifically configured to:

[0334] Obtain a backup map bit depth from gain map metadata, and the gain map information includes the gain map metadata.

[0335] Optionally, the backup map bit depth is stored in the gain map metadata in a binary structure.

[0336] Optionally, the processor 610 is specifically configured to:

[0337] In a case where the backup map bit depth is not included in the gain map metadata, select a backup map bit depth according to a maximum bit depth supported by an electronic device, and the gain map information includes the gain map metadata;

[0338] The selected backup map bit depth is less than or equal to the maximum bit depth supported by the electronic device.

[0339] Optionally, in a case where the first reference map is a standard dynamic range (SDR) image, the selected backup map bit depth is greater than a bit depth of the first reference map; or,

[0340] In a case where the first reference map is a high dynamic range (HDR) image, the selected backup map bit depth is less than a bit depth of the first reference map.

[0341] Optionally, the gain map information comprises gain map metadata, and the gain map metadata comprises the gain map bit depth, and a data type of the gain map bit depth is binary metadata.

[0342] Or,

[0343] The gain map information comprises gain map auxiliary information in an image format, and the gain map auxiliary information comprises the gain map bit depth.

[0344] Optionally, the processor 610 is specifically configured to:

[0345] In a case where the gain map application color gamut space is the backup map color gamut space and the reference map color gamut space is different from the backup map color gamut space, performing conversion processing on the first reference map;

[0346] Performing color gamut space conversion on the first reference map after the conversion processing;

[0347] Applying the first gain map to the first reference map after the color gamut space conversion to generate a first backup map.

[0348] Optionally, the processor 610 is specifically configured to:

[0349] In a case where the gain map application color gamut space is the reference map color gamut space and the reference map color gamut space is different from the backup map color gamut space, performing conversion processing on the first reference map;

[0350] Applying the first gain map to the first reference map after the conversion processing;

[0351] Performing color gamut space conversion on the first reference map after the first gain map is applied to generate a first backup map.

[0352] Optionally, the conversion processing comprises at least one of the following:

[0353] Integer-to-floating-point conversion;

[0354] Linear conversion, perceptual quantization (PQ) domain conversion, hybrid logarithmic (HLG) domain conversion, or gamma domain conversion.

[0355] Optionally, the first operation further satisfies any one of the following:

[0356] In a case where the gain map application color gamut space is the backup map color gamut space and the reference map color gamut space is different from the backup map color gamut space, determining to convert the reference map color gamut space to the backup map color gamut space;

[0357] In a case where the color gamut space of the gain map is the reference map color gamut space, and the reference map color gamut space is different from the backup map color gamut space, the reconstructed linear backup map color gamut space obtained is converted to the color space indicated by the backup map color gamut space metadata;

[0358] In a case where the reference map color gamut space is the same as the backup map color gamut space, no color gamut space conversion is performed.

[0359] Optionally, the encoded information of the gain map of each of the three channels includes encoded information obtained by respectively encoding a gray image of the gain map of each of the three channels.

[0360] Or,

[0361] The encoded information of the encapsulated image of the gain map of the three channels includes encoded information obtained by encoding an image obtained by encapsulating the gain map of the three channels.

[0362] In a case where the terminal is an encoding end device:

[0363] The processor 610 is configured to generate a second gain map based on a second reference map and a second backup map.

[0364] The processor 610 is further configured to encode the second reference map and the second gain map respectively to obtain reference map information and gain map information.

[0365] The gain map information includes a backup map bit depth; and / or,

[0366] In a case where the second gain map is a gain map of three channels, the gain map information includes encoded information of a gain map of each of the three channels; or, the gain map information includes encoded information of an encapsulated image of the gain map of the three channels; and / or,

[0367] The gain map information includes a gain map bit depth.

[0368] Optionally, the gain map information includes gain map metadata, and the backup map bit depth is stored in the gain map metadata in a binary structure.

[0369] Optionally, the gain map information includes gain map metadata, and the gain map metadata includes the gain map bit depth, and a data type of the gain map bit depth is binary metadata.

[0370] Or,

[0371] The gain map information includes gain map auxiliary information in an image format, and the gain map auxiliary information includes the gain map bit depth.

[0372] Optionally, the encoding information of the gain map of each of the three channels comprises: encoding information obtained by respectively encoding a gray scale image of the gain map of each of the three channels.

[0373] Or,

[0374] The encoding information of the encapsulated image of the gain map of the three channels comprises: encoding information obtained by encoding an image obtained by encapsulating the gain map of the three channels.

[0375] It can be understood that the implementation process of each implementation mode mentioned in the embodiment can refer to the related description of the method embodiment of FIG. 7 or FIG. 8, and achieve the same or corresponding technical effects. To avoid repetition, it will not be repeated here.

[0376] The embodiment of the application further provides a network side device, comprising a processor and a communication interface, the communication interface and the processor are coupled, the processor is used to run programs or instructions, and the steps of the method embodiment shown in FIG. 7 or FIG. 8 are realized. The network side device embodiment corresponds to the above-mentioned encoding end device or decoding end device method embodiment, and each implementation process and implementation mode of the above-mentioned method embodiment can be applied to the network side device embodiment, and the same technical effects can be achieved.

[0377] Specifically, the embodiment of the application further provides a network side device, which can be the apparatus shown in FIG. 9 or FIG. 10. As shown in FIG. 13, the network side device 700 comprises an antenna 701, a radio frequency device 702, a baseband device 703, a processor 704 and a memory 705. The antenna 701 is connected with the radio frequency device 702. In the uplink direction, the radio frequency device 702 receives information through the antenna 701, and sends the received information to the baseband device 703 for processing. In the downlink direction, the baseband device 703 processes the information to be sent, and sends it to the radio frequency device 702. The radio frequency device 702 processes the received information and sends it out through the antenna 701.

[0378] The method performed by the network side device in the above embodiment can be implemented in the baseband device 703, which comprises a baseband processor.

[0379] The baseband device 703 may, for example, comprise at least one baseband board, and a plurality of chips are arranged on the baseband board, as shown in FIG. 13. One of the chips is, for example, a baseband processor, which is connected with the memory 705 through a bus interface to call the programs in the memory 705 and perform the network device operations shown in the above method embodiment.

[0380] The network side device can further include a network interface 706, for example, a common public radio interface (CPRI).

[0381] Specifically, the network side device 700 of the embodiment of the present application further includes instructions or programs stored on the memory 705 and executable on the processor 704, the processor 704 invokes the instructions or programs in the memory 705 to execute the method performed by each module shown in FIG. 9 or FIG. 10 and achieve the same technical effects. To avoid repetition, details are not described herein.

[0382] Specifically, the embodiment of the present application further provides a network side device. As shown in FIG. 14, the network side device 800 includes a processor 801, a network interface 802 and a memory 803. The network side device can be the backup map generation apparatus shown in FIG. 12. The network interface 802 is, for example, a common public radio interface (CPRI).

[0383] Specifically, the network side device 800 of the embodiment of the present application further includes instructions or programs stored on the memory 803 and executable on the processor 801, the processor 801 invokes the instructions or programs in the memory 803 to execute the method performed by each module shown in FIG. 9 or FIG. 10 and achieve the same technical effects. To avoid repetition, details are not described herein.

[0384] The embodiment of the present application further provides a readable storage medium, the readable storage medium stores programs or instructions, the programs or instructions are executed by a processor to implement each process of the above-mentioned backup map generation method or encoding method embodiment, and the same technical effects can be achieved. To avoid repetition, details are not described herein.

[0385] The processor is the processor in the terminal or the network side device in the above-mentioned embodiments. The readable storage medium includes a computer readable storage medium, such as a computer readable only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc. In some examples, the readable storage medium can be a non-transitory readable storage medium.

[0386] The embodiment of the present application further provides a chip, the chip includes a processor and a communication interface, the communication interface and the processor are coupled, the processor is used to run programs or instructions to implement each process of the above-mentioned backup map generation method or encoding method embodiment, and the same technical effects can be achieved. To avoid repetition, details are not described herein.

[0387] It should be understood that the chip mentioned in the embodiment of the present application can also be referred to as a system chip, a system on chip, a chip system or a system on chip, etc.

[0388] The embodiment of the present application further provides a computer program / product stored in a storage medium, which is executed by at least one processor to implement the processes of the above-mentioned backup graph generation method or encoding method embodiment, and can achieve the same technical effects. To avoid repetition, details are not described herein.

[0389] The embodiment of the present application further provides a wireless communication system, which comprises a decoding end device and an encoding end device. The decoding end device can be used to execute the steps of the backup graph generation method described above, and the encoding end device can be used to execute the steps of the encoding method described above.

[0390] It should be noted that in this document, the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusion, so that processes, methods, articles, or devices that include a series of elements not only include those elements, but also include other elements not explicitly listed, or further include elements inherent to such processes, methods, articles, or devices. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, but can also include performing functions in a substantially simultaneous manner or in reverse order, for example, the described method can be performed in an order different from that described, and various steps can also be added, omitted or combined. In addition, the features described with reference to certain examples can be combined in other examples.

[0391] From the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment method can be realized by means of computer software product and general hardware platform, of course, it can also be realized by hardware. The computer software product is stored in a storage medium (such as ROM, RAM, magnetic disc, optical disc, etc.), which includes a plurality of instructions for making the terminal or network side device execute the method described in each embodiment of the present application.

[0392] The embodiments of the present application are described above in combination with the drawings, but the present application is not limited to the above-mentioned specific embodiments, which are only illustrative and not restrictive. Those skilled in the art can make many forms of embodiments under the inspiration of the present application without departing from the scope of the present application and the protection scope of the claims.

Claims

1. A method for generating a backup image, comprising: decoding reference image information and gain image information to obtain a first reference image and a first gain image, respectively; performing a first operation based on the first reference image and the first gain image, and generating a first backup image based on the first operation; wherein, in a case where the first gain image is a gain image of three channels, the gain image information comprises encoded information of a gain image of each of the three channels, or the gain image information comprises encoded information of a packed image of the gain images of the three channels; and / or, the first operation satisfies at least one of the following conditions: determining that a resolution of the first backup image is the same as a resolution of the first reference image; obtaining a backup image resolution from metadata, and applying the first gain image based on the backup image resolution; obtaining a backup image bit depth, and performing backup image conversion based on the backup image bit depth; in a case where it is determined to perform color gamut space conversion, performing conversion processing on the first reference image; obtaining a gain image bit depth, and applying the first gain image on the first reference image based on the gain image bit depth.

2. The method of claim 1, wherein, the obtaining of the backup image bit depth comprises: obtaining the backup image bit depth from gain image metadata, the gain image information comprising the gain image metadata.

3. The method of claim 1 or 2, wherein, the backup image bit depth is stored in the gain image metadata in a binary structure.

4. The method of claim 1, wherein, the obtaining of the backup image bit depth comprises: in a case where the backup image bit depth is not included in the gain image metadata, selecting a backup image bit depth according to a maximum bit depth supported by an electronic device, the gain image information comprising the gain image metadata; wherein the selected backup image bit depth is less than or equal to the maximum bit depth supported by the electronic device.

5. The method of claim 4, wherein, in a case where the first reference image is a standard dynamic range (SDR) image, the selected backup image bit depth is greater than a bit depth of the first reference image; or in a case where the first reference image is a high dynamic range (HDR) image, the selected backup image bit depth is less than a bit depth of the first reference image.

6. The method of any one of claims 1-5, wherein, the gain image information comprises gain image metadata, the gain image metadata comprising the gain image bit depth, and a data type of the gain image bit depth is binary metadata; or the gain image information comprises gain image auxiliary information in an image format, the gain image auxiliary information comprising the gain image bit depth.

7. The method of any one of claims 1-6, wherein, the performing of the first operation based on the first reference image and the first gain image, and the generating of the first backup image based on the first operation, comprises: in a case where a gain image application color gamut space is a backup image color gamut space, and a reference image color gamut space is different from the backup image color gamut space, performing conversion processing on the first reference image; performing color gamut space conversion on the first reference image after the conversion processing; applying the first gain image on the first reference image after the color gamut space conversion to generate the first backup image.

8. The method of any one of claims 1-6, wherein, the performing of the first operation based on the first reference image and the first gain image, and the generating of the first backup image based on the first operation, comprises: in a case where a gain image application color gamut space is a reference image color gamut space, and a reference image color gamut space is different from a backup image color gamut space, performing conversion processing on the first reference image; applying the first gain image on the first reference image after the conversion processing; The first reference map after the first gain map is applied is color gamut space converted to generate a first backup map.

9. The method of any one of claims 1-8, wherein, The conversion process includes at least one of the following: integer to floating point conversion; linear conversion, perceptual quantization PQ domain conversion, hybrid logarithm HLG domain conversion, or gamma domain conversion.

10. The method of any one of claims 1-9, wherein, The first operation also satisfies any one of the following: In the case where the color gamut space of the gain map application is the color gamut space of the backup map, and the color gamut space of the reference map is different from the color gamut space of the backup map, it is determined that the color gamut space of the reference map is converted to the color gamut space of the backup map; In the case where the color gamut space of the gain map application is the color gamut space of the reference map, and the color gamut space of the reference map is different from the color gamut space of the backup map, it is determined that the obtained reconstructed linear backup map color gamut space is converted to the color space indicated by the backup map color gamut space metadata; In the case where the color gamut space of the reference map is the same as the color gamut space of the backup map, no color gamut space conversion is performed.

11. The method of any one of claims 1-10, wherein, The encoding information of the gain map of each channel of the three channels includes: the encoding information obtained by respectively encoding the gray scale images of the gain map of each channel of the three channels; Or, The encoding information of the packaged image of the gain map of the three channels includes: the encoding information obtained by encoding the image obtained by packaging the gain map of the three channels.

12. An encoding method, comprising: generating a second gain map based on a second reference map and a second backup map; encoding the second reference map and the second gain map respectively to obtain reference map information and gain map information; wherein the gain map information includes a backup map bit depth; and / or, in the case where the second gain map is a gain map of three channels, the gain map information respectively includes encoding information of the gain map of each channel of the three channels; or, the gain map information includes encoding information of a packaged image of the gain map of the three channels; and / or, the gain map information includes a gain map bit depth.

13. The method of claim 12, wherein, The gain map information includes gain map metadata, and the backup map bit depth is stored in the gain map metadata in a binary structure.

14. The method of claim 12 or 13, wherein, The gain map information includes gain map metadata, and the gain map metadata includes the gain map bit depth, and the data type of the gain map bit depth is binary metadata; Or, The gain map information includes gain map auxiliary information in the image format, and the gain map auxiliary information includes the gain map bit depth.

15. The method of any one of claims 12-14, wherein, The encoding information of the gain map of each channel of the three channels includes: the encoding information obtained by respectively encoding the gray scale images of the gain map of each channel of the three channels; Or, The encoding information of the packaged image of the gain map of the three channels includes: the encoding information obtained by encoding the image obtained by packaging the gain map of the three channels.

16. A backup map generation apparatus, comprising: a processing module configured to decode reference map information and gain map information respectively to obtain a first reference map and a first gain map; the processing module is further configured to perform a first operation based on the first reference map and the first gain map, and generate a first backup map based on the first operation. In a case where the first gain map is a gain map of three channels, the gain map information respectively comprises encoded information of a gain map of each of the three channels, or the gain map information comprises encoded information of a packaged image of the gain maps of the three channels. And / or, The first operation satisfies at least one of the following conditions: It is determined that the resolution of the first backup map is the same as the resolution of the first reference map; Backup map resolution is obtained from metadata, and the first gain map is applied based on the backup map resolution; Backup map bit depth is obtained, and backup map conversion is performed based on the backup map bit depth; In a case where it is determined to perform color gamut space conversion, the first reference map is converted; Gain map bit depth is obtained, and the first gain map is applied on the first reference map based on the gain map bit depth.

17. The apparatus of claim 16, wherein, The processing module is specifically configured to: Backup map bit depth is obtained from gain map metadata, and the gain map information comprises the gain map metadata.

18. The apparatus of claim 16, wherein, The processing module is specifically configured to: In a case where the backup map bit depth is not included in the gain map metadata, the backup map bit depth is selected according to the maximum bit depth supported by the electronic device, and the gain map information comprises the gain map metadata; Wherein, the selected backup map bit depth is less than or equal to the maximum bit depth supported by the electronic device.

19. The apparatus of any one of claims 16-18, wherein, The processing module is specifically configured to: In a case where the gain map application color gamut space is the backup map color gamut space, and the reference map color gamut space is different from the backup map color gamut space, the first reference map is converted; The first reference map after conversion is converted in color gamut space; The first reference map after color gamut space conversion is applied with the first gain map to generate the first backup map.

20. The apparatus of any one of claims 16-18, wherein, The processing module is specifically configured to: In a case where the gain map application color gamut space is the reference map color gamut space, and the reference map color gamut space is different from the backup map color gamut space, the first reference map is converted; The first reference map after conversion is applied with the first gain map; The first reference map after applying the first gain map is converted in color gamut space to generate the first backup map.

21. An encoding apparatus, comprising: a processing module configured to generate a second gain map based on a second reference map and a second backup map; the processing module is further configured to respectively encode the second reference map and the second gain map to obtain reference map information and gain map information; wherein the gain map information comprises backup map bit depth; and / or, In a case where the second gain map is a gain map of three channels, the gain map information respectively comprises encoded information of a gain map of each of the three channels, or the gain map information comprises encoded information of a packaged image of the gain maps of the three channels; and / or, The gain map information comprises gain map bit depth.

22. A communication device comprising a processor and a memory, the memory storing programs or instructions executable on the processor, the programs or instructions being executed by the processor to implement the steps of the backup map generation method according to any one of claims 1-11, or to implement the steps of the encoding method according to any one of claims 12-15.

23. A readable storage medium, on which a program or instructions are stored, which, when executed by a processor, implement the steps of the spare map generation method according to any one of claims 1-11, or implement the steps of the encoding method according to any one of claims 12-15.

24. A computer program / program product, which, when executed by at least one processor, implement the steps of the spare map generation method according to any one of claims 1-11, or implement the steps of the encoding method according to any one of claims 12-15.

Images