Image encoding and decoding method, device, electronic device, and storage medium

Abstract

The embodiments of the present application provide an image encoding and decoding method, device, electronic device and storage medium, in which the image encoding method includes: acquiring a video image, identifying image characteristics of the video image, respectively identifying in-loop identification information and out-loop identification information according to the image characteristics, processing the video image according to the in-loop identification information and the out-loop identification information, encoding the processed video image to generate a code stream, and writing the in-loop identification information and the out-loop identification information into the code stream. The embodiments of the present application jointly realize the processing of video image outside the encoding-decoding loop and inside the encoding-decoding loop according to image characteristics in an image transmission process, so as to effectively improve the efficiency of video encoding and decoding, and improve the decoded image quality at the same code rate.

Description

[Technical field]

[0001] This application claims priority to a Chinese patent application bearing application number 202110420800.9 filed with the China Patent Office on April 19, 2021, the entire contents of which are incorporated herein by reference.

[0002] The present application relates to the technical field of image processing, for example, to an image encoding / decoding method, device, electronic device, and storage medium. [Background technology]

[0003] With the development of information technology, the Internet traffic in the world is rapidly increasing, among which the traffic in the form of video accounts for about 80% of the total traffic. Video exists in every aspect of life in various forms, including commonly seen broadcast television, Internet media, video calling, video conferencing, video surveillance, etc. The development of network bandwidth, display and storage technologies has further promoted the development of high quality display of video, giving birth to High Dynamic Range (HDR) video. HDR video has a larger exposure dynamic range, and compared with traditional video, HDR video can more accurately represent the real world and bring users a better experience. The Joint Video Expert Teams (JVET) under the International Organization for Standardization (ISO) / International Electrotechnical Commission (IEC) is working on the formulation of international video encoding and decoding standards and has adopted videos converted by Hybrid Log Gamma (HLG) and Perceptual Quantizer (PQ) technologies as universal test sequences for HDR videos. However, HDR videos have a wider luminance range and therefore occupy more storage space, so more efficient video encoding and decoding technologies are needed to process HDR videos.

[0004] The luma and chroma mapping transformation can be performed inside and outside the video encoding / decoding loop, where the encoding / decoding loop includes an encoding loop and a decoding loop, and the processing process in the loop is the process in which the conventional encoding / decoding tools are performed, among which the encoding / decoding tools can include block division, intra prediction, inter prediction, transform and quantization, loop filtering, etc. The inside of the encoding / decoding loop may be abbreviated as inside the loop, and the outside of the encoding / decoding loop may be abbreviated as outside the loop. Currently, the next-generation video encoding / decoding international standard published by JVET adopts the inside-loop luma mapping with chroma scaling (LMCS) technology, which maps the luma component of the video image and scales the chroma component depending on the mapping of the luma component, thereby realizing the purpose of saving code streams and improving coding efficiency. Meanwhile, the inside-loop LMCS technology has problems of poor adaptability and low coding efficiency. In the process of formulating the international video coding standard in the past, an out-of-loop luma and chroma mapping technology (called a reshaper) was proposed to pre-process the input video before applying it to video coding, perform luma and chroma mapping conversion, and then post-process the output video after decoding to restore the original video. The out-of-loop reshaper cannot cooperate with block division, intra prediction, inter prediction, transform and quantization, loop filtering, and other technologies in the video coding and decoding process, so the coding performance cannot be ensured. Currently, there is an urgent need for an encoding and decoding method that uses out-of-loop and in-loop luma and chroma mapping conversion in a combined manner to effectively improve coding efficiency. Summary of the Invention [Problem to be solved by the invention]

[0005] The embodiments of the present application mainly aim to propose an image encoding / decoding method, device, electronic device, and storage medium that aim to effectively improve video encoding / decoding efficiency and improve decoded image quality at the same code rate by realizing an encoding / decoding method that supports the combined use of out-loop and in-loop luma and chromaticity mapping transformations. [Means for solving the problem]

[0006] The embodiment of the present application is The present invention provides an image encoding method including the steps of obtaining a video image and identifying image characteristics of the video image, identifying in-loop identification information and out-of-loop identification information according to the image characteristics, respectively, processing the video image according to the in-loop identification information and the out-of-loop identification information, encoding the processed video image to generate a code stream, and writing the in-loop identification information and the out-of-loop identification information into the code stream.

[0007] The embodiment of the present application is The present invention further provides an image decoding method, including the steps of obtaining a codestream, analyzing to obtain in-loop identification information and out-loop identification information in the codestream, processing the codestream according to the in-loop identification information and the out-loop identification information, and decoding the processed codestream to generate a video image.

[0008] The embodiment of the present application is an image capture module configured to capture a video image and identify image characteristics of the video image; an identification module configured to identify in-loop identification information and out-of-loop identification information, respectively, in response to the image characteristics; an image processing module configured to process the video images according to the in-loop identification information and the out-of-loop identification information; and an encoding processing module configured to encode the processed video images to generate a codestream, and to write the in-loop identification information and the out-of-loop identification information into the codestream.

[0009] The embodiment of the present application is a codestream analysis module configured to obtain a codestream and analyze the codestream to obtain in-loop identification information and out-of-loop identification information; a codestream processing module configured to process the codestream in accordance with the in-loop identification information and the out-of-loop identification information; and an image decoding module configured to decode the processed codestream to generate a video image.

[0010] The embodiment of the present application is one or more processors; a memory configured to store one or more programs; The present invention further provides an electronic device in which, when the one or more programs are executed by the one or more processors, the one or more processors realize an image encoding method and / or an image decoding method described in any of the embodiments of the present application.

[0011] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored therein, the computer program storing a computer program being executed by a processor to realize an image encoding method and / or an image decoding method according to any of the embodiments of the present application. Effect of the Invention

[0012] The embodiment of the present application identifies image characteristics of the acquired video image by the encoding side, uses the image characteristics to identify in-loop identification information and out-loop identification information, processes the video image according to the in-loop identification information and the out-loop identification information, encodes the processed video image into a code stream, and writes the in-loop identification information and the out-loop identification information into the code stream. The embodiment of the present application supports the combined use of out-loop and in-loop luma and chroma mapping transformation, thereby effectively improving video encoding and decoding efficiency and improving decoded image quality at the same code rate. [Brief description of the drawings]

[0013] [Figure 1] FIG. 2 is a schematic diagram of an encoding / decoding process in one loop according to an embodiment of the present application. [Diagram 2] FIG. 2 is a schematic diagram of an out-of-loop encoding-decoding process according to an embodiment of the present application; [Diagram 3] 1 is a flowchart of an image encoding method according to an embodiment of the present application. [Figure 4] 10 is a flowchart of another image encoding method according to an embodiment of the present application. [Diagram 5] 10 is a flowchart of another image encoding method according to an embodiment of the present application. [Figure 6] 10 is a flowchart of another image encoding method according to an embodiment of the present application. [Figure 7] 10 is a flowchart of another image encoding method according to an embodiment of the present application. [Figure 8] FIG. 1 illustrates an image encoding method according to an embodiment of the present application. [Figure 9] FIG. 10 illustrates another image encoding method according to an embodiment of the present application. [Figure 10] 1 is a flowchart of an image decoding method according to an embodiment of the present application; [Figure 11] FIG. 2 illustrates an image decoding method according to an embodiment of the present application. [Figure 12]1 is a structural schematic diagram of an image encoding device according to an embodiment of the present application; [Figure 13] FIG. 2 is a structural schematic diagram of an image decoding device according to an embodiment of the present application. [Figure 14] 1 is a structural schematic diagram of an electronic device according to an embodiment of the present application. [Figure 15] FIG. 2 is a structural schematic diagram of an encoder according to an embodiment of the present application. [Figure 16] FIG. 2 is a structural schematic diagram of an encoder according to an embodiment of the present application. [Figure 17] FIG. 2 is a structural schematic diagram of one decoder according to an embodiment of the present application; [Figure 18] FIG. 2 is a structural schematic diagram of one decoder according to an embodiment of the present application; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] It should be understood that the specific embodiments described herein are merely for the purpose of illustrating the present application, and are not intended to limit the present application.

[0015] In the following description, suffixes such as "module", "component" or "unit" for indicating elements are merely used to facilitate the description of the present application and have no specific meaning in themselves. Therefore, "module", "component" or "unit" may be used in combination. In the following specification, the present application will be described in detail in conjunction with the embodiments with reference to the drawings. If there is no contradiction, the embodiments and features in the embodiments in the present application may be combined with each other. In addition, the terms "first", "second", "third", "fourth", etc. in the specification of the present application are used to distinguish similar objects and do not describe a specific order or chronological order.

[0016] FIG. 1 is a schematic diagram of an encoding and decoding process in one loop according to an embodiment of the present application, and FIG. 1 shows a schematic diagram of luma mapping and chroma scaling in the loop. Luma mapping and chroma scaling are newly added processing processes before loop filtering, and include two main parts, one is to realize loop mapping of luma component using a self-adaptive piecewise linear model, and the other is to process chroma component using luma-dependent chroma residual scaling. As shown in FIG. 1, the newly added processing processes included in the mapping area include inverse quantization, inverse transform, luma intra prediction, and sum of luma prediction and residual, and the newly added processing processes included in the original area (i.e., unmapped area) include loop filtering, motion compensation prediction, chroma intra prediction, sum of chroma prediction and residual, and storing the decoded image as a reference image. The working blocks of luma mapping and chroma scaling in the loop include forward and backward luma signal mapping, and luma-dependent chroma scaling process.

[0017] JPEG0007672511000001.jpg57170

[0018] JPEG0007672511000002.jpg36170

[0019] JPEG0007672511000003.jpg20170

[0020] JPEG0007672511000004.jpg80170

[0021] JPEG0007672511000005.jpg28170

[0022] JPEG0007672511000006.jpg18170

[0023] JPEG0007672511000007.jpg32170

[0024] JPEG0007672511000008.jpg12170

[0025] JPEG0007672511000009.jpg17170

[0026] JPEG0007672511000010.jpg86170

[0027] Chroma residual scaling is used to complement the interaction between the luma signal and the corresponding chroma signal. Whether chroma residual scaling is enabled is specified at the slice layer. If luma mapping is enabled, an additional control identifier specifies whether luma-dependent chroma residual scaling is enabled. If luma mapping is not enabled, luma-dependent chroma residual scaling is also not enabled. Note that luma-dependent chroma residual scaling is not always enabled for small coded chroma blocks.

[0028] JPEG0007672511000011.jpg67170

[0029] JPEG0007672511000012.jpg32170

[0030] JPEG0007672511000013.jpg27170

[0031] JPEG0007672511000014.jpg27170

[0032] FIG. 2 is a schematic diagram of an out-of-loop encoding / decoding process according to an embodiment of the present application. Referring to FIG. 2, an input HDR video can be obtained, the HDR video can be analyzed and processed to generate metadata, the metadata can be sent to a decoding side for use in reconstructing an HDR video, the HDR video can be encoded and decoded, and the decoded video can be output as a standard dynamic range (SDR) video.

[0033] FIG. 3 is a flowchart of an image encoding method according to an embodiment of the present application, which is applicable to encoding an image, and the method may be performed by an image encoding device in an embodiment of the present application, which may be realized by software and / or hardware, and generally can be integrated into an image encoder, and the method of the embodiment of the present application specifically includes the following steps:

[0034] In step 110, a video image is acquired and image characteristics of the video image are identified.

[0035] Wherein, the video image may be an image of one or more frames in a video stream or a media file, and the image characteristics may be information of the characteristics of the video image, which may be used to determine information for performing luminance and chromaticity mapping conversion processing on the video image.

[0036] In an embodiment of the present application, images of one or more frames in a video sequence can be obtained, and image characteristics of the video image of each frame can be identified, for example, the video image of each frame can be input into a pre-trained neural network model, and the neural network model can identify image characteristics corresponding to the video image of each frame.

[0037] In step 120, the in-loop identification information and the out-loop identification information are respectively identified according to the image characteristics.

[0038] Among them, the in-loop identification information may be information for controlling a luminance and chromaticity mapping conversion process within a video image encoding / decoding loop, and the out-loop identification information may be information for controlling a luminance and chromaticity mapping conversion process outside the video image encoding / decoding loop.

[0039] For example, it is possible to determine whether or not it is necessary to perform luminance and chromaticity mapping conversion processing within the encoding / decoding loop depending on the image characteristics, and whether or not it is necessary to perform luminance and chromaticity mapping conversion processing outside the encoding / decoding loop, and to generate corresponding in-loop identification information and out-loop identification information according to the determination result.

[0040] In step 130, the video images are processed according to the in-loop identification information and the out-of-loop identification information.

[0041] In an embodiment of the present application, it is determined whether or not it is necessary to perform luminance and chromaticity mapping conversion processing on a video image outside the encoding / decoding loop, and whether or not it is necessary to perform luminance and chromaticity mapping conversion processing on a video image inside the encoding / decoding loop, depending on the in-loop identification information and the out-loop identification information, and it is possible to sequentially perform luminance and chromaticity mapping conversion processing on the video image outside the encoding / decoding loop and inside the encoding / decoding loop depending on the corresponding determination result.

[0042] In step 140, the processed video images are encoded to generate a codestream, and the in-loop and out-of-loop identification information is written into the codestream.

[0043] Among them, the codestream may be a data stream obtained after a video image is encoded.

[0044] For example, the video image can be encoded after undergoing luminance and chromaticity mapping conversion, and the video image can be encoded into a data stream, and the determined in-loop identification information and out-of-loop identification information can be added to the data stream. The encoded code stream can also be transmitted to realize the transmission of the video image.

[0045] The embodiment of the present application identifies image characteristics of an acquired video image, uses the image characteristics to generate in-loop identification information and out-loop identification information, processes the video image according to the in-loop identification information and out-loop identification information, encodes the processed video image into a code stream, and adds the generated in-loop identification information and out-loop identification information to the code stream, thereby realizing the combined use of out-loop and in-loop luma and chromaticity mapping transformation in the image encoding process, and improving the video image encoding effect.

[0046] In one embodiment, further to the embodiments of the above application, the image characteristics include at least one of luminance dynamic range, color space, texture information, shape features, and spatial relationships.

[0047] In an embodiment of the present application, the luminance dynamic range may be the range from the minimum to maximum luminance in the video image, the color space may be the color range covered by the image colors in the video image, the texture information may be spatial arrangement information of quantized colors or intensities in the video image, the shape features may be contour and area features of targets in the video image, and the spatial relationship may be the spatial position relationship or relative orientation relationship between multiple divisible targets in the video image.

[0048] In one embodiment, in addition to the embodiments of the above application, the in-loop identification information and / or the out-loop identification information is carried in at least one of the following information: network abstraction layer unit header information, a sequence layer parameter set, a picture layer parameter set, a slice layer parameter set, supplemental enhancement information, video usability information, and custom unit information.

[0049] For example, the generated in-loop identification information and out-of-loop identification information may be added to the network abstraction layer unit header information, the sequence layer parameter set, the picture layer parameter set, the slice layer parameter set, the supplementary enhancement information, and the video usability information of the video image codestream, or one new unit information may be written into the codestream, which may be a custom-generated information unit.

[0050] FIG. 4 is a flowchart of another image encoding method according to an embodiment of the present application, which is implemented based on the embodiment of the above application. Referring to FIG. 4, the method according to the embodiment of the present application specifically includes the following steps:

[0051] In step 210, a video image is acquired and image characteristics of the video image are identified.

[0052] In step 220, the in-loop identification information and the out-loop identification information are respectively identified according to the image characteristics.

[0053] In step 230, a first mapping transformation is performed on the video image according to the out-loop identification information, and a second mapping transformation is performed on the video image that has undergone the first mapping transformation according to the in-loop identification information.

[0054] In an embodiment of the present application, it is possible to use the outside-loop identification information outside the encoding loop to determine whether or not a first mapping conversion is to be performed, and to use the in-loop identification information inside the encoding loop to determine whether or not a second mapping conversion needs to be performed on the video image. When it is determined to perform the first mapping conversion on the video image using the outside-loop identification information, the first mapping conversion processing is performed on the video image outside the encoding loop according to the control parameters in the outside-loop identification information, and when it is determined to perform the second mapping conversion on the video image according to the in-loop identification information, the second mapping conversion processing is performed on the video image that has undergone the first mapping conversion processing inside the encoding loop.

[0055] In step 240, the processed video images are encoded to generate a codestream, and the in-loop and out-of-loop identification information is written into the codestream.

[0056] FIG. 5 is a flowchart of another image encoding method according to an embodiment of the present application, which is implemented based on the embodiment of the above application. Referring to FIG. 5, the method according to the embodiment of the present application specifically includes the following steps:

[0057] In step 310, a video image is acquired and image characteristics of the video image are identified.

[0058] In step 320, the in-loop identification information and the out-loop identification information are respectively determined according to the image characteristics.

[0059] In step 330, it is determined to perform a first mapping transformation on the video image according to the out-loop identification information, and not to perform a second mapping transformation on the video image that has undergone the first mapping transformation according to the in-loop identification information.

[0060] In an embodiment of the present application, it is possible to use the outside-loop identification information outside the encoding loop to determine whether or not to perform a first mapping conversion, and to use the in-loop identification information inside the encoding loop to determine whether or not it is necessary to perform a second mapping conversion on the video image. When it is determined to perform the first mapping conversion on the video image using the outside-loop identification information, the first mapping conversion processing is performed on the video image outside the encoding loop according to the control parameters in the outside-loop identification information; when it is determined not to perform the second mapping conversion on the video image according to the in-loop identification information, the second mapping conversion processing is not performed on the video image that has undergone the first mapping conversion processing inside the encoding loop.

[0061] In step 340, the video image after the first mapping transformation process is encoded to generate a codestream, and the in-loop identification information and the out-of-loop identification information are written into the codestream.

[0062] FIG. 6 is a flowchart of another image encoding method according to an embodiment of the present application, which is implemented based on the embodiment of the above application. Referring to FIG. 6, the method according to the embodiment of the present application specifically includes the following steps:

[0063] At step 410, a video image is acquired and image characteristics of the video image are identified.

[0064] In step 420, the in-loop identification information and the out-loop identification information are respectively determined according to the image characteristics.

[0065] In step 430, it is determined not to perform a first mapping transformation on the video image according to the out-of-loop identification information, and a second mapping transformation is performed on the video image according to the in-loop identification information.

[0066] In an embodiment of the present application, the out-loop identification information can be used to determine whether to perform a first mapping conversion outside the encoding loop, and the in-loop identification information can be used within the encoding loop to determine whether a second mapping conversion needs to be performed on the video image, it can be determined according to the out-loop identification information not to perform the first mapping conversion on the video image outside the encoding loop, and when it is determined according to the in-loop identification information to perform a second mapping conversion on the video image within the encoding loop, the control parameters in the in-loop identification information can be used to control the second mapping conversion process of the video image within the encoding loop.

[0067] In step 440, the video image after the second mapping transformation process is encoded to generate a codestream, and the in-loop identification information and the out-of-loop identification information are written into the codestream.

[0068] FIG. 7 is a flowchart of another image encoding method according to an embodiment of the present application, which is implemented based on the embodiment of the above application. Referring to FIG. 7, the method according to the embodiment of the present application specifically includes the following steps:

[0069] In step 510, a video image is obtained and image characteristics of the video image are identified.

[0070] In step 520, the in-loop identification information and the out-loop identification information are respectively determined according to the image characteristics.

[0071] In step 530, it is determined not to perform a first mapping transformation on the video image according to the out-of-loop identification information, and it is determined not to perform a second mapping transformation on the video image according to the in-loop identification information.

[0072] For example, the out-loop identification information can be used to determine whether a first mapping transformation needs to be performed outside the encoding loop, and the in-loop identification information can be used to determine whether a second mapping transformation needs to be performed within the encoding loop, and it can be determined not to perform the first mapping transformation on the video image and not to perform the second mapping transformation on the video image.

[0073] In step 540, the video images are encoded to generate a codestream, and the in-loop and out-of-loop identification information is written into the codestream.

[0074] In one embodiment, in addition to the embodiments of the above application, the first mapping transformation and the second mapping transformation include a luminance mapping transformation and a chromaticity mapping transformation, of which the luminance mapping transformation includes mapping of luminance component numerical values, the mapping of luminance component numerical values ​​is used to change the luminance dynamic range, and the chromaticity mapping transformation includes mapping of chromaticity component numerical values ​​or a chromaticity scaling process dependent on the luminance mapping transformation.

[0075] For example, the first mapping transformation and the second mapping transformation can both include a luminance mapping transformation and a chromaticity mapping transformation, where the luminance mapping transformation includes a mapping of luminance component values, where the mapping of luminance component values ​​is used to change the luminance dynamic range, and the chromaticity mapping transformation includes a mapping of chromaticity component values ​​or a chromaticity scaling process dependent on the luminance mapping transformation. For example, the first mapping transformation can be a mapping transformation based on a reshaper, and the second mapping transformation can be a mapping transformation based on an LMCS technique.

[0076] In one embodiment, in addition to the embodiments of the above application, the in-loop identification information and / or the out-loop identification information includes control information and parameter information, in which the control information is carried in at least one of network abstraction layer unit header information, sequence layer parameter set, picture layer parameter set, slice layer parameter set, supplemental enhancement information, video usability information, and custom unit information, and the parameter information in the codestream is carried in at least one of a self-adaptation parameter set, supplemental enhancement information, and custom network abstraction layer unit.

[0077] In the embodiments of the present application, the in-loop identification information and the out-loop identification information may consist of control information and parameter information, of which the control information may be information for controlling whether to perform a first mapping transformation or a second mapping transformation, the control information may be specifically carried in one or more of network abstraction layer unit header information, a sequence layer parameter set, a picture layer parameter set, a slice layer parameter set, supplemental enhancement information, video usability information, and custom unit information, the parameter information may be parameter information used in the first mapping transformation or the second mapping transformation, and the parameter information may be carried in one or more of a self-adaptation parameter set, supplemental enhancement information, and custom network abstraction layer unit.

[0078] In one embodiment, in addition to the embodiment of the above application, the parameter information of the in-loop identification information and the parameter information of the out-loop identification information in the code stream are each The parameter information of the in-loop identification information and the parameter information of the out-loop identification information are carried in a data packet for luma and chroma mapping conversion in the self-adaptive parameter set; The parameter information of the in-loop identification information is carried in a data packet for luma and chroma mapping transformation in the self-adaptive parameter set, and the parameter information of the out-loop identification information is carried in a data packet for custom supplemental enhancement information; The parameter information of the in-loop identification information and the parameter information of the out-loop identification information are carried in a data packet for luma and chroma mapping conversion in the custom network abstraction layer unit.

[0079] In an embodiment of the present application, parameter information of the in-loop identification information and the out-loop identification information may be written into data packets for luma and chroma mapping conversion in a self-adaptation parameter set of the codestream, or the in-loop identification information may be written into data packets for luma and chroma mapping conversion in a self-adaptation parameter set of the codestream, and the out-loop identification information may be written into data packets of custom supplemental enhancement information, or the parameter information of the in-loop identification information and the out-loop identification information may be written into data packets for luma and chroma mapping conversion in a custom network abstraction layer unit of the codestream.

[0080] Fig. 8 is a diagram illustrating an image encoding method according to an embodiment of the present application. Referring to Fig. 8, this embodiment is a video encoding method applied to an apparatus for encoding video. An input of the apparatus is an image included in a video, and an output is an image codestream or a transport stream or a media file including the image codestream.

[0081] In step 301, an image is read, which is a frame in a video sequence.

[0082] In step 302, analyze image characteristics, including but not limited to luminance dynamic range, color space, texture information, shape features, and spatial relationships, and determine whether to perform a first mapping transformation according to the analyzed information, and set control information for the first mapping transformation; Wherein, the luminance dynamic range is the range from the minimum value to the maximum value of luminance, the color space is the color range covered by the colors of the image, the texture information is the spatial arrangement information of the quantized colors or intensities of the image, the shape feature is the contour and area feature of the target in the image, and the spatial relationship is the spatial position or relative direction relationship of multiple separable targets in the image. The control information of the first mapping transformation is used to indicate whether the first mapping transformation is performed.

[0083] In step 303, a first mapping conversion process acting outside the encoding loop is performed according to the identification information of the first mapping conversion; Wherein, the identification information of the first mapping transformation includes control information of the first mapping transformation and parameter information of the first mapping transformation, the control information of the first mapping transformation is set in step 302 and is used to identify the activation status of the first mapping transformation, and the parameter information of the first mapping transformation is a parameter of the first mapping transformation processing process; In step 304, analyze the characteristics of the image that has entered the encoding loop, including but not limited to luminance dynamic range, color space, texture information, shape features, and spatial relationships, and determine whether to perform a second mapping transformation according to the characteristics; and set control information for the second mapping transformation; Among them, the images that have entered the encoding loop include images that have undergone the first mapping conversion process or images that have not undergone the first mapping conversion process; Wherein, the luminance dynamic range is the range from the minimum value to the maximum value of luminance, the color space is the color range covered by the colors of the image, the texture information is the spatial arrangement information of the quantized colors or intensities of the image, the shape feature is the contour and area feature of the target in the image, and the spatial relationship is the spatial position or relative direction relationship of multiple separable targets in the image. The control information of the second mapping transformation is used to indicate whether or not the second mapping transformation is performed.

[0084] In step 305, a second mapping conversion process is performed in the encoding loop according to the control information of the set second mapping conversion; Wherein, the identification information of the second mapping transformation includes control information of the second mapping transformation and parameter information of the second mapping transformation, the control information of the second mapping transformation is set in step 304 and is used to identify the activation status of the second mapping transformation, and the parameter information of the second mapping transformation is a parameter of the second mapping transformation processing process; In step 306, the processed image is encoded to generate a codestream, and identification information of the first mapping transformation and the second mapping transformation is written into the codestream, and the identification information is carried into all or part of the network abstraction layer unit header information, the sequence layer parameter set, the picture layer parameter set, the slice layer parameter set, the supplemental enhancement information, the video usability information, or is made into a new information unit; Among them, the processed image refers to the image after undergoing a normal encoding process and / or a first mapping transformation and / or a second mapping transformation, where the normal encoding process includes block division, transformation and quantization, intra / inter prediction and other processing processes.

[0085] In step 307, the image codestream or a transport stream or media file containing the image codestream is output.

[0086] 9 is a diagram illustrating another image encoding method according to an embodiment of the present application. Referring to FIG. 9, this embodiment is a video encoding method applied to an apparatus for encoding a video. An input of the apparatus is an image included in a video, and an output is an image codestream or a transport stream or a media file including the image codestream.

[0087] In step 401, an image is read, which is a frame in a video sequence.

[0088] In step 402, analyze image characteristics, including but not limited to luminance dynamic range, color space, texture information, shape features, and spatial relationships, and determine whether to perform a first mapping transformation and a second mapping transformation according to the analyzed information, and set control information of the first mapping transformation and control information of the second mapping transformation; Wherein, the luminance dynamic range is the range from the minimum value to the maximum value of luminance, the color space is the color range covered by the colors of the image, the texture information is the spatial arrangement information of the quantized colors or intensities of the image, the shape feature is the contour and area feature of the target in the image, and the spatial relationship is the spatial position or relative direction relationship of multiple separable targets in the image. Among them, the control information of the first mapping transformation is used to indicate whether or not to perform the first mapping transformation, and the control information of the second mapping transformation is used to indicate whether or not to perform the second mapping transformation.

[0089] In step 403, a first mapping conversion process acting outside the encoding loop is performed according to the identification information of the first mapping conversion; Wherein, the identification information of the first mapping transformation includes control information of the first mapping transformation and parameter information of the first mapping transformation, the control information of the first mapping transformation is set in step 402 and is used to identify the activation status of the first mapping transformation, and the parameter information of the first mapping transformation is a parameter of the first mapping transformation processing process; In step 404, a second mapping conversion process is performed in the encoding loop according to the control information of the set second mapping conversion; Wherein, the identification information of the second mapping transformation includes control information of the second mapping transformation and parameter information of the second mapping transformation, the control information of the second mapping transformation is set in step 402 and is used to identify the activation status of the second mapping transformation, and the parameter information of the second mapping transformation is a parameter of the second mapping transformation processing process; In step 405, the processed image is encoded to generate a codestream, and identification information of the first mapping transformation and the second mapping transformation is written into the codestream, and the identification information is carried into all or part of the network abstraction layer unit header information, the sequence layer parameter set, the picture layer parameter set, the slice layer parameter set, the supplemental enhancement information, the video usability information, or is made into a new information unit; Among them, the processed image refers to the image after undergoing a normal encoding process and / or a first mapping transformation and / or a second mapping transformation, where the normal encoding process includes block division, transformation and quantization, intra / inter prediction and other processing processes.

[0090] In step 406, the image codestream or a transport stream or media file containing the image codestream is output.

[0091] In one exemplary embodiment, the first mapping transformation and the second mapping transformation are both implementations of luminance and chromaticity mapping transformations, among which one implementation of the luminance mapping transformation is a linear or nonlinear mapping of luminance component values, which reduces or expands the luminance dynamic range, and one implementation of the chromaticity mapping transformation includes a linear or nonlinear mapping of chromaticity component values, or a chromaticity scaling process dependent on the luminance mapping transformation.

[0092] FIG. 10 is a flowchart of an image decoding method according to an embodiment of the present application, which is applicable to decoding images in an image video transmission process, and the method may be performed by an image decoding device in an embodiment of the present application, which can be realized in the form of software and / or hardware, and can generally be integrated into an image decoder, and the method of the embodiment of the present application specifically includes the following steps:

[0093] In step 610, a codestream is obtained, and in-loop identification information and out-of-loop identification information in the codestream are analyzed to obtain.

[0094] Wherein, the codestream may be a data stream after a video image is encoded, and can be generated by an image encoding device and sent to an image decoding device.

[0095] In an embodiment of the present application, an image decoding device can receive a transmitted code stream and extract in-loop identification information and out-loop identification information from the code stream, where the in-loop identification information can be information for controlling a luma and chroma mapping conversion process within a video image encoding / decoding loop, and the out-loop identification information can be information for controlling a luma and chroma mapping conversion process outside the video image encoding / decoding loop.

[0096] In step 620, the codestream is processed according to the in-loop identification information and the out-of-loop identification information.

[0097] For example, it is possible to determine whether or not luminance and chromaticity mapping conversion processing needs to be performed on the code stream within the decoding loop, and whether or not luminance and chromaticity mapping conversion processing needs to be performed on the code stream outside the decoding loop, depending on the in-loop identification information and the out-loop identification information, and to sequentially perform luminance and chromaticity mapping conversion processing on the code stream within the decoding loop and outside the decoding loop depending on the corresponding determination results, and it can be understood that the processing in the image decoding device may be the inverse processing of the processing in the image encoding device.

[0098] In step 630, the processed codestream is decoded to generate a video image.

[0099] In the embodiment of the present application, the received code stream can be decoded according to a video decoding algorithm to obtain a decoded video image, which can be further sent to a display interface for presentation. It can be understood that the code stream decoding process can be processed according to the decoding process in the related art, and the code stream decoding process can correspond to the video image encoding process in a video encoding device.

[0100] An embodiment of the present application receives a codestream, extracts the out-loop identification information and in-loop identification information carried in the codestream, processes the received codestream according to the in-loop identification information and the out-loop identification information, and decodes the codestream into a video image, thereby realizing the combined use of out-loop and in-loop luma and chroma mapping transformations in the decoding process, and improving the quality of the decoded video image.

[0101] In one embodiment, in addition to the embodiment of the above application, the in-loop identification information and / or the out-loop identification information are The information carried by the at least one of the following: network abstraction layer unit header information, a sequence layer parameter set, a picture layer parameter set, a slice layer parameter set, supplemental enhancement information, video usability information, and custom unit information.

[0102] In one embodiment, in addition to the embodiments of the above application, the processing of the codestream in accordance with the in-loop identification information and the out-of-loop identification information as described above further comprises: performing a fourth mapping transformation on the code stream in response to the in-loop identification information, and performing a third mapping transformation on the code stream that has been subjected to the fourth mapping transformation in response to the out-loop identification information; performing a fourth mapping transformation on the codestream according to the in-loop identification information, and determining not to perform a third mapping transformation on the codestream that has been subjected to the fourth mapping transformation according to the out-loop identification information; determining not to perform a fourth mapping transformation on the code stream in response to the in-loop identification information, and performing a third mapping transformation on the code stream in response to the out-loop identification information; determining not to perform a fourth mapping transformation on the code stream in response to the in-loop identification information; and determining not to perform a third mapping transformation on the code stream in response to the out-loop identification information.

[0103] Among them, the third mapping transformation and the fourth mapping transformation may be processes for luma and chromaticity mapping transformation, the third mapping transformation may be an inverse transformation of the first mapping transformation in the video encoding device, and the fourth mapping transformation may be an inverse transformation of the second mapping transformation in the video encoding device. For example, the fourth mapping transformation may be a mapping transformation based on LMCS technology, and the third mapping transformation may be a mapping transformation based on a reshaper.

[0104] In an embodiment of the present application, the in-loop identification information can be used in the decoding loop to determine whether to perform a fourth mapping transformation on the codestream, and the out-loop identification information can be used outside the decoding loop to determine whether a third mapping transformation needs to be performed on the codestream, which can include: performing a fourth mapping transformation on the codestream in the decoding loop, and performing a third mapping transformation on the codestream that has undergone the fourth mapping transformation outside the decoding loop, or performing a fourth mapping transformation on the codestream in the decoding loop, and not performing a third mapping transformation on the codestream that has undergone the fourth mapping transformation outside the decoding loop, or not performing a fourth mapping transformation on the codestream in the decoding loop, and performing a third mapping transformation on the codestream outside the decoding loop, or not performing a fourth mapping transformation on the codestream in the decoding loop, and not performing a third mapping transformation on the codestream outside the decoding loop, or not performing a fourth mapping transformation on the codestream in the decoding loop, and not performing a third mapping transformation on the codestream outside the decoding loop.

[0105] In one embodiment, in addition to the embodiments of the above application, the third mapping transformation and the fourth mapping transformation respectively include a luminance mapping transformation and a chromaticity mapping transformation, in which the luminance mapping transformation includes mapping of luminance component numerical values, the mapping of luminance component numerical values ​​is used to change the luminance dynamic range, and the chromaticity mapping transformation includes mapping of chromaticity component numerical values ​​or a chromaticity scaling process dependent on the luminance mapping transformation.

[0106] In one embodiment, in addition to the embodiments of the above application, the in-loop identification information and / or the out-loop identification information includes control information and parameter information, in which the control information is carried in at least one of network abstraction layer unit header information, sequence layer parameter set, picture layer parameter set, slice layer parameter set, supplemental enhancement information, video usability information, and custom unit information, and the parameter information in the codestream is carried in at least one of a self-adaptation parameter set, supplemental enhancement information, and custom network abstraction layer unit.

[0107] In one embodiment, in addition to the embodiment of the above application, the parameter information of the in-loop identification information and the parameter information of the out-loop identification information in the code stream are each The parameter information of the in-loop identification information and the parameter information of the out-loop identification information are carried in a data packet for luma and chroma mapping conversion in the self-adaptive parameter set; The parameter information of the in-loop identification information is carried in a data packet for luma and chroma mapping transformation in the self-adaptive parameter set, and the parameter information of the out-loop identification information is carried in a data packet for custom supplemental enhancement information; The parameter information of the in-loop identification information and the parameter information of the out-loop identification information are carried in a data packet for luma and chroma mapping conversion in the custom network abstraction layer unit.

[0108] Fig. 11 is a diagram illustrating an image decoding method according to an embodiment of the present application. Referring to Fig. 11, in one exemplary embodiment, the embodiment is a video decoding method applied to an apparatus for decoding a video code stream. The input of the apparatus is an image code stream or a transport stream or a media file containing an image code stream, and the output is images constituting a video.

[0109] In step 501, the codestream is read.

[0110] In step 502, the code stream is analyzed to obtain identification information of mapping transformations, the identification information of the mapping transformations including identification information of a third mapping transformation and identification information of a fourth mapping transformation; Wherein, the identification information of the third mapping transformation includes control information of the third mapping transformation and parameter information of the third mapping transformation, the control information of the third mapping transformation is used for identifying an enabled status of the third mapping transformation, and the parameter information of the third mapping transformation is a parameter of a third mapping transformation processing process; Wherein, the identification information of the fourth mapping transformation includes control information of the fourth mapping transformation and parameter information of the fourth mapping transformation, the control information of the fourth mapping transformation is used for identifying an enabled status of the fourth mapping transformation, and the parameter information of the fourth mapping transformation is a parameter of a fourth mapping transformation processing process; In step 503, it is determined whether to perform a fourth mapping transformation according to the identification information of the fourth mapping transformation; In step 504, a fourth mapping transformation is performed in the decoding loop according to the identification information of the fourth mapping transformation; Wherein, the fourth mapping transformation here is the inverse process corresponding to the embodiment step 305; In step 505, it is determined whether to perform a third mapping transformation according to the identification information of the third mapping transformation; In step 506, a third mapping process is performed outside the decoding loop according to the identification information of the third mapping process; Wherein, the third mapping transformation here is the inverse process corresponding to the embodiment step 303; In step 507, the decoded image is output.

[0111] In one embodiment, based on the embodiments of the above application, syntax and semantics related to in-loop and out-of-loop luma and chroma mapping transformations at the sequence layer, picture layer and slice layer are provided.

[0112] Table 1 shows the identification information of the luma and chroma mapping transformation carried in the sequence layer parameter set (SPS) in the codestream described in the above embodiment.

[0113] JPEG0007672511000015.jpg43170

[0114] JPEG0007672511000016.jpg53170

[0115] JPEG0007672511000017.jpg66170

[0116] JPEG0007672511000018.jpg46170

[0117] JPEG0007672511000019.jpg49170

[0118] Table 3 shows identification information used for in-loop and out-loop luma and chroma mapping conversion carried in the picture header (PH) parameter set in the codestream described in the above embodiment.

[0119] JPEG0007672511000020.jpg52170

[0120] JPEG0007672511000021.jpg92170

[0121] JPEG0007672511000022.jpg120170

[0122] Table 4 shows identification information used for in-loop and out-loop luma and chroma mapping conversion carried in a slice header (SH) parameter set in the codestream described in the above embodiment.

[0123] JPEG0007672511000023.jpg19170

[0124] JPEG0007672511000024.jpg32170

[0125] JPEG0007672511000025.jpg46170

[0126] JPEG0007672511000026.jpg86170

[0127] In one exemplary embodiment, syntax and semantics related to in-loop and out-of-loop luma and chroma mapping transformations at the sequence layer are provided.

[0128] table 5 is the identification information of the luma and chroma mapping transformation carried in the sequence layer parameter set (SPS) in the codestream described in the above embodiment.

[0129] JPEG0007672511000027.jpg32170

[0130] JPEG0007672511000028.jpg17170

[0131] JPEG0007672511000029.jpg57170

[0132] JPEG0007672511000030.jpg44170

[0133] JPEG0007672511000031.jpg89159

[0134] table 6 is identification information used for in-loop and out-loop luma and chroma mapping conversion carried in the slice header (SH) parameter set in the codestream described in the above embodiment.

[0135] JPEG0007672511000032.jpg18170

[0136] JPEG0007672511000033.jpg23170

[0137] JPEG0007672511000034.jpg32170

[0138] JPEG0007672511000035.jpg37170

[0139] JPEG0007672511000036.jpg132159

[0140] In one exemplary embodiment, syntax and semantics related to in-loop and out-of-loop luma and chroma mapping transformations at the picture layer are provided.

[0141] JPEG0007672511000037.jpg30170

[0142] JPEG0007672511000038.jpg39170

[0143] table 8 is identification information used for in-loop and out-loop luma and chroma mapping conversion carried in the picture header parameter set (PH) in the codestream described in the above embodiment.

[0144] JPEG0007672511000039.jpg45170

[0145] JPEG0007672511000040.jpg79170

[0146] JPEG0007672511000041.jpg135170

[0147] In one exemplary embodiment, syntax and semantics related to in-loop and out-of-loop luma and chroma mapping transformations at the picture layer, slice layer are provided.

[0148] JPEG0007672511000042.jpg29170

[0149] JPEG0007672511000043.jpg38170

[0150] table 10 is identification information used for in-loop and out-loop luma and chroma mapping conversion carried in the slice header parameter set (SH) in the codestream described in the above embodiment.

[0151] JPEG0007672511000044.jpg17170

[0152] JPEG0007672511000045.jpg26170

[0153] JPEG0007672511000046.jpg35170

[0154] JPEG0007672511000047.jpg82170

[0155] In one exemplary embodiment, a control syntax related to in-loop and out-of-loop luma and chroma mapping conversion in a picture layer Adaptation Parameter Set (APS) is defined, and a decoder can select to perform in-loop luma and chroma mapping conversion, out-of-loop luma and chroma mapping conversion, or no luma and chroma mapping conversion according to the syntax defined in this embodiment.

[0156] JPEG0007672511000048.jpg59170

[0157] JPEG0007672511000049.jpg52170

[0158] JPEG0007672511000050.jpg44170

[0159] JPEG0007672511000051.jpg82170

[0160] JPEG0007672511000052.jpg105170

[0161] JPEG0007672511000053.jpg91170

[0162] JPEG0007672511000054.jpg80169

[0163] table 14 Shown below is one preferred syntax configuration.

[0164] JPEG0007672511000055.jpg57169

[0165] JPEG0007672511000056.jpg103170

[0166] JPEG0007672511000057.jpg103170

[0167] JPEG0007672511000058.jpg54168

[0168] table 15 Shown below is one preferred syntax configuration.

[0169] JPEG0007672511000059.jpg22168

[0170] JPEG0007672511000060.jpg191170

[0171] 12 is a structural diagram of an image encoding device according to an embodiment of the present application, which can execute the image encoding method according to any embodiment of the present application, and has corresponding functional modules and beneficial effects for executing the method. The device can be realized by software and / or hardware, and specifically includes an image acquisition module 701, an identification module 702, an image processing module 703 and an encoding processing module 704.

[0172] The image capture module 701 is configured to capture a video image and identify image characteristics of said video image.

[0173] The identification module 702 is configured to identify the in-loop identification information and the out-of-loop identification information, respectively, according to the image characteristics.

[0174] An image processing module 703 is configured to process the video images according to the in-loop identification information and the out-of-loop identification information.

[0175] An encoding processing module 704 is configured to encode the processed video images to generate a codestream, and to write the in-loop identification information and the out-of-loop identification information into the codestream.

[0176] In an embodiment of the present application, the image acquisition module identifies image characteristics of the acquired video image, the identification module uses the image characteristics to generate in-loop identification information and out-loop identification information, the image processing module processes the video image according to the in-loop identification information and the out-loop identification information, the encoding processing module encodes the processed video image into a code stream, and adds the generated in-loop identification information and out-loop identification information to the code stream, thereby realizing the combined use of out-loop and in-loop luma and chromaticity mapping transformation in the encoding process, and improving the video image encoding effect.

[0177] In one embodiment, in addition to the embodiments of the above application, the image characteristics of the device are: The information includes at least one of the following: luminance dynamic range, color space, texture information, shape features, and spatial relationships.

[0178] In one embodiment, in addition to the embodiment of the above application, at least one of the in-loop identification information and the out-loop identification information in the device is: The information carried by the at least one of the following: network abstraction layer unit header information, a sequence layer parameter set, a picture layer parameter set, a slice layer parameter set, supplemental enhancement information, video usability information, and custom unit information.

[0179] In one embodiment, further to the embodiments of the above application, the image processing module 703 in the device comprises any one of a first processing unit, a second processing unit, a third processing unit, and a fourth processing unit.

[0180] The first processing unit is configured to perform a first mapping transformation on the video image according to the out-loop identification information, and to perform a second mapping transformation on the video image that has undergone the first mapping transformation according to the in-loop identification information.

[0181] The second processing unit is configured to perform a first mapping transformation on the video image according to the out-of-loop identification information, and to determine not to perform a second mapping transformation on the video image that has undergone the second mapping transformation according to the in-loop identification information.

[0182] A third processing unit is configured to determine not to perform a first mapping transformation on the video image according to the out-of-loop identification information, and to perform a second mapping transformation on the video image according to the in-loop identification information.

[0183] A fourth processing unit is configured to determine not to perform a first mapping transformation on the video image in response to the out-of-loop identification information, and to determine not to perform a second mapping transformation on the video image in response to the in-loop identification information.

[0184] In one embodiment, in addition to the embodiments of the above application, the first mapping transformation and the second mapping transformation in the device respectively include a luminance mapping transformation and a chromaticity mapping transformation, in which the luminance mapping transformation includes mapping of luminance component numerical values, the mapping of luminance component numerical values ​​is used to change the luminance dynamic range, and the chromaticity mapping transformation includes mapping of chromaticity component numerical values, or a chromaticity scaling process dependent on the luminance mapping transformation.

[0185] In one embodiment, in addition to the embodiments of the above application, at least one of the in-loop identification information and the out-loop identification information in the device includes control information and parameter information, in which the control information is carried in at least one of network abstraction layer unit header information, sequence layer parameter set, picture layer parameter set, slice layer parameter set, supplemental enhancement information, video usability information, and custom unit information, and the parameter information in the codestream is carried in at least one of a self-adaptation parameter set, supplemental enhancement information, and custom network abstraction layer unit.

[0186] In one embodiment, in addition to the embodiment of the above application, in the code stream, the parameter information of the in-loop identification information and the out-loop identification information of the device are each The parameter information of the in-loop identification information and the parameter information of the out-loop identification information are carried in a data packet for luma and chroma mapping conversion in the self-adaptive parameter set; The parameter information of the in-loop identification information is carried in a data packet for luma and chroma mapping transformation in the self-adaptive parameter set, and the parameter information of the out-loop identification information is carried in a data packet for custom supplemental enhancement information; The parameter information of the in-loop identification information and the parameter information of the out-loop identification information are carried in a data packet for luma and chroma mapping conversion in the custom network abstraction layer unit.

[0187] 13 is a structural diagram of an image decoding device according to an embodiment of the present application, which can execute the image decoding method according to any embodiment of the present application, and has corresponding functional modules and beneficial effects for executing the method. The device can be realized by software and / or hardware, and specifically includes a codestream analysis module 801, a codestream processing module 802 and an image decoding module 803.

[0188] The codestream analysis module 801 is configured to obtain a codestream, and analyze the codestream to obtain in-loop identification information and out-of-loop identification information.

[0189] The codestream processing module 802 is configured to process the codestream according to the in-loop identification information and the out-of-loop identification information.

[0190] An image decoding module 803 is configured to decode the processed codestream to generate a video image.

[0191] In an embodiment of the present application, a codestream analysis module receives a codestream and extracts the out-loop identification information and in-loop identification information carried in the codestream, a codestream processing module processes the received codestream according to the in-loop identification information and the out-loop identification information, and an image decoding module decodes the codestream into a video image, thereby realizing the combined use of out-loop and in-loop luma and chroma mapping transformation in the decoding process, and improving the quality of the decoded video image.

[0192] In one embodiment, in addition to the embodiment of the above application, at least one of the in-loop identification information and the out-loop identification information in the device is: The information carried by the at least one of the following: network abstraction layer unit header information, a sequence layer parameter set, a picture layer parameter set, a slice layer parameter set, supplemental enhancement information, video usability information, and custom unit information.

[0193] In one embodiment, further to the embodiments of the above application, the codestream processing module 802 in the device comprises a first processing unit, a second processing unit, a third processing unit, or a fourth processing unit.

[0194] The first processing unit is configured to perform a fourth mapping transformation on the codestream in response to the in-loop identification information, and to perform a third mapping transformation on the codestream that has undergone the fourth mapping transformation in response to the out-loop identification information.

[0195] The second processing unit is configured to perform a fourth mapping transformation on the codestream in response to the in-loop identification information, and to determine not to perform a third mapping transformation on the codestream that has undergone the fourth mapping transformation in response to the out-loop identification information.

[0196] A third processing unit is configured to determine not to perform a fourth mapping transformation on the codestream in response to the in-loop identification information, and to perform a third mapping transformation on the codestream in response to the out-loop identification information.

[0197] A fourth processing unit is configured to determine not to perform a fourth mapping transformation on the codestream in response to the in-loop identification information, and to determine not to perform a third mapping transformation on the codestream in response to the out-loop identification information.

[0198] In one embodiment, in addition to the embodiment of the above application, the third mapping transformation and the fourth mapping transformation in the device respectively include a luminance mapping transformation and a chromaticity mapping transformation, in which the luminance mapping transformation includes mapping of luminance component numerical values, the mapping of luminance component numerical values ​​is used to change the luminance dynamic range, and the chromaticity mapping transformation includes mapping of chromaticity component numerical values, or a chromaticity scaling process dependent on the luminance mapping transformation.

[0199] In one embodiment, in addition to the embodiments of the above application, at least one of the in-loop identification information and the out-loop identification information in the device includes control information and parameter information, in which the control information is carried in at least one of network abstraction layer unit header information, sequence layer parameter set, picture layer parameter set, slice layer parameter set, supplemental enhancement information, video usability information, and custom unit information, and the parameter information in the codestream is carried in at least one of a self-adaptation parameter set, supplemental enhancement information, and custom network abstraction layer unit.

[0200] In one embodiment, in addition to the embodiment of the above application, in the code stream, the parameter information of the in-loop identification information and the out-loop identification information of the device are each The parameter information of the in-loop identification information and the parameter information of the out-loop identification information are carried in a data packet for luma and chroma mapping conversion in the self-adaptive parameter set; The parameter information of the in-loop identification information is carried in a data packet for luma and chroma mapping transformation in the self-adaptive parameter set, and the parameter information of the out-loop identification information is carried in a data packet for custom supplemental enhancement information; The parameter information of the in-loop identification information and the parameter information of the out-loop identification information are carried in a data packet for luma and chroma mapping conversion in the custom network abstraction layer unit.

[0201] FIG. 14 is a structural schematic diagram of an electronic device according to an embodiment of the present application, the electronic device comprising a processor 60, a memory 61, an input device 62 and an output device 63. The number of processors 60 in the electronic device may be one or more, and FIG. 14 takes one processor 60 as an example. The processor 60, memory 61, input device 62 and output device 63 in the electronic device may be connected by a bus or other form, and FIG. 14 takes connection by a bus as an example.

[0202] The memory 61 is used as a computer-readable storage medium to store software programs, computer-executable programs and modules, such as the modules corresponding to the image encoding device and / or image decoding device in the embodiments of the present application (image acquisition module 701, identification module 702, image processing module 703 and encoding processing module 704, and / or code stream analysis module 801, code stream processing module 802 and image decoding module 803). The processor 60 executes various functional applications and data processing of the electronic device by operating the software programs, instructions and modules stored in the memory 61, i.e., realizes the above-mentioned image encoding method and / or image decoding method.

[0203] The memory 61 may mainly include a program storage area and a data storage area, among which the program storage area may store an operating system and an application program required for at least one function, and the data storage area may store data generated according to the use of the electronic device. The memory 61 may also include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some examples, the memory 61 may further include a memory provided remotely to the processor 60, and these remote memories may be connected to the electronic device by a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local network, a mobile communication network, and combinations thereof.

[0204] The input device 62 is used to receive input digital or textual information and generate input key signals associated with user settings and function control of the electronic device. The output device 63 can include a display device such as a display.

[0205] Fig. 15 is a structural schematic diagram of an encoder according to an embodiment of the present application, and in one exemplary embodiment, the encoder shown in Fig. 15 is applied to an apparatus for encoding a video, the input of which is an image included in a video, and the output of which is an image code stream or a transport stream or a media file including the image code stream. As shown in FIG. 15, the encoder includes an input device 901 , a processing device 902 and an output device 903 .

[0206] The input device 901 is Input image configured to The processing device 902 includes: Image processing and encoding configured to The specific process is consistent with the operations in the above examples. The output device 903 is Output the code stream It is configured as follows.

[0207] Fig. 16 is a structural schematic diagram of an encoder according to an embodiment of the present application, and in one exemplary embodiment, the encoder shown in Fig. 16 is applied to an apparatus for encoding a video, the input of which is an image included in a video, and the output is an image code stream or a transport stream or a media file including the image code stream. As shown in FIG. 16, the encoder includes an input device 1001 , a pre-processor 1002 , a processing device 1003 and an output device 1004 .

[0208] The input device 1001 is Input image configured to Applicable preprocessor 1002 Process the image It is configured to The specific process is the same as steps 302 and 303 of the image encoding method according to the above embodiment; The processing device 1003 includes: Image processing and encoding configured to The specific process corresponds to steps 304, 305, and 306 of the image encoding method according to the above embodiment. The output device 1004 includes: Output the code stream It is configured as follows.

[0209] Fig. 17 is a structural diagram of a decoder according to an embodiment of the present application, in one exemplary embodiment, the decoder shown in Fig. 17 is applied to an apparatus for decoding a video code stream, the input of the apparatus is an image code stream or a transport stream or a media file containing an image code stream, and the output is images constituting a video. As shown in FIG. 17, the decoder includes an input device 1101 , a processing device 1102 and an output device 1103 .

[0210] The input device 1101 is Input Codestream configured to The processing device 1102 parses the code stream to get the image and processes the image configured to The specific process is consistent with the operation of the video decoding method in the embodiment of the above application.

[0211] The output device 1103 is Outputting an image It is configured as follows.

[0212] Fig. 18 is a structural diagram of a decoder according to an embodiment of the present application, and in one exemplary embodiment, the decoder shown in Fig. 18 is applied to an apparatus for decoding a video code stream, the input of which is an image code stream or a transport stream or a media file containing an image code stream, and the output is images constituting a video. As shown in FIG. 18, the decoder includes an input device 1201 , a processing device 1202 , a post-processor 1203 and an output device 1204 .

[0213] The input device 1201 is Input Codestream configured to The processing device 1202 parses the code stream to get the image and then reverse processes the image It is configured to The specific process is consistent with the operations of steps 501, 502, 503 and 504 of the image decoding method in the above embodiment.

[0214] Applicable Post Processor 1203 Process the image It is configured to The specific process is consistent with the operation of step 506 in the image decoding method of the above embodiment.

[0215] The output device 1204 is , output the image It is configured as follows: .

[0216] An embodiment of the present application further provides a storage medium comprising computer-executable instructions, which, when executed by a computer processor, are used to perform an image coding method, the method comprising: acquiring a video image and identifying image characteristics of the video image; identifying in-loop identification information and out-loop identification information in accordance with the image characteristics; processing the video image in accordance with the in-loop identification information and the out-of-loop identification information; encoding the processed video images to generate a codestream, and writing the in-loop identification information and the out-of-loop identification information into the codestream.

[0217] and / or the computer executable instructions, when executed by a computer processor, are used to perform an image decoding method, the method comprising: obtaining a codestream and analyzing the codestream to obtain in-loop identification information and out-of-loop identification information; processing the codestream in accordance with the in-loop identification information and the out-of-loop identification information; and decoding the processed codestream to generate a video image.

[0218] From the above description of the embodiments, those skilled in the art can clearly understand that the present application can be realized through software and necessary general-purpose hardware, and of course, it may be realized by hardware, but in many cases, the former is a more preferred embodiment. Based on this understanding, the essence of the technical solution of the present application or the part contributing to the related technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (FLASH), a hard disk or an optical disk, etc., and a computer device (which may be a personal computer, a server, or a network device, etc.) includes some instructions for executing the method described in each embodiment of the present application.

[0219] In the above-described device embodiments, each unit and module provided is merely defined according to functional logic, but is not limited to the above-described definition as long as the corresponding function can be realized. Furthermore, it should be noted that the specific names of each functional unit are merely intended to facilitate distinction from each other, and are not intended to limit the scope of protection of the present application.

[0220] Those skilled in the art will understand that all or any of the steps in the methods, systems, and functional modules / units in the devices disclosed in the above specification may be implemented as software, firmware, hardware, or an appropriate combination thereof.

[0221] In hardware embodiments, the definition between functional modules / units mentioned in the above description does not necessarily correspond to the definition of physical components, for example, one physical component may have multiple functions, or one function or step may be performed jointly by several physical components. Some or all of the physical components may be implemented as software executed by a processor, such as a central processor, digital signal processor or microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as a dedicated integrated circuit. Such software may be distributed on computer-readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). As is well known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (e.g., computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory or other memory technology, CD-ROM (Compact Disc Read-Only Memory), DVD (Digital Video Disc) or other optical disk storage, magnetic cartridges, magnetic tape, magnetic disk storage or other magnetic storage devices, or any medium that can be used to store the desired information and accessible by a computer. Additionally, as known to those skilled in the art, communication media typically include computer readable instructions, data structures, program modules or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0222] Although some embodiments of the present application have been described above with reference to the drawings, the scope of the patent right of the present application is not limited thereby. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and substance of the present application should be included in the scope of the claims of the present application.

Claims

1. acquiring a video image and identifying image characteristics of the video image; identifying in-loop identification information and out-loop identification information in accordance with the image characteristics; processing the video image in accordance with the in-loop identification information and the out-of-loop identification information; encoding the processed video images to generate a codestream, and writing the in-loop identification information and the out-of-loop identification information into the codestream; the out-loop identification information is used to indicate whether to perform a first mapping transformation on the video image, the first mapping transformation including an out-loop luminance mapping transformation and a chromaticity mapping transformation; and the in-loop identification information is used to indicate whether to perform a second mapping transformation on the video image, the second mapping transformation including an in-loop luminance mapping transformation and a chromaticity mapping transformation. Image encoding method.

2. The image characteristics include at least one of a luminance dynamic range, a color space, texture information, shape features, and spatial relationships; The image coding method according to claim 1 .

3. At least one of the in-loop identification information and the out-loop identification information is carried in at least one of network abstraction layer unit header information, sequence layer parameter set, picture layer parameter set, slice layer parameter set, supplemental enhancement information, video usability information, and custom unit information; The image coding method according to claim 1 .

4. The step of processing the video image in accordance with the in-loop identification information and the out-of-loop identification information comprises: performing the first mapping transformation on the video image in response to the out-loop identification information, and performing the second mapping transformation on the video image that has been subjected to the first mapping transformation in response to the in-loop identification information; performing the first mapping transformation on the video image in response to the out-loop identification information, and determining not to perform the second mapping transformation on the video image that has been subjected to the first mapping transformation in response to the in-loop identification information; determining not to perform the first mapping transformation on the video image in response to the out-loop identification information, and performing the second mapping transformation on the video image in response to the in-loop identification information; determining not to perform the first mapping transformation on the video image in response to the out-of-loop identification information; and determining not to perform the second mapping transformation on the video image in response to the in-loop identification information. The image coding method according to claim 1 .

5. The luminance mapping transformation includes a mapping of luminance component values, the mapping of luminance component values ​​being used to change a luminance dynamic range, and the chromaticity mapping transformation includes a mapping of chromaticity component values ​​or a chromaticity scaling process dependent on the luminance mapping transformation. The image coding method according to claim 4.

6. At least one of the in-loop identification information and the out-loop identification information includes control information and parameter information, wherein the control information is information for controlling whether or not to perform the first mapping transformation or the second mapping transformation, and the parameter information is parameter information used for the first mapping transformation or the second mapping transformation, the control information is carried in at least one of network abstraction layer unit header information, sequence layer parameter set, picture layer parameter set, slice layer parameter set, supplemental enhancement information, video usability information, and custom unit information, and the parameter information in the codestream is carried in at least one of a self-adaptation parameter set, supplemental enhancement information, and custom network abstraction layer unit. The image coding method according to claim 1 .

7. The parameter information of the in-loop identification information and the parameter information of the out-loop identification information in the code stream are each The parameter information of the in-loop identification information and the parameter information of the out-loop identification information are carried in a data packet for luma and chroma mapping conversion in the self-adaptive parameter set; The parameter information of the in-loop identification information is carried in a data packet for luma and chroma mapping transformation in the self-adaptive parameter set, and the parameter information of the out-loop identification information is carried in a data packet for custom supplemental enhancement information; the parameter information of the in-loop identity and the parameter information of the out-loop identity are carried in a data packet for a luma and chroma mapping transformation in the custom network abstraction layer unit; The image coding method according to claim 6.

8. obtaining a codestream and analyzing the codestream to obtain in-loop identification information and out-of-loop identification information; processing the codestream in accordance with the in-loop identification information and the out-of-loop identification information; and decoding the processed codestream to generate a video image; the out-loop identification information is used to indicate whether to perform a third mapping transformation on the video image, the third mapping transformation including an out-loop luminance mapping transformation and a chromaticity mapping transformation; and the in-loop identification information is used to indicate whether to perform a fourth mapping transformation on the video image, the fourth mapping transformation including an in-loop luminance mapping transformation and a chromaticity mapping transformation. Image decoding method.

9. At least one of the in-loop identification information and the out-loop identification information is carried in at least one of network abstraction layer unit header information, sequence layer parameter set, picture layer parameter set, slice layer parameter set, supplemental enhancement information, video usability information, and custom unit information; The image decoding method according to claim 8.

10. The step of processing the codestream in accordance with the in-loop identification information and the out-loop identification information comprises: performing the fourth mapping transformation on the code stream in response to the in-loop identification information, and performing the third mapping transformation on the code stream that has been subjected to the fourth mapping transformation in response to the out-loop identification information; performing the fourth mapping transformation on the code stream in response to the in-loop identification information, and determining not to perform the third mapping transformation on the code stream that has been subjected to the fourth mapping transformation in response to the out-loop identification information; determining not to perform the fourth mapping transformation on the code stream in response to the in-loop identification information, and performing the third mapping transformation on the code stream in response to the out-loop identification information; determining not to perform the fourth mapping transformation on the codestream in response to the in-loop identification information; and determining not to perform the third mapping transformation on the codestream in response to the out-loop identification information. The image decoding method according to claim 8.

11. The luminance mapping transformation includes a mapping of luminance component values, the mapping of luminance component values ​​being used to change the luminance dynamic range, and the chromaticity mapping transformation includes a mapping of chromaticity component values ​​or a chromaticity scaling process dependent on the luminance mapping transformation. The image decoding method according to claim 10.

12. At least one of the in-loop identification information and the out-loop identification information includes control information and parameter information, wherein the control information is information for controlling whether or not to perform the third mapping transformation or the fourth mapping transformation, and the parameter information is parameter information used for the third mapping transformation or the fourth mapping transformation, the control information is carried in at least one of network abstraction layer unit header information, sequence layer parameter set, picture layer parameter set, slice layer parameter set, supplemental enhancement information, video usability information, and custom unit information, and the parameter information in the codestream is carried in at least one of a self-adaptation parameter set, supplemental enhancement information, and custom network abstraction layer unit. The image decoding method according to claim 8.

13. The parameter information of the in-loop identification information and the parameter information of the out-loop identification information in the code stream are each The parameter information of the in-loop identification information and the parameter information of the out-loop identification information are carried in a data packet for luma and chroma mapping conversion in the self-adaptive parameter set; The parameter information of the in-loop identification information is carried in a data packet for luma and chroma mapping transformation in the self-adaptive parameter set, and the parameter information of the out-loop identification information is carried in a data packet for custom supplemental enhancement information; the parameter information of the in-loop identity and the parameter information of the out-loop identity are carried in a data packet for a luma and chroma mapping transformation in the custom network abstraction layer unit; The image decoding method according to claim 12.

14. an image capture module configured to capture a video image and identify image characteristics of the video image; an identification module configured to identify in-loop identification information and out-of-loop identification information, respectively, in response to the image characteristics; an image processing module configured to process the video images according to the in-loop identification information and the out-of-loop identification information; an encoding processing module configured to encode the processed video images to generate a codestream and to write the in-loop identification information and the out-of-loop identification information into the codestream; the out-loop identification information is used to indicate whether to perform a first mapping transformation on the video image, the first mapping transformation including an out-loop luminance mapping transformation and a chromaticity mapping transformation; and the in-loop identification information is used to indicate whether to perform a second mapping transformation on the video image, the second mapping transformation including an in-loop luminance mapping transformation and a chromaticity mapping transformation. Image encoding device.

15. a codestream analysis module configured to obtain a codestream and analyze the codestream to obtain in-loop identification information and out-of-loop identification information; a codestream processing module configured to process the codestream in accordance with the in-loop identification information and the out-of-loop identification information; an image decoding module configured to decode the processed codestream to generate a video image; the out-loop identification information is used to indicate whether to perform a third mapping transformation on the video image, the third mapping transformation including an out-loop luminance mapping transformation and a chromaticity mapping transformation; and the in-loop identification information is used to indicate whether to perform a fourth mapping transformation on the video image, the fourth mapping transformation including an in-loop luminance mapping transformation and a chromaticity mapping transformation. Image decoding device.

16. one or more processors; a memory configured to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors realize the image coding method according to any one of claims 1 to 7 and / or the image decoding method according to any one of claims 8 to 13. electronic equipment.

17. A computer-readable storage medium having a computer program stored therein, the computer program being executed by a processor to realize the image encoding method according to any one of claims 1 to 7 and / or the image decoding method according to any one of claims 8 to 13. A computer-readable storage medium.

Images