Method of decoding bitstream, method of encoding video data in bitstream, decoder, encoder and storage medium

Abstract

The invention relates to a method of decoding a bitstream, a method of encoding video data in a bitstream, a decoder, an encoder and a storage medium. The invention also relates to a method of decoding video data from a bitstream, the bitstream comprising video data corresponding to a plurality of slices, where the bitstream comprises: a header comprising syntactic elements to be used in decoding one or more slices; wherein the decoding comprises analyzing the head of the picture; parsing at least one syntactic element associated with the APS ID prior to decoding the syntactic element associated with the tool; and decoding the bit stream using the syntactic element. Related encoding methods, apparatuses and computer programs are also disclosed.

Description

[0001] (This application is a divisional application of an application filed on December 18, 2020, with application number 2020800970779 and invention name “Advanced Syntax for Video Encoding and Decoding”.) Technical Field

[0002] The present invention relates to video encoding and decoding, and in particular to high-level syntax for use in bitstreams. Background Art

[0003] Recently, the Joint Video Experts Team (JVET) (a collaborative team consisting of MPEG and ITU-T Study Group 16 VCEG) began work on a new video coding standard called Versatile Video Coding (VVC). The goal of VVC is to provide significant improvements in compression performance over the existing HEVC standard (i.e., typically twice as good as before) and to be completed in 2020. Key target applications and services include, but are not limited to, 360-degree and high dynamic range (HDR) video. In total, JVET evaluated feedback from 32 organizations using formal subjective tests conducted by independent test labs. Some recommendations showed that compression efficiency was typically improved by 40% or more when compared to using HEVC. Particular results were shown on ultra-high definition (UHD) video test material. Therefore, for the final standard, we can expect compression efficiency improvements far exceeding the 50% target.

[0004] The JVET Exploration Model (JEM) uses all HEVC tools and has introduced several new tools. These changes require changes to the structure of the bitstream, especially the high-level syntax that may have an impact on the overall bitrate of the bitstream.

[0005] An important change in the high-level syntax is the introduction of a "picture header" into the bitstream. A picture header is a header that specifies the syntax elements to be used when decoding the individual slices in a particular picture (or frame). Therefore, the picture header is placed in the bitstream before the slice-related data, and each slice has its own "slice header". Figure 6 This structure is described in more detail.

[0006] The document JVET-P0239 entitled “AHG 17: PictureHeader” of the 16th meeting (October 1-11, 2019, Geneva, Switzerland) proposed the introduction of a mandatory picture header into VVC, and this was adopted as Generic Video Coding (Draft 7), uploaded as file JVET_P2001.

[0007] However, this header has a large number of parameters, all of which need to be parsed in order to use any particular encoding tool.

[0008] The present invention involves an improvement to the structure of a picture header to simplify the parsing process, which results in a reduction in complexity without any reduction in encoding performance.

[0009] In particular, by placing syntax elements related to the APS ID information at the beginning of the picture header, these elements may be parsed first, which may obviate the need to parse the rest of the header.

[0010] Similarly, in case there are syntax elements related to APS ID information in the slice header, these syntax elements are set at the beginning of the slice header.

[0011] In one example, it is proposed to move the syntax elements related to APS ID in the early stage of picture header and slice header. The purpose of this modification is to reduce the parsing complexity of some streaming applications that need to track APS ID in picture header and slice header to remove unused APS. The proposed modification has no impact on BDR performance.

[0012] This reduces the parsing complexity for streaming applications, where the APS ID information may be all the information needed for the header.For the same reason, other streaming related syntax elements may be moved towards the top of the header.

[0013] It should be understood that the term "beginning" does not mean the first entry in the corresponding header, as there may be multiple introductory syntax elements before the syntax element related to the APS ID information. The detailed description sets forth various examples, but the general definition is that the syntax elements related to the APS ID information are provided before the syntax elements related to the decoding tool. In a specific example, the syntax elements related to the APS ID of the ALF, LMCS, and scaling lists are provided just after the poc_msb_val syntax element. Summary of the invention

[0014] According to an aspect of the present invention, there is provided a method for decoding video data from a bitstream, the bitstream including video data corresponding to one or more slices, wherein the bitstream includes a header, the header including syntax elements to be used when decoding the one or more slices, wherein the decoding includes: parsing at least one syntax element related to an APS ID decoding tool from the header before parsing syntax elements dedicated to decoding the decoding tool; and decoding the bitstream using the syntax elements.

[0015] In another aspect of the present invention, a method of encoding video data into a bitstream is provided, the bitstream comprising video data corresponding to one or more slices, wherein the bitstream comprises a header, the header comprising syntax elements to be used when decoding the one or more slices, and the encoding comprises: encoding the header using at least one syntax element related to an APS ID decoding tool before a syntax element dedicated to decoding the decoding tool.

[0016] The following are optional features for both the aforementioned decoding method and encoding method aspects.

[0017] Optionally, the header is a picture header comprising syntax elements to be used when decoding multiple slices in a picture.

[0018] Optionally, at least one syntax element related to the APS ID is located after the syntax element indicating that the current picture has never been used as a reference picture.

[0019] Optionally, at least one syntax element related to the APS ID is located after the syntax element indicating that the current picture is a gradual decoding refresh picture.

[0020] Optionally, at least one syntax element related to the APS ID is located after the syntax element indicating a change of a previously decoded picture in the decoded picture buffer.

[0021] Optionally, at least one syntax element related to the APS ID is located after the picture type information.

[0022] Optionally, at least one syntax element related to the APS ID is located after the picture parameter set information.

[0023] Optionally, the header is a stripe header.

[0024] Optionally, at least one syntax element related to the APS ID is located after the sub-picture information.

[0025] Optionally, at least one syntax element related to the APS ID is located after the syntax element indicating the sub-picture ID of the current slice.

[0026] Optionally, at least one syntax element related to the APS ID is located after the syntax element indicating the address of the slice.

[0027] Optionally, at least one syntax element related to the APS ID is located after the syntax element indicating the number of blocks in the slice.

[0028] Optionally, at least one syntax element related to the APS ID is located after the syntax element indicating the type of the current slice.

[0029] Optionally, syntax elements dedicated to decoding a decoding tool include low-level tool parameters.

[0030] Optionally, the low-level tool parameters include deblocking filter parameters.

[0031] Optionally, the low-level tool parameters include quantization parameters.

[0032] Optionally, the low-level tool parameters include SAO parameters.

[0033] Optionally, the low-level tool parameters comprise a syntax element indicating that the signs of the residual samples of the two colors have inverted signs.

[0034] Optionally, the low-level tool parameters include motion parameters.

[0035] Optionally, the low-level tool parameters include QP offset parameters.

[0036] Optionally, the low-level tool parameters include partitioning parameters.

[0037] Optionally, the low-level tool parameters include reference picture list parameters.

[0038] Optionally, the low-level tool parameters include a syntax element indicating that the current picture is an output picture.

[0039] Optionally, the low-level tool parameters comprise a syntax element indicating a color plane index.

[0040] Optionally, the low-level tool parameters include a CABAC initialization flag.

[0041] Optionally, the low-level tool parameters include concatenated prediction parameters.

[0042] Optionally, the low-level tool parameters include weighted forecast parameters.

[0043] Optionally, the at least one syntax element related to the APS ID includes an APS ID LMCS.

[0044] Optionally, the at least one syntax element related to the APS ID comprises an APS ID scaling list.

[0045] Optionally, the APS scaling list contains information of a scaling matrix for enabling the LNST method.

[0046] Optionally, the at least one syntax element related to the APS ID includes an APS ID ALF list.

[0047] Optionally, the APS ALF list contains information related to signaling clipping values.

[0048] According to an aspect of the present invention, a method for encoding video data into a bitstream is provided, the bitstream comprising video data corresponding to a plurality of pictures and an APS transmission, the encoding comprising: signaling the presence of a resynchronization picture into the bitstream, resetting the APS ID based on the signaling so that the APS ID of a previous picture is not used in a subsequent picture.

[0049] Optionally, determining whether the picture is related to the resynchronization process includes: determining whether the picture is an IRAP picture or a GDR picture.

[0050] According to an aspect of the present invention, there is provided a decoder for decoding video data from a bitstream, the bitstream comprising video data corresponding to one or more slices, wherein the bitstream comprises a header, the header comprising syntax elements to be used when decoding the one or more slices, wherein the decoder comprises: a component for parsing at least one syntax element related to an APS ID decoding tool from the header before parsing syntax elements specific to the decoding tool; and a component for decoding the bitstream using the syntax elements.

[0051] According to an aspect of the present invention, there is provided an encoder for encoding video data into a bitstream, the bitstream comprising video data corresponding to one or more slices, wherein the bitstream comprises a header, the header comprising syntax elements to be used when decoding the one or more slices, the encoder comprising: means for encoding the header using at least one syntax element related to an APS ID decoding tool before a syntax element dedicated to decoding the decoding tool.

[0052] According to yet another aspect of the present invention, there is provided an encoder for encoding video data into a bitstream, the bitstream comprising video data corresponding to a plurality of pictures and an APS transmission, the encoder comprising: a component for signaling the presence of a resynchronization picture into the bitstream; and a component for resetting the APS ID based on the signaling so that the APS ID of a previous picture is not used in a subsequent picture.

[0053] In another aspect of the present invention, a computer-readable storage medium is provided, which stores a program or an instruction, and when the program or the instruction is executed by a programmable device, the program or the instruction causes the programmable device to perform the above method.

[0054] According to one aspect of the present invention, a method for decoding video data from a bitstream is provided, the bitstream comprising video data corresponding to a plurality of slices, wherein the bitstream comprises a header, the header comprising syntax elements to be used when decoding one or more slices, wherein the decoding comprises: parsing a picture header; parsing at least one syntax element associated with an APS ID before syntax elements associated with a decoding tool; and decoding the bitstream using the syntax elements.

[0055] This provides simpler and faster parsing processing, especially for streaming applications.

[0056] In a related aspect of the present invention, a method of encoding video data into a bitstream is provided, the bitstream including video data corresponding to a plurality of slices, wherein the bitstream includes: a header including syntax elements to be used when decoding one or more slices; the encoding includes encoding the picture header using syntax elements associated with an APS ID before syntax elements associated with a decoding tool.

[0057] Optionally, the at least one APS ID related syntax element includes an APS ID LMCS.

[0058] Optionally, the at least one APS ID related syntax element includes an APS ID scaling list.

[0059] Optionally, the APS scaling list contains information enabling a scaling matrix for the LNST method.

[0060] Optionally, the at least one APS ID related syntax element comprises an APS ID ALF list. Preferably, wherein the APS ALF list contains information related to signaling of a clipping value.

[0061] To reduce parsing complexity, at least one APS ID related syntax element may follow a syntax element independent of the value from any other header.

[0062] Optionally, the header is a picture header including syntax elements to be used when decoding multiple slices in a picture.

[0063] Optionally, at least one APS ID-related syntax element follows one or more of the following: a syntax element indicating that the current picture has never been used as a reference picture, a syntax element indicating that the current picture is a gradually decoded refresh picture, a syntax element indicating a change of a previously decoded picture in a decoded picture buffer, picture parameter set information, sub-picture information, or picture type information.

[0064] To facilitate parsing, at least one APS ID related syntax element may follow the fixed length syntax element. This allows the APS ID related syntax elements to be parsed first, as they are always in the same position in the header.

[0065] Optionally, syntax elements associated with a decoding tool include low-level tool parameters.

[0066] Optionally, the low-level tool parameters include one or more of the following: a deblocking filter parameter, a quantization parameter, an SAO parameter, a syntax element indicating that the signs of residual samples of two colors have inverted signs, a motion parameter, a QP offset parameter, a partition parameter, a reference picture list parameter, a syntax element indicating that the current picture is an output picture, or a syntax element indicating a color plane index.

[0067] Optionally, the method further comprises determining whether the picture is relevant for a resynchronization process, and if so, decoding comprises resetting the APS ID.

[0068] According to another aspect of the present invention, a method for decoding video data from a bitstream is provided, the bitstream comprising video data corresponding to a plurality of slices, wherein the bitstream comprises: a header comprising syntax elements used when decoding one or more slices; wherein the decoding comprises parsing the picture header; determining whether the picture is related to resynchronization processing based on a signal in the bitstream, and if so, the decoding comprises resetting the APS ID.

[0069] This provides flexibility in the use of resynchronization frames while simplifying the parsing process.

[0070] Optionally, the APS associated with the APS ID is not used in the decoding process of future decoded pictures.

[0071] According to another aspect of the present invention, a method for encoding video data into a bitstream is provided, the bitstream comprising video data corresponding to a plurality of pictures and an APS transmission; the encoding comprising: signaling the presence of a resynchronization picture into the bitstream, resetting the APS ID based on the signaling so that the APS ID of the previous picture is not used in a subsequent picture.

[0072] Optionally, determining whether the picture is relevant to the resynchronization process includes determining whether the picture is an IRAP picture or a GDR picture.

[0073] Optionally, the header is a stripe header.

[0074] Optionally, at least one APS ID-related syntax element follows one or more of the following: a syntax element indicating the POC of the current slice, a syntax element indicating the sub-picture ID of the current slice, a syntax element indicating the address of the slice, a syntax element indicating the number of blocks in the slice, or a syntax element indicating the type of the current slice.

[0075] Optionally, syntax elements associated with a decoding tool include low-level tool parameters.

[0076] Optionally, the low-level tool parameters include one or more of the following: reference picture list parameters, CABAC initialization flags, collocated prediction parameters, weighted prediction parameters, QP parameters or SAO parameters.

[0077] Optionally, further aspects of the present invention relate to an encoder and a decoder as defined to perform the above encoding and decoding methods, respectively.

[0078] Further aspects of the invention relate to a program which, when executed, causes the method in any of the aforementioned method aspects to be performed. The program may be provided separately, or may be carried by or in a carrier medium. The carrier medium may be non-transitory, such as a storage medium, in particular a computer-readable storage medium. The carrier medium may also be transient, such as a signal or other transmission medium. The signal may be transmitted via any suitable network including the Internet.

[0079] Further characteristics of the invention are characterized by the other independent and dependent claims.

[0080] Any features in one aspect of the invention may be applied to other aspects of the invention in any appropriate combination. In particular, method aspects may be applied to apparatus aspects, and vice versa.

[0081] Furthermore, features implemented in hardware may be implemented in software and vice versa. Any references herein to software and hardware features should be interpreted accordingly.

[0082] Any apparatus features as described herein may also be provided as method features, and vice versa.As used herein, means-plus-function features may alternatively be expressed in terms of their corresponding structure (such as a suitably programmed processor and associated memory, etc.).

[0083] It will also be appreciated that specific combinations of the various features described and defined in any aspect of the present invention may be independently implemented, provided and / or used. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] Reference will now be made, by way of example, to the accompanying drawings, in which:

[0085] Figure 1is a diagram for explaining the coding structure used in HEVC and VVC;

[0086] Figure 2 is a block diagram schematically illustrating a data communication system in which one or more embodiments of the present invention may be implemented;

[0087] Figure 3 is a block diagram illustrating components of a processing device that may implement one or more embodiments of the present invention;

[0088] Figure 4 is a flow chart showing the steps of an encoding method according to an embodiment of the present invention;

[0089] Figure 5 is a flow chart showing the steps of a decoding method according to an embodiment of the present invention;

[0090] Figure 6 shows the structure of a bit stream in an exemplary coding system VVC;

[0091] Figure 7 Luma Modelling Chroma Scaling (LMCS) is shown;

[0092] Figure 8 Shows the sub-tools of LMCS;

[0093] Fig. 9 is a diagram showing a system including an encoder or a decoder and a communication network according to an embodiment of the present invention;

[0094] Fig.10 is a schematic block diagram of a computing device for implementing one or more embodiments of the present invention;

[0095] Fig.11 is a diagram showing a network camera system; and

[0096] Fig.12 is a diagram showing a smart phone. DETAILED DESCRIPTION

[0097] Figure 1 The invention relates to a coding structure used in the High Efficiency Video Coding (HEVC) video standard. A video sequence 1 consists of a series of digital images i. Each of these digital images is represented by one or more matrices. The matrix coefficients represent pixels.

[0098] The images 2 of the sequence may be partitioned into slices 3. In some cases, a slice may constitute the entire image. The slices are partitioned into non-overlapping coding tree units (CTUs). The coding tree unit (CTU) is the basic processing unit of the High Efficiency Video Coding (HEVC) video standard and conceptually corresponds in structure to the macroblock unit used in several previous video standards. The CTU is sometimes also referred to as the largest coding unit (LCU). The CTU has luma and chroma component parts, each of which is called a coding tree block (CTB). These different color components are not in the same Figure 1 Shown in.

[0099] A CTU is typically 64 pixels x 64 pixels in size. Each CTU may be iteratively partitioned into smaller variable-sized coding units (CUs) 5 using a quadtree decomposition.

[0100] The coding unit is the basic coding element and consists of two subunits called prediction units (PUs) and transform units (TUs). The maximum size of a PU or TU is equal to the CU size. A prediction unit corresponds to a partition of a CU for prediction of pixel values. Various different partitions of a CU into PUs are possible, as shown in 606, including a partition into 4 square PUs, and two different partitions into 2 rectangular PUs. A transform unit is a basic unit for spatial transformation using DCT. A CU can be partitioned into TUs based on a quadtree representation 607.

[0101] Each slice is embedded in a network abstraction layer (NAL) unit. In addition, the coding parameters of the video sequence are stored in a dedicated NAL unit called a parameter set. In HEVC and H.264 / AVC, two parameter set NAL units are used: first, the sequence parameter set (SPS) NAL unit, which collects all parameters that remain unchanged during the entire video sequence. Generally, it handles the encoding profile, the size of the video frame, and other parameters. Second, the picture parameter set (PPS) NAL unit, which includes parameters that can be changed from one image (or frame) of the sequence to other images (or frames). HEVC also includes a video parameter set (VPS) NAL unit, which contains parameters describing the overall structure of the bitstream. VPS is a new type of parameter set defined in HEVC and is applied to all layers of the bitstream. A layer can contain multiple temporal sublayers, and all version 1 bitstreams are limited to a single layer. HEVC has certain hierarchical extensions for scalability and multi-view, and these extensions will allow multiple layers with a backward compatible version 1 base layer.

[0102] Figure 2The data communication system in which one or more embodiments of the present invention can be implemented is illustrated. The data communication system includes a transmission device (in this case, a server 201) which is operable to transmit data packets of a data stream to a receiving device (in this case, a client terminal 202) via a data communication network 200. The data communication network 200 can be a wide area network (WAN) or a local area network (LAN). Such a network can be, for example, a wireless network (Wifi / 802.11a or b or g), an Ethernet network, an Internet network, or a hybrid network consisting of several different networks. In a specific embodiment of the present invention, the data communication system can be a digital television broadcasting system, in which the server 201 sends the same data content to multiple clients.

[0103] The data stream 204 provided by the server 201 may consist of multimedia data representing video and audio data. In some embodiments of the present invention, the audio and video data streams may be captured by the server 201 using a microphone and a camera, respectively. In some embodiments, the data streams may be stored on the server 201 or received by the server 201 from other data providers, or generated at the server 201. The server 201 is provided with an encoder for encoding the video and audio streams, in particular for providing a compressed bit stream for transmission, which is a more compact representation of the data presented as input to the encoder.

[0104] In order to obtain a better ratio of quality of transmitted data to the amount of transmitted data, the video data may be compressed, for example, according to the HEVC format or the H.264 / AVC format.

[0105] The client 202 receives the transmitted bitstream and decodes the reconstructed bitstream to reproduce the video image on the display device and reproduce the audio data using the speaker.

[0106] Despite Figure 2 A streaming scenario is considered in the example of , but it will be appreciated that in some embodiments of the invention, data communication between the encoder and the decoder may be carried out using, for example, a media storage device (such as an optical disc).

[0107] In one or more embodiments of the present invention, a video image is transmitted along with data representing a compensating offset to be applied to reconstructed pixels of the image to provide filtered pixels in the final image.

[0108] Figure 3 The processing device 300 configured to implement at least one embodiment of the present invention is schematically illustrated. The processing device 300 may be a device such as a microcomputer, a workstation, or a light portable device.

[0109] The apparatus 300 comprises a communication bus 313 connected to:

[0110] - a central processing unit 311 denoted as CPU, such as a microprocessor or the like;

[0111] - a read-only memory 306, denoted ROM, for storing a computer program implementing the invention;

[0112] - a random access memory 312, represented as a RAM, for storing executable codes of the method of the embodiment of the present invention, and registers suitable for recording variables and parameters required for implementing the method for encoding a digital image sequence and / or the method for decoding a bit stream according to the embodiment of the present invention; and

[0113] A communication interface 302 connected to a communication network 303, via which digital data to be processed are transmitted or received.

[0114] Optionally, the device 300 may further include the following components:

[0115] - a data storage component 304, such as a hard disk, for storing a computer program for implementing the method of one or more embodiments of the present invention and data used or generated during the implementation of one or more embodiments of the present invention;

[0116] a disk drive 305 for a disk 306, which is suitable for reading data from the disk 306 or writing data to said disk;

[0117] - A screen 309 for displaying data and / or serving as a graphical interface for interaction with a user by means of a keyboard 310 or any other pointing means.

[0118] Device 300 may be connected to various peripheral devices such as digital camera 320 or microphone 308 , each of which is connected to an input / output card (not shown) to provide multimedia data to device 300 .

[0119] The communication bus provides communication and interoperability between the various elements included in or connected to the device 300. The representation of a bus is not limiting, and in particular, the central processing unit is operable to communicate instructions to any element of the device 300 directly or via other elements of the device 300.

[0120] The disk 306 may be replaced by any information medium, such as a rewritable or non-rewritable compact disk (CD-ROM), a ZIP disk or a memory card, and, in general, by an information storage component that can be read by a microcomputer or a microprocessor, the disk 306 being integrated into the device or not, possibly removable and suitable for storing one or more programs whose execution enables the implementation of the method for encoding a digital image sequence and / or the method for decoding a bit stream according to the present invention.

[0121] The executable code may be stored in a read-only memory 306, on a hard disk 304 or on a removable digital medium such as, for example, a disk 306 as previously described, etc. According to a variant, the executable code of the program may be received via the interface 302 by means of a communication network 303 to be stored in one of the storage means of the device 300, such as a hard disk 304, etc., before being executed.

[0122] The central processing unit 311 is adapted to control and direct the execution of instructions or parts of software code of one or more programs according to the invention, the execution of instructions stored in one of the above-mentioned storage means. At power-on, one or more programs stored in non-volatile memory (e.g., on hard disk 304 or in read-only memory 306) are transferred to random access memory 312 (which then contains the executable code of one or more programs) and registers for storing variables and parameters necessary for implementing the invention.

[0123] In this embodiment, the device is a programmable device that implements the invention using software. Alternatively, however, the invention may be implemented in hardware (for example in the form of an application specific integrated circuit or ASIC).

[0124] Figure 4 Block diagram illustrating an encoder according to at least one embodiment of the invention. The encoder is represented by connected modules, each module being adapted to implement, e.g. in the form of programming instructions executed by the CPU 311 of the apparatus 300, at least one corresponding step of a method for implementing at least one embodiment of encoding an image in a sequence of images according to one or more embodiments of the invention.

[0125] The encoder 400 receives a digital image i 0 to i n The original sequence 401 of is taken as input. Each digital image is represented by a set of samples (called pixels).

[0126] After implementing the encoding process, the encoder 400 outputs a bitstream 410. The bitstream 410 includes a plurality of coding units or slices, each slice including a slice header for transmitting an encoding value of an encoding parameter used for slice encoding, and a slice body including encoded video data.

[0127] Module 402 takes the input digital image i 0 to i n 401 is partitioned into blocks of pixels. A block corresponds to an image portion and can have a variable size (e.g., 4×4, 8×8, 16×16, 32×32, 64×64, 128×128 pixels, and several rectangular block sizes can also be considered). A coding mode is selected for each input block. Two families of coding modes are provided: coding modes based on spatial prediction coding (intra-frame prediction) and coding modes based on temporal prediction (inter-frame coding, merge, skip). Possible coding modes are tested.

[0128] Module 403 implements an intra prediction process in which a given block to be encoded is predicted by a predictor calculated from neighboring pixels of the block to be encoded. If intra coding is selected, the selected intra predictor and an indication of the difference between the given block and its predictor are encoded to provide a residual.

[0129] Temporal prediction is implemented by a motion estimation module 404 and a motion compensation module 405. First, a reference image from a reference image set 416 is selected, and a portion of the reference image (also referred to as a reference region or image portion) is selected by the motion estimation module 404, which is the region closest to a given block to be encoded. The motion compensation module 405 then uses the selected region to predict the block to be encoded. The difference between the selected reference region and the given block (also referred to as a residual block) is calculated by the motion compensation module 405. The selected reference region is indicated by a motion vector.

[0130] Thus, in both cases (spatial and temporal prediction), the residual is calculated by subtracting the prediction from the original block.

[0131] In the intra prediction implemented by module 403, the prediction direction is encoded. In the temporal prediction, at least one motion vector is encoded. In the inter prediction implemented by modules 404, 405, 416, 418, 417, at least one motion vector or data for identifying such a motion vector is encoded for the temporal prediction.

[0132] If inter prediction is selected, information about the motion vector and the residual block is encoded. To further reduce the bit rate, the motion vector is encoded by the difference relative to the motion vector predictor, assuming that the motion is homogeneous. The motion vector predictor from the set of motion information predictors is obtained by the motion vector prediction and encoding module 417 from the motion vector field 418.

[0133] The encoder 400 further comprises a selection module 406 for selecting a coding mode by applying a coding cost criterion such as a rate-distortion criterion, etc. To further reduce redundancy, a transform such as DCT is applied to the residual block by a transform module 407, and then the obtained transform data is quantized by a quantization module 408 and entropy encoded by an entropy encoding module 409. Finally, the encoded residual block of the current block being encoded is inserted into a bitstream 410.

[0134] The encoder 400 also performs decoding of the encoded images to produce reference images for motion estimation of subsequent images. This enables the encoder and decoder receiving the bitstream to have the same reference frame. The inverse quantization module 411 performs inverse quantization of the quantized data, followed by an inverse transform by the inverse transform module 412. The inverse intra prediction module 413 uses the prediction information to determine which predictor to use for a given block, and the inverse motion compensation module 414 actually adds the residual obtained by module 412 to the reference area obtained from the reference image set 416.

[0135] Post filtering is then applied by module 415 to filter the reconstructed pixel frame. In an embodiment of the invention, a SAO loop filter is used, wherein a compensating offset is added to the pixel values ​​of the reconstructed pixels of the reconstructed image.

[0136] Figure 5 A block diagram of a decoder 60 according to an embodiment of the present invention is shown, which can be used to receive data from an encoder. The decoder is represented by connected modules, each module being suitable for implementing the corresponding steps of the method implemented by the decoder 60, for example in the form of programming instructions to be executed by the CPU 311 of the device 300.

[0137] The decoder 60 receives a bitstream 61 comprising coding units, each coding unit consisting of a header containing information about the encoded parameters and a body containing the encoded video data. Figure 6 The structure of the bitstream in VVC is described in more detail. Figure 4 As illustrated, for a given block, the coded video data is entropy encoded on a predetermined number of bits and the index of the motion vector predictor is encoded. The received coded video data is entropy decoded by module 62. The residual data is then dequantized by module 63, after which an inverse transform is applied by module 64 to obtain pixel values.

[0138] Mode data indicating an encoding mode is also entropy-decoded, and based on the mode, an encoding block of the image data is intra-type decoded or inter-type decoded.

[0139] In the case of intra mode, the intra inverse prediction module 65 determines the intra predictor based on the intra prediction mode specified in the bitstream.

[0140] If the mode is inter, motion prediction information is extracted from the bitstream to find the reference region used by the encoder. The motion prediction information consists of a reference frame index and a motion vector residual. The motion vector predictor is added to the motion vector residual to obtain the motion vector by the motion vector decoding module 70.

[0141] The motion vector decoding module 70 applies motion vector decoding to each current block encoded by motion prediction. Once the index of the motion vector predictor for the current block has been obtained, the actual value of the motion vector associated with the current block can be decoded and used to apply inverse motion compensation by module 66. The reference image portion indicated by the decoded motion vector is extracted from the reference image 68 to apply inverse motion compensation 66. The motion vector field data 71 is updated with the decoded motion vector for inverse prediction of subsequently decoded motion vectors.

[0142] Finally, a decoded block is obtained. Post filtering is applied by a post filtering module 67. The decoder 60 finally provides a decoded video signal 69.

[0143] Figure 6 The organization of the bitstream in an exemplary coding system VVC as described in JVET_P2001-VE is shown.

[0144] The bitstream 61 according to the VVC coding system consists of an ordered sequence of syntax elements and encoded data. The syntax elements and the encoded data are placed into network abstraction layer (NAL) units 601-608. There are different NAL unit types. The network abstraction layer provides the ability to encapsulate the bitstream into different protocols (such as RTP / IP (standing for Real Time Protocol / Internet Protocol), ISO basic media file format, etc.). The network abstraction layer also provides a framework for anti-packet loss.

[0145] NAL units are divided into video coding layer (VCL) NAL units and non-VCL NAL units. VCL NAL units contain actual coded video data. Non-VCL NAL units contain additional information. The additional information can be parameters required for decoding the coded video data or supplementary data that can enhance the usability of the decoded video data. NAL units 606 correspond to slices and constitute VCL NAL units of the bitstream.

[0146] Different NAL units 601-605 correspond to different parameter sets, which are non-VCL NAL units. The decoder parameter set (DPS) NAL unit 301 contains parameters that are constant for a given decoding process. The video parameter set (VPS) NAL unit 602 contains parameters defined for the entire video and therefore the entire bitstream. The DPS NAL unit can define parameters that are more static than those in the VPS. In other words, the parameters of the DPS change less frequently than the parameters of the VPS.

[0147] The sequence parameter set (SPS) NAL unit 603 contains parameters defined for a video sequence. In particular, the SPS NAL unit may define the sub-picture layout and associated parameters of a video sequence. The parameters associated with each sub-picture specify the coding constraints applied to the sub-picture. In particular, a flag indicating that the temporal prediction between sub-pictures is restricted to data from the same sub-picture is included. Another flag may enable or disable the loop filter across sub-picture boundaries.

[0148] Picture parameter set (PPS) NAL unit 604, the PPS contains parameters defined for a picture or group of pictures. Adaptation parameter set (APS) NAL unit 605 contains parameters for the loop filter, which is typically an adaptive loop filter (ALF) or a shaper model (or a luma mapping with chroma scaling (LMCS) model) or a scaling matrix used at the slice level.

[0149] The syntax of the PPS as proposed in the current version of VVC includes syntax elements that specify the size of the picture in units of luma samples and the partitioning of each picture into blocks and slices.

[0150] The PPS contains syntax elements that allow the slice position in the frame to be determined. Since a sub-picture forms a rectangular area in a frame, the set of slices, tile parts or tiles belonging to the sub-picture can be determined from the parameter set NAL unit. The PPS as an APS has an ID mechanism to limit the amount of the same PPS sent.

[0151] The main difference between the PPS and the picture header is its transmission, which is usually sent for a group of pictures, compared to the PH which is systematically sent for each picture. Therefore, in contrast to the PH, the PPS contains parameters that may be constant for several pictures.

[0152] The bitstream may also contain supplemental enhancement information (SEI) NAL units ( Figure 6The periodicity of the occurrence of these parameter sets in the bitstream is variable. A VPS defined for the entire bitstream may appear only once in the bitstream. In contrast, an APS defined for a slice may appear once for each slice in each picture. In practice, different slices may rely on the same APS, and therefore there are usually fewer APSs than slices in each picture. In particular, the APS is defined in the picture header. However, the ALF APS may be refined in the slice header.

[0153] The Access Unit Delimiter (AUD) NAL unit 607 separates two access units. An access unit is a collection of NAL units that may include one or more coded pictures with the same decoding timestamp. This optional NAL unit contains only one syntax element in the current VVC specification: pic_type, which indicates that the slice_type value is used for all slices of the coded pictures in the AU. If pic_type is set equal to 0, the AU contains only intra slices. If equal to 1, it contains P and I slices. If equal to 2, it contains B, P, or intra slices.

[0154] This NAL unit contains only one syntax element, pic-type.

[0155] Table 1 Syntax AUD

[0156]

[0157] In JVET-P2001-VE, pic-type is defined as follows:

[0158] "pic_type indicates that the slice_type values ​​of all slices of the coded pictures in the AU containing the AU delimiter NAL unit are members of the set listed in Table 2 for the given pic_type value. The value of pic_type shall be equal to 0, 1, or 2 in bitstreams conforming to this version of this specification. Other values ​​of pic_type are reserved for future use by ITUT|ISO / IEC. Decoders conforming to this version of this specification shall ignore the reserved values ​​of pic_type."

[0159] Table 2 Explanation of pic-type

[0160] pic_type Possible slice_type values ​​in AU 0 I 1 P,I 2 B,P,I

[0161] The PH NAL unit 608 is a picture header NAL unit that groups common parameters for a set of slices of one coded picture.A picture may refer to one or more APSs to indicate the AFL parameters, shaper models, and scaling matrices used by slices of the picture.

[0162] Each VCL NAL unit 606 contains a slice. A slice may correspond to an entire picture or a sub-picture, a single block, or multiple blocks or a fragment of a block. For example, Figure 3 A slice of comprises several blocks 620. A slice consists of a slice header 610 and a raw byte sequence payload RBSP 611, which contains the encoded pixel data encoded as encoded blocks 640.

[0163] The syntax of the PPS as proposed in the current version of VVC includes syntax elements that specify the size of a picture in units of luma samples and the partitioning of each picture in units of blocks and slices.

[0164] The PPS contains syntax elements that allow the slice positions in a frame to be determined.Since a sub-picture forms a rectangular area in a frame, the set of slices, tile parts or blocks belonging to a sub-picture can be determined from the parameter set NAL units.

[0165] NAL unit slice

[0166] The NAL unit slice layer contains a slice header and slice data, as shown in Table 3.

[0167] Table 3 Stripe layer syntax

[0168]

[0169] APS

[0170] The adaptation parameter set (APS) NAL unit 605 is defined in Table 4 showing the syntax elements.

[0171] As depicted in Table 4, there are 3 possible types of APS given by the aps_params_type syntax element:

[0172] ALF_AP: used for ALF parameters

[0173] LMCS_APS: used for LMCS parameters

[0174] SCALLING_APS: used for scaling list related parameters

[0175] Table 4 Adaptive parameter set syntax

[0176]

[0177]

[0178] These three types of APS parameters are discussed below in turn.

[0179] ALF APS

[0180] The ALF parameters are described in the adaptive loop filter data syntax element (Table 5). First, two flags are dedicated to specify whether the ALF filter is sent for luma and / or chroma. If the luma filter flag is enabled, another flag is decoded to know whether the clipping value (alf_luma_clip_flag) is signaled. Then, the number of filters signaled is decoded using the alf_luma_num_filters_signalled_minus1 syntax element. If necessary, the syntax element representing the ALF coefficient increment "alf_luma_coeff_delta_idx" is decoded for each enabled filter. The absolute value and sign of each coefficient of each filter is then decoded.

[0181] If alf_luma_clip_flag is enabled, the clipping index for each coefficient of each enabled filter is decoded.

[0182] In the same way, the ALF chroma coefficients are decoded when needed.

[0183] Table 5 Adaptive loop filter data syntax

[0184]

[0185]

[0186] LMCS syntax elements for both luma mapping and chroma scaling

[0187] Table 6 below gives all LMCS syntax elements (LMCS_APS) encoded in the Adaptation Parameter Set (APS) syntax structure when the aps_params_type parameter is set to 1. Up to four LMCS APSs may be used in a coded video sequence, however, for a given picture only a single LMCS APS may be used.

[0188] These parameters are used to construct the forward and inverse mapping functions for luma and the scaling function for chroma.

[0189] Table 6 Luma mapping with chroma scaling data syntax

[0190]

[0191] Zoom List AP

[0192] The scaling list provides the possibility to update the quantization matrix used for quantization. In VVC, the scaling matrix is ​​in the APS as described in the scaling list data syntax element (Table 7). The syntax specifies whether the scaling matrix is ​​used for the LFNST (Low Frequency Non-Separable Transform) tool based on the flag scaling_matrix_for_lfnst_disabled_flag. Then, the syntax elements required to construct the scaling matrix (scaling_list_copy_mode_flag, scaling_list_pred_mode_flag, scaling_list_pred_id_delta, scaling_list_dc_coef, scaling_list_delta_coef) are decoded.

[0193] Table 7 Scaling List Data Syntax

[0194]

[0195]

[0196] Image header

[0197] The picture header is sent at the beginning of each picture. The picture header table syntax currently contains 82 syntax elements and about 60 test conditions or loops. This is very large compared to previous headers in previous drafts of the standard. A complete description of all these parameters can be found in JVET_P2001-VE. Table 8 shows these parameters in the current picture header decoding syntax. In this simplified table version, some syntax elements have been grouped for ease of reading.

[0198] The relevant syntactic elements that can be decoded are:

[0199] Whether to use the image or reference frame

[0200] Output frame

[0201] ●Use sub-images (if necessary)

[0202] · Reference image list (if required)

[0203] Color plane (if needed)

[0204] Partition update (if the overwrite flag is enabled)

[0205] Incremental QP parameters (if needed)

[0206] Motion information parameters (if required)

[0207] ALF parameters (if required)

[0208] SAO parameters (if required)

[0209] ●Quantization parameters (if necessary)

[0210] LMCS parameters (if required)

[0211] Scale list parameters (if needed)

[0212] Image header expansion (if necessary)

[0213] The first three flags with fixed length are non_reference_picture_flag, gdr_pic_flag, no_output_of_prior_pics_flag, which give information about the characteristics of the pictures within the bitstream.

[0214] Then, if gdr_pic_flag is enabled, recovery_poc_cnt is decoded.

[0215] The PPS parameters "PS_parameters()" sets the PPS ID and some other information (if necessary). It contains three syntax elements:

[0216] ph_pic_parameter_set_id

[0217] ph_poc_msb_present_flag

[0218] ·poc_msb_val

[0219] When sub-picture parameters Subpic_parameter() is enabled at the SPS and if the signaling of sub-picture ID is disabled, the sub-picture parameters Subpic_parameter() is enabled. It also contains some information about the virtual boundaries. For sub-picture parameters, eight syntax elements are defined:

[0220] ·ph_subpic_id_signalling_present_flag

[0221] ·ph_subpic_id_len_minus1

[0222] ph_subpic_id

[0223] ●ph_loop_filter_across_virtual_boundaries_disabled_present_flag

[0224] ●ph_num_ver_virtual_boundaries

[0225] ph_virtual_boundaries_pos_x

[0226] ·ph_num_hor_virtual_boundaries

[0227] ·ph_virtual_boundaries_pos_y

[0228] Then, if the sequence contains a single colour plane, the colour plane id "colour_plane_id" is decoded, followed by pic_output_flag if present.

[0229] The parameters of the reference picture list are then decoded into "reference_picture_list_parameters()", which contains the following syntax elements:

[0230] pic_rpl_present_flag

[0231] pic_rpl_sps_flag

[0232] pic_rpl_idx

[0233] pic_poc_lsb_lt

[0234] ·pic_delta_poc_msb_present_flag

[0235] pic_delta_poc_msb_cycle_lt

[0236] If necessary, another set of syntax elements related to the reference picture list structure "ref_pic_list_struct" containing eight other syntax elements can also be decoded.

[0237] If needed, the set of partitioning parameters "partitioning_parameters()" is decoded and contains the following 13 syntax elements:

[0238] ●partition_constraints_override_flag

[0239] ·pic_log2_diff_min_qt_min_cb_intra_slice_luma

[0240] ·pic_log2_diff_min_qt_min_cb_inter_slice

[0241] ●pic_max_mtt_hierarchy_depth_inter_slice

[0242] ·pic_max_mtt_hierarchy_depth_intra_slice_luma

[0243] ●pic_log2_diff_max_bt_min_qt_intra_slice_luma

[0244] ·pic_log2_diff_max_tt_min_qt_intra_slice_luma

[0245] ·pic_log2_diff_max_bt_min_qt_inter_slice

[0246] ·pic_log2_diff_max_tt_min_qt_inter_slice

[0247] ·pic_log2_diff_min_qt_min_cb_intra_slice_chroma

[0248] ●pic_max_mtt_hierarchy_depth_intra_slice_chroma

[0249] ·pic_log2_diff_max_bt_min_qt_intra_slice_chroma

[0250] ●pic_log2_diff_max_tt_min_qt_intra_slice_chroma

[0251] Following these partition parameters, four delta QP syntax elements "Delta_QP_Parameters()" may be decoded if needed:

[0252] ●pic_cu_qp_delta_subdiv_intra_slice

[0253] ●pic_cu_qp_delta_subdiv_inter_slice

[0254] ●pic_cu_chroma_qp_offset_subdiv_intra_slice

[0255] ·pic_cu_chroma_qp_offset_subdiv_inter_slice

[0256] Then, if necessary, pic_joint_cbcr_sign_flag is decoded, followed by a set of 3 SAO syntax elements "SAO_parameters()":

[0257] ·pic_sao_enabled_present_flag

[0258] pic_sao_luma_enabled_flag

[0259] ●pic_sao_chroma_enabled_flag

[0260] Then, if ALF is enabled at the SPS level, a set of ALF APS id syntax elements are decoded.

[0261] First, decode pic_alf_enabled_present_flag to determine whether pic_alf_enabled_flag should be decoded. If pic_alf_enabled_flag is enabled, ALF is enabled for all slices of the current picture.

[0262] If ALF is enabled, the amount of ALF APSid for luma is decoded using the pic_num_alf_aps_ids_luma syntax element. For each APS id, the APS id value for luma is decoded as "pic_alf_aps_id_luma".

[0263] For chroma, the syntax element pic_alf_chroma_idc is decoded to determine if ALF is enabled for chroma, only for Cr, or only for Cb. If enabled, the value of the APS id for chroma is decoded using the pic_alf_aps_id_chroma syntax element.

[0264] After the set of ALF APS id parameters, the quantization parameters of the picture header are decoded if necessary:

[0265] pic_dep_quant_enabled_flag

[0266] ●sign_data_hiding_enabled_flag

[0267] Then, if needed, a set of three deblocking filter syntax elements "deblocking_filter_parameters()" are decoded:

[0268] ●pic_deblocking_filter_override_present_flag

[0269] ●pic_deblocking_filter_override_flag

[0270] ●pic_deblocking_filter_disabled_flag

[0271] pic_beta_offset_div2

[0272] pic_tc_offset_div2

[0273] A set of LMCS APS ID syntax elements are decoded after LMCS is enabled at the SPS. First, pic_lmcs_enabled_flag is decoded to determine if LMCS is enabled for the current picture. If LMCS is enabled, the ID value is decoded as pic_lmcs_aps_id. For chroma, only pic_chroma_residual_scale_flag is decoded to enable or disable the method for chroma.

[0274] Then, if the scaling list is enabled at the SPS level, the set of scaling list APS IDs is decoded. The pic_scaling_list_present_flag is decoded to determine whether the scaling matrix is ​​enabled for the current picture. Then the value of the APS ID pic_scaling_list_aps_id is decoded.

[0275] Finally, if necessary, the picture header extension syntax element is decoded.

[0276] Table 8 Partial picture header

[0277]

[0278]

[0279]

[0280] Strip Header

[0281] The slice header is sent at the beginning of each slice. The slice header table syntax currently contains 57 syntax elements. This is very large compared to the previous slice header in the early version of the standard. A complete description of all slice header parameters can be found in JVET_P2001-VE. Table 9 shows these parameters in the current picture header decoding syntax. In this simplified table version, some syntax elements have been grouped to make the table more readable.

[0282] Table 9 Partial strip header

[0283]

[0284]

[0285]

[0286] First, slice_pic_order_cnt_lsb is decoded to determine the POC of the current slice.

[0287] Then, if necessary, decode slice_subpic_id to determine the sub-picture id of the current slice. Then, decode slice_address to determine the address of the current slice. If the number of tiles in the current picture is greater than 1, decode num_tiles_in_slice_minus1.

[0288] Then decode the slice_type.

[0289] Decode reference picture list parameter sets; these are similar to the parameter sets in the picture header.

[0290] When the slice type is not intra, and if necessary, cabac_init_flag and / or collocated_from_lo_flag and collocated_ref_idx are decoded. These data are related to collocated CABAC coding and motion vectors.

[0291] In the same manner, when the slice type is not intra, parameters of weighted prediction pred_weight_table() are decoded.

[0292] In slice_quantization_parameters(), slice_qp_delta is systematically decoded before other parameters of the QP offset if necessary.

[0293] slice_cb_qp_offset,slice_cr_qp_offset,slice_joint_cbcr_qp_offset,cu_chroma_qp_offset_enabled_flag.

[0294] The enabling flag of SAO is decoded for both luma and chroma: slice_sao_luma_flag, slice_sao_chroma_flag (if signaled in the picture header).

[0295] The APS ALF ID is then decoded if necessary, with similar data restrictions as in the picture header.

[0296] Then, the deblocking filter parameters are decoded before other data (slice_deblocking_filter_parameters()).

[0297] As in the picture header, the ALF APS ID is set at the end of the slice header.

[0298] APS in the image header

[0299] As depicted in Table 8, the APS ID information of the three tools ALF, LMCS, and scaling list are at the end on the picture header syntax element.

[0300] Streaming Applications

[0301] Some streaming applications extract only certain parts of the bitstream. These extractions can be spatial (as sub-pictures) or temporal (sub-parts of a video sequence). These extracted parts can then be merged with other bitstreams. Others reduce the frame rate by extracting only some frames. Typically, the main purpose of these streaming applications is to use the maximum allowed bandwidth to produce the highest quality for the end user.

[0302] In VVC, for frame rate reduction, APS ID numbering has been restricted so that new APS id numbering for a frame cannot be used for frames above in the temporal hierarchy. However, for streaming applications that extract portions of the bitstream, it is necessary to track APS IDs to determine which APSs should be retained for a sub-portion of the bitstream, since frames (because of IRAP) do not reset the numbering of APS IDs.

[0303] LMCS (Luminance Mapping with Chroma Scaling)

[0304] The Luma Mapping with Chroma Scaling (LMCS) technique is a sample value conversion method applied to a block before applying a loop filter in a video decoder such as VVC.

[0305] LMCS can be divided into two sub-tools. The first sub-tool is applied to luma blocks, while the second sub-tool is applied to chroma blocks, as described below:

[0306] 1) The first sub-tool is an in-loop mapping of the luminance component based on an adaptive piecewise linear model. The in-loop mapping of the luminance component adjusts the dynamic range of the input signal to improve compression efficiency by redistributing codewords across the dynamic range. The luminance mapping utilizes a forward mapping function into the "mapping domain" and a corresponding inverse mapping function back into the "input domain".

[0307] 2) The second sub-tool is related to the chroma components that apply luma-dependent chroma residual scaling. Chroma residual scaling is designed to compensate for the interaction between luma signals and their corresponding chroma signals. Chroma residual scaling depends on the average of neighboring luma samples reconstructed above and / or to the left of the current block.

[0308] Like most other tools in video encoders (such as VVC), LMCS can be enabled / disabled at the sequence level using the SPS flag. It is also signaled at the slice level whether chroma residual scaling is enabled. If luma mapping is enabled, an additional flag is signaled to indicate whether luma-dependent chroma residual scaling is enabled. When luma mapping is not used, luma-dependent chroma residual scaling is completely disabled. In addition, luma-dependent chroma residual scaling is always disabled for chroma blocks of size less than or equal to 4.

[0309] Figure 7 The principle of LMCS as described above for the Luma Mapping Sub-Tool is shown. Figure 7 The shaded blocks in the figure are the new LMCS functional blocks, which include the forward and inverse mapping of the luminance signal. It is important to note that when LMCS is used, some decoding operations are applied in the “mapping domain”. These operations are performed by the Figure 7 They usually correspond to the inverse quantization, inverse transform, luma intra prediction and reconstruction steps (which consists in adding the luma prediction to the luma residual). Figure 7 The solid blocks in indicate where the decoding process is applied in the original (ie non-mapped) domain, and this includes loop filtering such as deblocking, ALF and SAO, motion compensated prediction, and storage of decoded pictures as reference pictures (DPB).

[0310] Figure 8 Shown with Figure 7 Similar diagram, but this time for the Chroma Scaling sub-tool of the LMCS tool. Figure 8The shaded blocks in are the new LMCS functional blocks, which include the luma-dependent chroma scaling process. However, in terms of chroma, there are some important differences compared to the luma case. Here, for chroma samples, only inverse quantization and inverse transform, represented by the blocks in the dashed line, are performed in the "mapped domain". All other steps of intra chroma prediction, motion compensation, loop filtering are performed in the original domain. Figure 8 As shown, for brightness mapping, there is only a scaling process, and there is no forward and inverse process.

[0311] Brightness mapping using a piecewise linear model

[0312] The Luma Mapping sub-tool uses a piecewise linear model. This means that the piecewise linear model divides the input signal dynamic range into 16 equal sub-ranges, and for each sub-range, its linear mapping parameters are represented using the number of codewords assigned to that range.

[0313] Semantics of brightness mapping

[0314] The syntax element lmcs_min_bin_idx specifies the minimum bin (interval) index used in the construction process of the Luma Map with Chroma Scaling (LMCS). The value of lmcs_min_bin_idx shall be in the range of 0 to 15 (inclusive).

[0315] The syntax element lmcs_delta_max_bin_idx specifies the delta value between 15 and the maximum bin index LmcsMaxBinIdx used in the construction process of luma maps with chroma scaling. The value of lmcs_delta_max_bin_idx shall be in the range of 0 to 15, inclusive. The value of LmcsMaxBinIdx is set equal to 15 - lmcs_delta_max_bin_idx. The value of LmcsMaxBinIdx shall be greater than or equal to lmcs_min_bin_idx.

[0316] The syntax element lmcs_delta_cw_prec_minus1 plus 1 specifies the number of bits used to represent the syntax lmcs_delta_abs_cw[i].

[0317] The syntax element lmcs_delta_abs_cw[i] specifies the absolute delta codeword value for the i-th bin.

[0318] The syntax element lmcs_delta_sign_cw_flag[i] specifies the sign of the variable lmcsDeltaCW[i]. When lmcs_delta_sign_cw_flag[i] is not present, it is inferred to be equal to 0.

[0319] LMCS intermediate variable calculation for brightness mapping

[0320] In order to apply the forward and inverse brightness mapping processes, some intermediate variables and data arrays are required.

[0321] First, export the variable OrgCW as follows:

[0322] OrgCW=(1< <BitDepth) / 16

[0323] Then, the variable lmcsDeltaCW[i] (where i=lmcs_min_bin_idx . . . LmcsMaxBinIdx) is calculated as follows:

[0324] lmcsDeltaCW[i]=(1-2*lmcs_delta_sign_cw_flag[i])*lmcs_delta_abs_cw[i]

[0325] The new variable lmcsCW[i] is derived as follows:

[0326] - For i=0...lmcs_min_bin_idx-1, lmcsCW[i] is set equal to 0.

[0327] - For i = lmcs_min_bin_idx ... LmcsMaxBinIdx, the following applies:

[0328] lmcsCW[i]=OrgCW+lmcsDeltaCW[i]

[0329] The value of lmcsCW[i] should be in the range of (OrgCW>>3) to (OrgCW<<3-1) (inclusive).

[0330] - For i=LmcsMaxBinIdx+1...15, lmcsCW[i] is set equal to 0.

[0331] The variable InputPivot[i] (where i=0...16) is derived as follows:

[0332] InputPivot[i]=i*OrgCW

[0333] The variables LmcsPivot[i] (where i=0...16), ScaleCoeff[i] and InvScaleCoeff[i] (where i=0...15) are calculated as follows:

[0334] LmcsPivot[0] = 0;

[0335] for(i=0;i<=15;i++){

[0336] LmcsPivot[i+1]=LmcsPivot[i]+lmcsCW[i]

[0337] ScaleCoeff[i]=(lmcsCW[i]*(1<<11)+(1<<(Log2(OrgCW)-1)))>>(Log2(OrgCW))

[0338] if(lmcsCW[i]==0)

[0339] InvScaleCoeff[i]=0

[0340] else

[0341] InvScaleCoeff[i]=OrgCW*(1<<11) / lmcsCW[i]

[0342] Forward Luminance Map

[0343] like Figure 7 As shown, when LMCS is applied to luma, luma remap samples called predMapSamples[i][j] are obtained from the prediction samples predSamples[i][j].

[0344] predMapSamples[i][j] is calculated as follows:

[0345] First, the index idxY is calculated from the prediction sample predSamples[i][j] at position (i, j).

[0346] idxY=predSamples[i][j]>>Log2(OrgCW)

[0347] Then, predMapSamples[i][j] is derived as follows by using the intermediate variable idxY of part 0, LmcsPivot[idxY], and InputPivot[idxY]:

[0348] predMapSamples[i][j]=LmcsPivot[idxY]

[0349] +(ScaleCoeff[idxY]*(predSamples[i][j]-InputPivot[idxY])+(1<<10))>>11

[0350] Brightness reconstruction samples

[0351] Obtain the reconstruction process from the predicted luminance sample predMapSample[i][j] and the residual luminance sample resiSamples[i][j].

[0352] Simply obtain the reconstructed luminance picture sample recSamples[i][j] by adding predMapSample[i][j] to resiSamplei[i][j] as follows:

[0353] recSamples[i][j] = Clip1(predMapSamples[i][j] + resiSamples[i][j]])

[0354] In the above relationship, the Clip 1 function is a clipping function to ensure that the reconstructed samples are between 0 and 1<<BitDepth - 1.

[0355] Inverse luminance mapping

[0356] When applying the inverse luminance mapping according to Figure 7 the following operations are applied to each sample recSample[i][j] of the current block being processed:

[0357] First, calculate the index idxY from the reconstructed sample recSamples[i][j] at position (i, j).

[0358] idxY = recSamples[i][j] >> Log2(OrgCW)

[0359] The inverse mapped luminance sample invLumaSample[i][j] is derived based on the following:

[0360] invLumaSample[i][j] =

[0361] InputPivot[idxYInv] + (InvScaleCoeff[idxYInv] *

[0362] (recSample[i][j] - LmcsPivot[idxYInv]) + (1 << 10)) >> 11

[0363] Then a clipping operation is performed to obtain the final sample:

[0364] finalSample[i][j] = Clip1(invLumaSample[i][j])

[0365] Chroma scaling

[0366] LMCS semantics for chroma scaling

[0367] The syntax element lmcs_delta_abs_crs in Table 6 specifies the absolute codeword value of the variable lmcsDeltaCrs. The value of lmcs_delta_abs_crs shall be in the range of 0 to 7 (inclusive). When not present, lmcs_delta_abs_crs is inferred to be equal to 0.

[0368] The syntax element lmcs_delta_sign_crs_flag specifies the sign of the variable lmcsDeltaCrs. When not present, lmcs_delta_sign_crs_flag is inferred to be equal to 0.

[0369] LMCS intermediate variable calculation for chroma scaling

[0370] In order to apply the chroma scaling process, some intermediate variables are needed.

[0371] The variable lmcsDeltaCrs is derived as follows:

[0372] lmcsDeltaCrs=(1-2*lmcs_delta_sign_crs_flag)*lmcs_delta_abs_crs

[0373] The variable ChromaScaleCoeff[i] (where i=0...15) is derived as follows:

[0374] if(lmcsCW[i]==0)

[0375] ChromaScaleCoeff[i]=(1<<11)

[0376] else

[0377] ChromaScaleCoeff[i]=OrgCW*(1<<11) / (lmcsCW[i]+lmcsDeltaCrs)

[0378] Chroma Scaling

[0379] In the first step, the variable invAvgLuma is derived to calculate the average luma value of the reconstructed luma samples around the current corresponding chroma block. The average luma is calculated from the left and top luma blocks surrounding the corresponding chroma block.

[0380] If no samples are available, the variable invAvgLuma is set as follows:

[0381] invAvgLuma=1<<(BitDepth-1)

[0382] Based on the intermediate array LmcsPivot[] of part 0, the variable idxYInv is then derived as follows:

[0383] For(idxYInv=lmcs_min_bin_idx; idxYInv<=LmcsMaxBinIdx; idxYInv++){

[0384] if(invAvgLuma <LmcsPivot[idxYInv+1])break

[0385] }

[0386] IdxYInv=Min(idxYInv,15)

[0387] The variable varScale is exported as follows:

[0388] varScale=ChromaScaleCoeff[idxYInv]

[0389] When the transform is applied to the current chroma block, the reconstructed chroma picture sample array recSamples is derived as follows:

[0390] recSamples[i][j]=Clip1(predSamples[i][j]+

[0391] Sign(resiSamples[i][j])*((Abs(resiSamples[i][j])*varScale+(1<<10))>>11))

[0392] If no transform has been applied to the current block, the following applies:

[0393] recSamples[i][j]=Clip1(predSamples[i][j])

[0394] Encoder Considerations

[0395] The basic principle of the LMCS encoder is to first assign more codewords to those dynamic range segments with codewords with lower variance than the average. In this alternative concept, the main goal of LMCS is to assign fewer codewords to those dynamic range segments with codewords with higher variance than the average. In this way, smooth areas of the picture will be encoded with more codewords than the average, and vice versa.

[0396] All parameters of the LMCS tool stored in the APS are determined on the encoder side (see Table 6). The LMCS encoder algorithm is based on the evaluation of the local luminance variance and the determination of the LMCS parameters is optimized according to the above basic principles. The optimization is then performed to obtain the best PSNR metric for the final reconstructed samples of a given block.

[0397] As will be discussed below, the parsing complexity can be reduced by placing the APS ID information at the beginning of the slice / picture header. This allows the syntax elements related to the APS ID to be parsed before the syntax elements related to the decoding tool.

[0398] Image header

[0399] The following description provides details or alternatives to the following table of syntax elements for the picture header. Table 10 shows an example modification of the picture header (see Table 8 above) where the APS ID information for LMCS, ALF, and scaling lists is set close to the beginning of the picture header to address the parsing issues of the prior art. This change is shown by the strikethrough and underlined text. Using this syntax element table, tracking of the APS ID has lower complexity.

[0400] Table 10 shows the modified part of the picture header of APS ID information

[0401]

[0402]

[0403]

[0404]

[0405] APS ID related syntax elements may include one or more of the following:

[0406] APS ID LMCS

[0407] ●APS ID zoom list

[0408] ○ The APS scaling list contains information for the scaling matrix that enables the LNST method.

[0409] ●APS ID ALF list

[0410] ○ APS ALF list contains information related to the signaling of clipping values

[0411] In one example, the APS ID related syntax elements are placed at or near the beginning of the picture header.

[0412] The advantage of this feature is reduced complexity for streaming applications that need to track APS ID usage to remove useless APS NAL units.

[0413] As mentioned above, the word "beginning" does not necessarily mean "first", because some syntax elements may precede the APS ID related syntax elements. These will be discussed in turn below.

[0414] The set of APS ID related syntax elements is set after one or more of the following:

[0415] ●Syntax elements useful for streaming applications that need to parse the bitstream to cut or split or extract video sequence parts without parsing tool parameters.

[0416] ● A syntax element indicating that the current picture has never been used as a reference picture (e.g., non_reference_picture_flag), which when equal to 1 indicates that the picture has never been used as a reference picture. This information is useful for many streaming applications, so it is preferred to set this flag before the set of APS ID LMCS syntax elements.

[0417] ○ A syntax element indicating that the current picture is a progressive decoding refresh picture (e.g., gdr_pic_flag syntax), which when equal to 1 indicates that the current picture is a GDR picture. Because this type of picture means that the bitstream is independent of the previous picture. It is important information for many streaming applications. For the same reason, if gdr_pic_flag is enabled, the APS IDLMCS syntax element set should be after the decoded recovery_poc_cnt syntax element.

[0418] • Syntax elements indicating changes of previously decoded pictures in the decoded picture buffer (eg no_output_of_prior_pics_flag), which affects the decoded picture buffer (DBP). This is important information for many applications.

[0419] • ph_pic_parameter_set_id and / or other syntax elements related to PPS information. In practice, PPSid is needed for many applications.

[0420] ● Syntax elements related to sub-picture information. Actually, this information is important for bitstream extraction for some streaming applications.

[0421] ●Picture type information sent in the picture header: In fact, picture type is important information for many applications.

[0422] In a particularly advantageous example, a set of APS ID related syntax elements is arranged after the syntax elements at the beginning of the picture header (these syntax elements have fixed length codewords without conditions for their decoding). The fixed length codewords correspond to the syntax elements with the descriptor u(N) in the syntax element table, where N is an integer value.

[0423] Compared to the current VVC specification, it corresponds to the non_reference_picture_flag, gdr_pic_flag and no_output_of_prior_pics_flag flags.

[0424] This has the advantage that for streaming applications these first codewords can be easily bypassed (since the number of bits is always the same at the beginning of the picture header) and directly reach the useful information for the respective application. Thus, the decoder can parse the APS ID related syntax element directly from the header, since it is always in the same bit position in the header.

[0425] In a variation that reduces parsing complexity, a set of APS ID related syntax elements is placed at the beginning of the picture header after a syntax element that does not require one or more values ​​from another header. Parsing of such syntax elements does not require other variables for its parsing, which reduces the complexity of parsing.

[0426] In one variation, the APS ID scaling list syntax element set is placed at the beginning of the picture header.

[0427] The advantage of this variant is reduced complexity for streaming applications that need to keep track of APS ID usage to remove useless APS NAL units.

[0428] It should be appreciated that the above types of APS ID related syntax elements may be processed individually or in combination. A non-exhaustive list of particularly advantageous combinations is discussed below.

[0429] The APS ID LMCS syntax element set and the APS ID scaling list syntax element set are set at or near the beginning of the picture header, and optionally after useful information for streaming applications.

[0430] The APS ID LMCS syntax element set and the APS ID ALF syntax element set are set at the beginning of the picture header and optionally followed by useful information for streaming applications.

[0431] The APS ID LMCS syntax element set, the APS ID scaling list syntax element set, and the APS ID ALF syntax element set are set at or near the beginning of the picture header, and optionally after useful information for streaming applications.

[0432] In one embodiment, APS ID related syntax elements (APS LMCS, APS scaling list and / or APS ALF) are set before the low-level tool parameters.

[0433] This provides a reduction in complexity for some streaming applications that need to keep track of different APS IDs for individual pictures. In fact, the information associated with the low-level tools is required for parsing and decoding of slice data, but is not required for such applications that only extract the bitstream without decoding. An example of a low-level tool is provided below:

[0434] - Deblocking filter parameter set; these parameters are for the deblocking filter required for decoding of only slice data.

[0435] ●A set of quantization parameters; these parameters are required for parsing and decoding of slice data.

[0436] ●SAO parameter set; these parameters are required for slice data parsing and decoding.

[0437] ●pic_joint_cbcr_sign_flag flag; these parameters are required for slice data parsing and decoding.

[0438] ● Motion parameter set; these parameters are required for slice data parsing and decoding.

[0439] ●QP offset parameter set; these parameters are required for slice data parsing and decoding.

[0440] ●Partition parameter set; these parameters are required for stripe data parsing and decoding.

[0441] ● Reference picture list parameter set; these parameters are required for slice data parsing and decoding.

[0442] ●pic_output_fl flags; these parameters are required for slice data decoding.

[0443] ●colour_plane_id flag; these parameters are required for parsing and decoding of the slice data.

[0444] It should be appreciated that the above types of APS ID related syntax elements may be processed individually or in combination. A non-exhaustive list of particularly advantageous combinations is discussed below.

[0445] The APS ID LMCS syntax element set and the APS ID scaling list syntax element set are set before the low-level tool syntax elements.

[0446] The APS ID LMCS syntax element set and the APS ID ALF syntax element set are set before the low-level tool syntax elements.

[0447] The APS ID LMCS syntax element set, the APS ID scaling list syntax element set, and the APS ID ALF syntax element set are set before the low-level tool syntax elements.

[0448] Resync pictures

[0449] In the bitstream, there are some resynchronization frames or slices. By using these frames, the bitstream can be read without considering the previously decoded information. For example, a clean random access picture (CRA picture) does not refer to any picture other than itself for inter prediction in its decoding process. This means that the slices of this picture are intra or IBC. In VVC, CRA pictures are IRAP (Intra Random Access Point) pictures, where each VCL NAL unit has nal unit_type equal to CRA_NUT.

[0450] In VVC, these resynchronization frames can be IRAP pictures or GDR pictures.

[0451] However, an IRAP or GDR picture, or an IRAP slice for some configurations, may have an APS ID in its picture header that is derived from a previously decoded APS. Therefore, the APS ID should be tracked for real random access points or when splitting the bitstream.

[0452] One way to improve this problem is to reset all APS IDs and / or not allow APS to be taken from other decoded APSs when decoding an IRAP or GDR picture. APS can be used for LMCS or scaling list or ALF. Therefore, the decoder determines whether the picture is related to the resynchronization process, and if so, the decoding includes resetting the APS ID.

[0453] In a variation, when one slice in the picture is an IRAP, all APS IDs are reset.

[0454] A reset of the APS ID means that the decoder considers all previous APSs to be no longer valid, so there are no APS ID values ​​referring to previous APSs.

[0455] This resynchronization process is particularly advantageous in combination with the above-described syntax structure because the APS ID information can be more easily parsed and decoded, thereby enabling faster and more efficient resynchronization when necessary.

[0456] APS ID related information in the stripe header

[0457] The following description relates to modifications to the syntax element table for the slice header. In the table, the APS ID information of the ALF is set close to the beginning of the slice header to solve the parsing problem discussed in the above invention. The modified slice header in Table 11 is based on the description of the current slice header of the syntax element (see Table 9 above), where the modifications are shown in strikethrough and underline.

[0458] Table 11 shows the modified partial slice header

[0459]

[0460]

[0461]

[0462] APS ID ALF at the beginning of the stripe header

[0463] As shown in Table 11, the APS ID ALF syntax element set is set at or near the beginning of the slice header. Specifically, the ALF APS contains at least syntax elements related to signaling of a clipping value.

[0464] This provides a reduction in complexity for streaming applications that need to track APS ID usage to remove unused APS NAL units.

[0465] A collection of APS ID ALF syntax elements may be placed after syntax elements useful for streaming applications that need to parse or cut or split or extract video sequence portions without parsing tool parameters. An example of such syntax element is provided below:

[0466] slice_pic_order_cnt_lsb syntax element, which indicates the POC of the current slice. This information is useful for many streaming applications, so it is preferred to set this flag before the set of APS ID ALF syntax elements.

[0467] slice_subpic_id syntax element, which indicates the sub-picture ID of the current slice. This information is useful for many streaming applications, so it is preferred to set this flag before the set of APS ID ALF syntax elements.

[0468] slice_address syntax element, which indicates the address of the slice. This information is useful for many streaming applications, so it is preferred to set this flag before the set of APS ID ALF syntax elements.

[0469] · num_tiles_in_slice_minus syntax element. This information is useful for many streaming applications, so it is preferred to set this flag before the set of APS ID ALF syntax elements.

[0470] • slice_type element, which indicates the type of the current slice. This information is useful for many streaming applications, so it is preferred to set this flag before the set of APS ID ALF syntax elements.

[0471] In another variation, when the ALF APS contains at least syntax elements related to signaling of clipping values, the set of APS ID ALF syntax elements are placed before the low-level tool parameters in the slice header.

[0472] This provides a reduction in complexity for some streaming applications that need to track the different APS ALF APS IDs for each picture. In fact, the information related to the low-level tools is required for parsing and decoding of the slice data, but is not required for such applications that only extract the bitstream without decoding. An example of such a syntax element is provided below:

[0473] Reference picture list parameter set in the slice header. Actually, these parameters are required for decoding the slice data.

[0474] · CABAC initialization flag set in the slice header. Actually, this flag is only used for slice data decoding.

[0475] · A set of concatenated prediction parameters in the slice header. Actually, these parameters are required for decoding the slice data.

[0476] · Weighted prediction parameter set in the slice header. Actually, these parameters are required for decoding the slice data.

[0477] · The quantization parameter update parameter set in the slice header. Actually, these parameters are required for decoding the slice data.

[0478] SAO enabled flag set in the slice header. Actually, these parameters are required for slice data decoding.

[0479] It should be understood that the above features can be provided in combination with each other. As with the specific combinations discussed above, doing so can provide specific advantages suitable for specific implementations; for example, increased flexibility, or specifying a "worst case" example. In other examples, complexity requirements may have a higher priority than (for example) rate reduction, and therefore features can be implemented separately.

[0480] Implementation of the Invention

[0481] Fig. 9 A system 191, 195 according to an embodiment of the present invention is shown, which includes at least one of the encoder 150 or the decoder 100 and a communication network 199. According to an embodiment, the system 195 is used to process and provide content (e.g., video and audio content for display / output or streaming video / audio content) to a user, who accesses the decoder 100, for example, through a user terminal including the decoder 100 or a user interface of a user terminal that can communicate with the decoder 100. Such a user terminal can be a computer, a mobile phone, a tablet computer, or any other type of device capable of providing / displaying (provided / streamed) content to a user. The system 195 obtains / receives the bitstream 101 (in the form of a continuous stream or signal (e.g., when displaying / outputting earlier video / audio)) via the communication network 199. According to an embodiment, the system 191 is used to process content and store processed content, such as video and audio content processed for display / output / streaming at a later time. The system 191 obtains / receives content including an original image sequence 151, which is received and processed by the encoder 150 (including filtering using a deblocking filter according to the present invention), and the encoder 150 generates a bitstream 101 to be transmitted to the decoder 100 via the communication network 191. The bitstream 101 is then transmitted to the decoder 100 in a variety of ways, for example, it can be pre-generated by the encoder 150 and stored as data in a storage device in the communication network 199 (for example, on a server or cloud storage device) until the user requests the content (i.e., bitstream data) from the storage device, at which time the data is transmitted / streamed from the storage device to the decoder 100. The system 191 may also include a content providing device for providing / streaming to the user (for example, by transmitting data of a user interface to be displayed on a user terminal) content information (for example, the title of the content and other meta / storage location data for identifying, selecting and requesting the content) of the content stored in the storage device, and for receiving and processing user requests for content so that the requested content can be transmitted / streamed from the storage device to the user terminal. Alternatively, the encoder 150 generates the bitstream 101 and transmits / streams it directly to the decoder 100 when the user requests the content. The decoder 100 then receives the bitstream 101 (or signal) and filters it using the deblocking filter according to the present invention to obtain / generate a video signal 109 and / or an audio signal, which is then used by the user terminal to provide the requested content to the user.

[0482] Any step of the method / processing according to the present invention or the function described herein may be implemented in hardware, software, firmware or any combination thereof. If implemented in software, the step / function may be stored as one or more instructions or codes or programs or computer-readable media on one or more hardware-based processing units or sent via one or more hardware-based processing units, and executed by one or more hardware-based processing units, such as a programmable computing machine, which may be a PC ("personal computer"), a DSP ("digital signal processor"), a circuit, a circuit system, a processor and memory, a general-purpose microprocessor or central processing unit, a microcontroller, an ASIC ("application-specific integrated circuit"), a field programmable logic array (FPGA) or other equivalent integrated or discrete logic circuit system. Therefore, the term "processor" as used herein may refer to any of the aforementioned structures or any other structures suitable for implementing the technology described herein.

[0483] Embodiments of the present invention may also be implemented by various devices or apparatuses, including wireless handsets, integrated circuits (ICs), or JC collections (e.g., chipsets). Various components, modules, or units are described herein to illustrate functional aspects of the apparatus / device configured to perform these embodiments, but do not necessarily need to be implemented by different hardware units. Instead, the various modules / units may be combined in a codec hardware unit or provided by a collection of interoperable hardware units, which include one or more processors in conjunction with appropriate software / firmware.

[0484] Embodiments of the present invention can be implemented by a computer of a system or device that reads out and executes computer executable instructions (e.g., one or more programs) recorded on a storage medium to perform one or more modules / units / functions in the above-mentioned embodiments and / or includes one or more processing units or circuits for performing one or more functions in the above-mentioned embodiments, and can be implemented by a method performed by a computer of a system or device, for example, reading out and executing computer executable instructions from a storage medium to perform one or more functions in the above-mentioned embodiments and / or controlling one or more processing units or circuits to perform one or more functions in the above-mentioned embodiments. The computer may include a network of separate computers or separate processing units to read out and execute computer executable instructions. Computer executable instructions may be provided to a computer from a computer-readable medium such as a communication medium, for example, via a network or a tangible storage medium. The communication medium may be a signal / bit stream / carrier. A tangible storage medium is a “non-transitory computer-readable storage medium” which may include, for example, a hard disk, a random access memory (RAM), a read-only memory (ROM), a storage device of a distributed computing system, an optical disk (such as a compact disk (CD), a digital versatile disk (DVD), or a Blu-ray disk (BD) TM ), one or more of a flash memory device, a memory card, etc. At least some steps / functions may also be implemented in hardware by a machine or dedicated components such as an FPGA (“field programmable gate array”) or an ASIC (“application-specific integrated circuit”).

[0485] Fig.101300 is a schematic block diagram of a computing device 1300 for implementing one or more embodiments of the present invention. The computing device 1300 may be a device such as a microcomputer, a workstation, or a light portable device. The computing device 1300 includes a communication bus connected to the following: - a central processing unit (CPU) 1301, such as a microprocessor; - a random access memory (RAM) 1302 for storing executable code of the method of the embodiment of the present invention and registers suitable for recording variables and parameters required to implement the method for encoding or decoding at least a part of an image according to the embodiment of the present invention, and its storage capacity can be expanded, for example, by an optional RAM connected to an expansion port; - a read-only memory (ROM) 1303 for storing a computer program for implementing an embodiment of the present invention; - a network interface (NET) 1304, which is usually connected to a communication network through which the digital data to be processed is transmitted or received, and the network interface (NET) 1304 may be a single network interface, or composed of a group of different network interfaces (for example, wired and wireless interfaces, or different kinds of wired or wireless interfaces), which are connected to the CPU when running. 1301, data packets are written to the network interface for transmission or read from the network interface for reception; - a user interface (UI) 1305, which can be used to receive input from a user or display information to a user; - a hard disk (HD) 1306, which can be provided as a mass storage device; - an input / output module (IO) 1307, which can be used to receive / send data from / to an external device (such as a video source or a display, etc.). The executable code can be stored in ROM 1303, on HD 1306, or on a removable digital medium such as a disk, etc. According to a variant, the executable code of the program can be received via NET 1304 by means of a communication network to be stored in one of the storage components (such as HD 1306, etc.) of the communication device 1300 before being executed. CPU 1301 is suitable for controlling and directing the execution of instructions or parts of the software code of one or more programs according to an embodiment of the present invention, which instructions are stored in one of the aforementioned storage components. For example, after power-up, CPU 1301 is capable of executing those instructions relating to a software application from main RAM memory 1302 after loading instructions from program ROM 1303 or HD 1306. Such a software application, when executed by CPU 1301, causes the steps of the method according to the invention to be performed.

[0486] It should also be understood that according to other embodiments of the present invention, a decoder according to the above embodiments is provided in a user terminal such as a computer, a mobile phone (cellular phone), a tablet or any other type of device (e.g., a display device) capable of providing / displaying content to a user. According to yet another embodiment, an encoder according to the above embodiments is provided in an image capture device, which also includes a camera, a video camera or a webcam (e.g., a closed-circuit television or video surveillance camera) for capturing and providing content for encoding by the encoder. See below Fig.11 and 12 Two such examples are provided.

[0487] Network Camera

[0488] Fig.11 21 is a diagram illustrating a network camera system 2100 including a network camera 2102 and a client device 2104 .

[0489] The network camera 2102 includes an imaging unit 2106 , an encoding unit 2108 , a communication unit 2110 , and a control unit 2112 .

[0490] The network camera 2102 and the client device 2104 are connected to each other via the network 200 to be able to communicate with each other.

[0491] The camera unit 2106 includes a lens and an image sensor (eg, a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS)), and captures an image of a subject and generates image data based on the image. The image may be a still image or a video image.

[0492] The encoding unit 2108 encodes the image data by using the encoding method described above.

[0493] The communication unit 2110 of the network camera 2102 transmits the encoded image data encoded by the encoding unit 2108 to the client device 2104 .

[0494] In addition, the communication unit 2110 receives commands from the client device 2104. The commands include commands for setting parameters for encoding by the encoding unit 2108.

[0495] The control unit 2112 controls other units in the network camera 2102 according to the command received by the communication unit 2110 .

[0496] The client device 2104 includes a communication unit 2114 , a decoding unit 2116 , and a control unit 2118 .

[0497] The communication unit 2114 of the client device 2104 transmits a command to the network camera 2102 .

[0498] Furthermore, the communication unit 2114 of the client device 2104 receives the encoded image data from the network camera 2102 .

[0499] The decoding unit 2116 decodes the encoded image data by using the decoding method described above.

[0500] The control unit 2118 of the client device 2104 controls other units in the client device 2104 according to a user operation or command received by the communication unit 2114 .

[0501] The control unit 2118 of the client device 2104 controls the display device 2120 to display the image decoded by the decoding unit 2116 .

[0502] The control unit 2118 of the client device 2104 also controls the display device 2120 to display a GUI (Graphical User Interface) for specifying values ​​of parameters of the network camera 2102 (including parameters for encoding by the encoding unit 2108 ).

[0503] The control unit 2119 of the client device 2104 also controls other units in the client device 2104 according to user operation input to the GUI displayed by the display device 2120 .

[0504] The control unit 2118 of the client device 2104 controls the communication unit 2114 of the client device 2104 according to the user operation input to the GUI displayed by the display device 2120 to transmit a command for specifying the value of the parameter of the network camera 2102 to the network camera 2102 .

[0505] Smartphone

[0506] Fig.12 is a diagram illustrating a smartphone 2200 .

[0507] The smartphone 2200 includes a communication unit 2202 , a decoding unit 2204 , a control unit 2206 , a display unit 2208 , an image recording device 2210 , and a sensor 2212 .

[0508] The communication unit 2202 receives the encoded image data via the network 200 .

[0509] The decoding unit 2204 decodes the encoded image data received by the communication unit 2202 .

[0510] The decoding unit 2204 decodes the encoded image data by using the decoding method described above.

[0511] The control unit 2206 controls other units in the smartphone 2200 according to user operations or commands received by the communication unit 2202 .

[0512] For example, the control unit 2206 controls the display unit 2208 to display the image decoded by the decoding unit 2204 .

[0513] Although the present invention has been described with reference to the embodiments, it should be understood that the present invention is not limited to the disclosed embodiments. It will be appreciated by those skilled in the art that various changes and modifications may be made without departing from the scope of the present invention as defined in the appended claims. All features disclosed in this specification (including any appended claims, abstracts and drawings) and / or all steps of any disclosed method or process may be combined in any combination, except for at least some mutually exclusive combinations of such features and / or steps. Unless otherwise expressly stated, each feature disclosed in this specification (including any appended claims, abstracts and drawings) may be replaced by alternative features for the same, equivalent or similar purposes. Therefore, unless otherwise expressly stated, each feature disclosed is only an example of a general series of equivalent or similar features.

[0514] It should also be understood that any results of the above-described comparison, determination, evaluation, selection, execution, proceeding, or consideration (e.g., selections made during an encoding or filtering process) may be indicated in data in the bitstream (e.g., a flag or data indicating the result) or may be determined / inferred from data in the bitstream, such that the indicated or determined / inferred result may be used for processing rather than actually being compared, determined, evaluated, selected, executed, performed, or considered, for example, during a decoding process.

[0515] In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. The mere fact that different features are recited in mutually different dependent claims does not indicate that a combination of these features cannot be used to advantage.

[0516] Reference signs appearing in the claims are by way of illustration only and shall have no limiting effect on the scope of the claims.

Claims

1. A method of decoding a bitstream, the bitstream comprising video data corresponding to one or more than one slice, in, The bitstream includes a picture header including syntax elements to be used when decoding one or more slices, The method comprises: Before parsing the syntax elements related to the reference picture list from the picture header, parsing at least one syntax element related to the adaptation parameter set ID (APS ID) from the picture header, the at least one syntax element comprising a luma mapping APS ID syntax element with chroma scaling (LMCS APS ID syntax element), the LMCS APS ID syntax element indicating the APS ID related to the LMCS; and The bitstream is decoded using the syntax elements.

2. The method according to claim 1, wherein: The at least one syntax element is parsed from the picture header before parsing the SAO-related syntax element from the picture header.

3. A method of encoding video data into a bitstream, the bitstream comprising video data corresponding to one or more than one slice, in, The bitstream includes a picture header including syntax elements to be used when decoding one or more slices, The method comprises: The picture header is encoded using at least one syntax element associated with an adaptive parameter set ID, i.e., an APS ID, before a syntax element associated with a reference picture list, wherein the at least one syntax element includes a luminance mapping APS ID syntax element with chroma scaling, i.e., an LMCS APS ID syntax element, and the LMCS APS ID syntax element indicates an APS ID associated with the LMCS.

4. The method according to claim 1 or 3, wherein: In the picture header, the at least one syntax element is located after a syntax element capable of indicating that the current picture has never been used as a reference picture.

5. The method according to claim 1 or 3, wherein: In the picture header, the at least one syntax element is located after a syntax element capable of indicating that the current picture is a progressive decoding refresh picture.

6. The method according to claim 1 or 3, wherein: The at least one syntax element in the picture header is located after the picture type information in the picture header.

7. The method according to claim 1 or 3, wherein: The at least one syntax element in the picture header is located after the picture parameter set information.

8. The method according to claim 1 or 3, wherein: The at least one syntax element comprises a zoom list APS ID syntax element indicating an APS ID associated with a zoom list.

9. The method according to claim 1 or 3, wherein: The at least one syntax element comprises an ALF APS ID brightness syntax element indicating an APS ID associated with the ALF.

10. The method according to claim 9, wherein: The APS of the ALF in the bitstream contains information related to signaling the clipping value.

11. A decoder for decoding a bitstream, the bitstream comprising video data corresponding to one or more than one slice, in, The bitstream includes a picture header including syntax elements to be used when decoding one or more slices, Wherein, the decoder comprises: means for parsing at least one syntax element related to an adaptation parameter set ID (APS ID) from the picture header before parsing syntax elements related to a reference picture list from the picture header, the at least one syntax element comprising a luma mapping APS ID syntax element with chroma scaling (LMCS APS ID syntax element), the LMCS APS ID syntax element indicating an APS ID related to LMCS; and Means for decoding the bitstream using the syntax elements.

12. An encoder for encoding video data into a bitstream, the bitstream comprising video data corresponding to one or more slices, in, The bitstream includes a picture header including syntax elements to be used when decoding one or more slices, The encoder comprises: A component for encoding the picture header using at least one syntax element related to an adaptive parameter set ID, i.e., an APS ID, before a syntax element related to a reference picture list, wherein the at least one syntax element includes a luminance mapping APS ID syntax element with chroma scaling, i.e., an LMCS APS ID syntax element, wherein the LMCS APS ID syntax element indicates an APS ID related to the LMCS.

13. A computer-readable storage medium storing a program or an instruction, which, when executed by a programmable device, causes the programmable device to perform the method according to claim 1 or 3.

Images