Method, decoder, encoder, computer program product, and computer-readable storage medium for video encoding and decoding

Abstract

A method for decoding video data from a bitstream, the bitstream including video data corresponding to one or more strips. Each strip may include one or more blocks. The bitstream includes a picture header and a strip header, the picture header including syntax elements to be used when decoding one or more strips, and the strip header including syntax elements to be used. Decoding a strip includes parsing the syntax elements. In the case where a strip contains multiple blocks, if the syntax elements are parsed and indicate that the picture header is signaled in the strip header, the parsing of the syntax elements indicating the address of the strip is omitted. The bitstream is decoded using the syntax elements.

Description

Technical Field

[0001] The present invention relates to video encoding and decoding, and more particularly to advanced syntax in a bitstream. Background Art

[0002] Recently, the Joint Video Exploration Team (JVET) (a collaborative team consisting of MPEG and ITU-T Study Group 16 VCEG) has started to study a new video coding standard called Versatile Video Coding (VVC). The goal of VVC is to provide a significant improvement in compression performance over the existing HEVC standard (i.e., typically twice as much as before) and to be completed in 2020. The main target applications and services include, but are not limited to, 360-degree and high dynamic range (HDR) video. In summary, JVET has evaluated feedback from 32 organizations using formal subjective tests conducted by independent test laboratories. Some of the suggestions indicate that the compression efficiency is generally improved by 40% or more when compared to using HEVC. Specific effects have been shown on Ultra High Definition (UHD) video test materials. Therefore, for the final standard, we can expect the improvement in compression efficiency to far exceed the targeted 50%.

[0003] The JVET Exploration Model (JEM) uses all HEVC tools and has introduced several new tools. These changes require a change in the structure of the bitstream, particularly the advanced syntax that may affect the total bit rate of the bitstream. Summary of the Invention

[0004] The present invention relates to improvements in advanced syntax structures, which result in reduced complexity without any reduction in coding performance.

[0005] In a first aspect according to the present invention, there is provided a method of decoding video data from a bitstream, the bitstream including video data corresponding to one or more than one strip, wherein each strip may include one or more than one block, wherein the bitstream includes a picture header and a strip header, the picture header including syntax elements to be used when decoding one or more than one strip, the strip header including syntax elements to be used when decoding a strip, the method comprising: parsing the syntax elements, and in the case where a strip (or picture) includes a plurality of blocks, if a syntax element indicating signaling of the picture header in the strip header is parsed, omitting parsing of a syntax element indicating the address of the strip; and decoding the bitstream using the syntax elements. In another aspect according to the present invention, there is provided a method of decoding video data from a bitstream, the bitstream including video data corresponding to one or more than one strip, wherein each strip may include one or more than one block, wherein the bitstream includes a picture header and a strip header, the picture header including syntax elements to be used when decoding one or more than one strip, the strip header including syntax elements to be used when decoding a strip, the method comprising: parsing the syntax elements, and in the case where a strip or picture includes a plurality of blocks, if a syntax element indicating signaling of the picture header in the strip header is parsed, omitting parsing of a syntax element indicating the address of the strip; and decoding the bitstream using the syntax elements. In a further additional aspect according to the present invention, there is provided a method of decoding video data from a bitstream, the bitstream including video data corresponding to one or more than one strip, wherein each strip may include one or more than one block, wherein the bitstream includes a picture header and a strip header, the picture header including syntax elements to be used when decoding one or more than one strip, the strip header including syntax elements to be used when decoding a strip, the bitstream being constrained such that in the case where the bitstream includes a syntax element having a value indicating that a strip or picture includes a plurality of blocks and the bitstream includes a syntax element indicating signaling of the picture header in the strip header, the bitstream further includes a syntax element indicating that a syntax element indicating the address of the strip will not be parsed, the method comprising decoding the bitstream using the syntax elements.

[0006] Thus, when the picture header is in the slice header, the strip address is not parsed, which reduces the bit rate, especially for low-latency and low-bit-rate applications. In addition, when signaling the picture in the strip header, the parsing complexity can be reduced.

[0007] In an embodiment, the omission is to be performed only when a raster scan strip mode is to be used for decoding a strip. This reduces the parsing complexity but still allows some bit rate reduction.

[0008] The omission may also include omitting the parsing of the syntactic element indicating the number of blocks in the strip. Thus, a further reduction in the bit rate can be achieved.

[0009] In a second aspect, there is provided a method for decoding video data from a bitstream, the bitstream including video data corresponding to one or more strips, where each strip may include one or more blocks, where the bitstream includes a picture header and a strip header, the picture header including syntactic elements to be used when decoding one or more strips, the strip header including syntactic elements to be used when decoding a strip, and the decoding including: parsing one or more syntactic elements, and in the case where the strip (or picture) includes a plurality of blocks, if the syntactic element indicating the picture header signaled in the strip header is parsed, omitting the parsing of the syntactic element indicating the number of blocks in the strip; and decoding the bitstream using the syntactic elements. In another aspect, there is provided a method for decoding video data from a bitstream, the bitstream including video data corresponding to one or more strips, where each strip may include one or more blocks, where the bitstream includes a picture header and a strip header, the picture header including syntactic elements to be used when decoding one or more strips, the strip header including syntactic elements to be used when decoding a strip, and the decoding including: parsing one or more syntactic elements, and in the case where the strip or picture includes a plurality of blocks, if the syntactic element indicating the picture header signaled in the strip header is parsed, omitting the parsing of the syntactic element indicating the number of blocks in the strip; and decoding the bitstream using the syntactic elements. In another aspect of the present invention, there is provided a method for decoding video data from a bitstream, the bitstream including video data corresponding to one or more strips, where each strip may include one or more blocks, where the bitstream includes a picture header and a strip header, the picture header including syntactic elements to be used when decoding one or more strips, the strip header including syntactic elements to be used when decoding a strip, the bitstream being constrained such that in the case where the bitstream includes a syntactic element having a value indicating that the strip or picture includes a plurality of blocks and the bitstream includes a syntactic element indicating the picture header signaled in the strip header, the bitstream further includes a syntactic element indicating that the syntactic element indicating the plurality of blocks in the strip will not be parsed, the method including decoding the bitstream using the syntactic elements.

[0010] Accordingly, the bit rate can be reduced, which is particularly beneficial for low-latency and low-bit-rate applications that do not require sending multiple blocks.

[0011] The omission can be performed only when the raster scan strip mode is to be used for decoding the strip. This reduces the parsing complexity but still allows some bit rate reduction.

[0012] The method may further include: parsing a syntax element indicating the number of blocks in the picture, and determining the number of blocks in the strip based on the number of blocks in the picture indicated by the parsed syntax element. This is advantageous because it allows the number of blocks in the strip to be easily predicted without further signaling in the strip header when signaling the picture header.

[0013] The omission may further include omitting the parsing of a syntax element indicating the address of the strip. Thus, the bit rate can be further reduced.

[0014] In a third aspect of the present invention, a method for decoding video data from a bitstream is provided, the bitstream including video data corresponding to one or more than one strip, wherein each strip may include one or more than one block, wherein the bitstream includes a picture header and a strip header, the picture header includes syntax elements to be used when decoding one or more than one strip, the strip header includes syntax elements to be used when decoding the strip, and the decoding includes: parsing one or more than one syntax element, and in the case where the strip (or picture) includes a plurality of blocks, if the number of blocks in the strip is equal to the number of blocks in the picture, omitting the parsing of a syntax element indicating the strip address; and decoding the bitstream using the syntax element. This utilizes the insight that if the number of blocks in the strip is equal to the number of blocks in the picture, it is ensured that the current picture contains only one strip. Thus, by omitting the strip address, the bit rate can be improved and the parsing and / or encoding complexity can be reduced.

[0015] The omission can be performed only when the raster scan strip mode is to be used for decoding the strip. Thus, the complexity can be reduced while still providing some bit rate reduction.

[0016] The decoding may further include: parsing a syntax element indicating the number of blocks in the strip in the strip; and parsing a syntax element indicating the number of blocks in the picture in the picture parameter set, wherein the omission of the parsing of a syntax element indicating the strip address is based on the parsed syntax elements.

[0017] The decoding may further include: parsing a syntax element indicating the number of blocks in the strip before one or more than one syntax element for signaling the strip address.

[0018] Decoding may further include: parsing, in a slice, a syntax element indicating whether a picture header is signaled in the slice header, and if the parsed syntax element indicates that the picture header is signaled in the slice header, determining (inferring) that the number of blocks in the slice is equal to the number of blocks in the picture.

[0019] In a fourth aspect, a method of decoding video data from a bitstream is provided, the bitstream including video data corresponding to one or more slices, where each slice may include one or more blocks, where the bitstream includes a picture header and a slice header, the picture header including syntax elements to be used when decoding one or more slices, the slice header including syntax elements to be used when decoding a slice, and the decoding including: parsing one or more syntax elements, and when the syntax element indicates that raster scan decoding mode is enabled for a slice, decoding at least one of a slice address and the number of blocks in the slice from the one or more syntax elements, where decoding at least one of a slice address and the number of blocks in the slice from the one or more syntax elements when raster scan decoding mode is enabled for a slice does not depend on the number of blocks in the picture; and decoding the bitstream using the syntax elements. Accordingly, the parsing complexity of the slice header can be reduced.

[0020] In a fifth aspect according to the present invention, a method including the methods of the first and second aspects is provided.

[0021] In a sixth aspect according to the present invention, a method including the methods of the first, second, and third aspects is provided.

[0022] According to a seventh aspect of the present invention, there is provided a method of encoding video data into a bitstream, the bitstream including video data corresponding to one or more than one strip, wherein each strip may include one or more than one block, wherein the bitstream includes a picture header and a strip header, the picture header including syntax elements to be used when decoding one or more than one strip, the strip header including syntax elements to be used when encoding a strip, and the encoding including: determining one or more than one syntax element for encoding the video data, and in the case where a strip (or picture) includes a plurality of blocks, if the syntax element indicates signaling the picture header in the strip header, omitting encoding of the syntax element indicating the address of the strip; and encoding the video data using the syntax element. According to an additional aspect of the present invention, there is provided a method of encoding video data into a bitstream, the bitstream including the video data corresponding to one or more than one strip, wherein each strip may include one or more than one block, wherein the bitstream includes a picture header and a strip header, the picture header including syntax elements to be used when decoding one or more than one strip, the strip header including syntax elements to be used when encoding a strip, and the encoding including: determining one or more than one syntax element for encoding the video data, and in the case where a strip or picture includes a plurality of blocks, if the syntax element indicates signaling the picture header in the strip header, omitting encoding of the syntax element indicating the address of the strip; and encoding the video data using the syntax element. According to an additional supplementary aspect of the present invention, there is provided a method of encoding video data into a bitstream, the bitstream including video data corresponding to one or more than one strip, wherein each strip may include one or more than one block, wherein the bitstream includes a picture header and a strip header, the picture header including syntax elements to be used when decoding one or more than one strip, the strip header including syntax elements to be used when encoding a strip, the bitstream being constrained such that in the case where the bitstream includes a syntax element having a value indicating that a strip or picture includes a plurality of blocks and the bitstream includes a syntax element indicating signaling the picture header in the strip header, the bitstream further includes a syntax element indicating that the syntax element indicating the address of the strip will not be parsed; the method including encoding the video data using the syntax element.

[0023] In one or more than one embodiment, the omission is performed only when a raster scan strip mode is used for encoding the strip.

[0024] The omission may further include omitting encoding of the syntax element indicating the number of blocks in the strip.

[0025] According to an eighth aspect of the present invention, there is provided a method of encoding video data into a bitstream, the bitstream including video data corresponding to one or more than one strip, wherein each strip may include one or more than one block, wherein the bitstream includes a picture header and a strip header, the picture header including syntax elements to be used when decoding one or more than one strip, the strip header including syntax elements to be used when decoding a strip, and the encoding includes: determining one or more than one syntax element for encoding the video data, and in the case where a strip includes a plurality of blocks, if a syntax element indicating the picture header to be signaled in the strip header is determined for encoding, omitting encoding of a syntax element indicating the number of blocks in the strip; and encoding the video data using the syntax elements. According to another additional aspect of the present invention, there is provided a method of encoding video data into a bitstream, the bitstream including video data corresponding to one or more than one strip, wherein each strip may include one or more than one block, wherein the bitstream includes a picture header and a strip header, the picture header including syntax elements to be used when decoding one or more than one strip, the strip header including syntax elements to be used when decoding a strip, and the encoding includes: determining one or more than one syntax element for encoding the video data, and in the case where a strip or a picture includes a plurality of blocks, if a syntax element indicating the picture header to be signaled in the strip header is determined for encoding, omitting encoding of a syntax element indicating the number of blocks in the strip; and encoding the video data using the syntax elements. According to another complementary aspect of the present invention, there is provided a method of encoding video data into a bitstream, the bitstream including video data corresponding to one or more than one strip, wherein each strip may include one or more than one block, wherein the bitstream includes a picture header and a strip header, the picture header including syntax elements to be used when decoding one or more than one strip, the strip header including syntax elements to be used when decoding a strip, the bitstream being constrained such that in the case where the bitstream includes a syntax element having a value indicating that a strip or a picture includes a plurality of blocks and the bitstream includes a syntax element indicating the picture header to be signaled in the strip header that is determined for encoding, the bitstream further includes a syntax element indicating that a syntax element indicating the number of blocks in the strip will not be parsed, the method including encoding the video data using the syntax elements.

[0026] In an embodiment, the omission is performed only when a raster scan strip mode is to be used for encoding a strip.

[0027] The encoding may also include encoding a syntax element indicating the number of blocks in the picture, wherein the number of blocks in the strip is based on the number of blocks in the picture indicated by the parsed syntax element.

[0028] The omission may also include omitting the encoding of a syntax element indicating the address of the strip.

[0029] According to a ninth aspect of the present invention, there is provided a method of encoding video data into a bitstream, the bitstream including video data corresponding to one or more strips, wherein each strip may include one or more blocks, wherein the bitstream includes a picture header and a strip header, the picture header including syntax elements to be used when decoding one or more strips, the strip header including syntax elements to be used when decoding a strip, and the encoding including: determining one or more syntax elements, and in the case where the strip (or picture) includes a plurality of blocks, if the number of blocks in the strip is equal to the number of blocks in the picture, omitting the encoding of a syntax element indicating the strip address; and encoding the video data using the syntax elements.

[0030] In one or more embodiments, the omission is performed only when a raster scan strip mode is to be used for decoding the strip.

[0031] The encoding may also include encoding in the strip a syntax element indicating the number of blocks in the strip; and encoding in the picture parameter set a syntax element indicating the number of blocks in the picture, wherein whether to omit or not omit the encoding of a syntax element indicating the strip address is based on the value of the encoded syntax element.

[0032] The encoding may also include: encoding in the strip a syntax element indicating the number of blocks in the strip before one or more syntax elements for signaling the strip address.

[0033] The encoding may also include encoding in the strip a syntax element indicating whether to signal the picture header in the strip header, and if the syntax element to be encoded indicates signaling the picture header in the strip header, determining that the number of blocks in the strip is equal to the number of blocks in the picture.

[0034] According to a tenth aspect of the present invention, there is provided a method of encoding video data into a bitstream, the bitstream including video data corresponding to one or more strips, wherein each strip may include one or more blocks, wherein the bitstream includes a picture header and a strip header, the picture header including syntax elements to be used when decoding one or more strips, the strip header including syntax elements to be used when decoding a strip, and the encoding includes: determining one or more syntax elements for encoding the video data, and when the syntax elements determined for encoding indicate that the raster scan decoding mode is enabled for a strip, encoding a syntax element indicating at least one of the strip address and the number of blocks in the strip, wherein when the raster scan decoding mode is enabled for a strip, decoding at least one of the strip address and the number of blocks in the strip from one or more syntax elements does not depend on the number of blocks in the picture; and encoding the bitstream using the syntax elements.

[0035] According to an eleventh aspect of the present invention, there is provided a method including the methods of the seventh and eighth aspects.

[0036] According to a twelfth aspect of the present invention, there is provided a method including the methods of the seventh, eighth, and ninth aspects.

[0037] According to a thirteenth aspect of the present invention, there is provided a decoder for decoding video data from a bitstream, the decoder being configured to perform the method of any one of the first to sixth aspects.

[0038] According to a fourteenth aspect of the present invention, there is provided an encoder for encoding video data into a bitstream, the encoder being configured to perform the method of any one of the seventh to twelfth aspects.

[0039] According to a fifteenth aspect of the present invention, there is provided a computer program which, when executed, causes the method of any one of the first to twelfth aspects to be performed. The program may be provided separately, or may be carried on, by, or in a carrier medium. The carrier medium may be non-transitory, such as a storage medium, particularly a computer-readable storage medium. The carrier medium may also be transitory, such as a signal or other transmission medium. The signal may be transmitted via any suitable network (including the Internet).

[0040] Any feature in one aspect of the present invention may be applied in any suitable combination to other aspects of the present invention. In particular, method aspects may be applied to device aspects and vice versa.

[0041] In addition, features implemented in hardware may be implemented in software, and vice versa. Any reference to software and hardware features herein shall be construed accordingly.

[0042] Any device feature described herein may also be provided as a method feature, and vice versa. As used herein, a component-plus-function feature may be alternatively expressed in terms of its corresponding structure (such as a suitably programmed processor and associated memory, etc.).

[0043] It should also be understood that specific combinations of the various features described and defined in any aspect of the present invention may be implemented, provided, and / or used independently. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Reference will now be made, by way of example, to the accompanying drawings, in which:

[0045] Figure 1 is a diagram for illustrating the coding structures used in HEVC and VVC;

[0046] Figure 2 is a block diagram schematically showing a data communication system in which one or more embodiments of the present invention may be implemented;

[0047] Figure 3 is a block diagram showing the components of a processing device in which one or more embodiments of the present invention may be implemented;

[0048] Figure 4 is a flowchart showing the steps of a coding method according to an embodiment of the present invention;

[0049] Figure 5 is a flowchart showing the steps of a decoding method according to an embodiment of the present invention;

[0050] Figure 6 shows the structure of a bitstream in an exemplary coding system VVC;

[0051] Figure 7 shows another structure of a bitstream in an exemplary coding system VVC;

[0052] Figure 8 shows Luma Modelling Chroma Scaling (LMCS);

[0053] Figure 9 shows the sub-tools of LMCS;

[0054] Figure 10 is a diagram of the raster scan strip mode and the rectangular strip mode of the current VVC draft standard;

[0055] Figure 11A diagram showing a system including an encoder or decoder and a communication network according to an embodiment of the present invention;

[0056] Figure 12 A schematic block diagram of a computing device for implementing one or more embodiments of the present invention;

[0057] Figure 13 A diagram showing a network camera system; and

[0058] Figure 14 A diagram showing a smart phone. Detailed Description

[0059] Figure 1 Relates to a coding structure used in the High Efficiency Video Coding (HEVC) video standard. A video sequence 1 consists of a series of digital images i. Each such digital image is represented by one or more matrices. Matrix coefficients represent pixels.

[0060] The images 2 of the sequence can be segmented into slices 3. In some cases, a slice can constitute the entire image. These slices are segmented into non - overlapping Coding Tree Units (CTUs). A Coding Tree Unit (CTU) is the basic processing unit of the High Efficiency Video Coding (HEVC) video standard and conceptually corresponds in structure to the macroblock units used in several previous video standards. CTU is sometimes also referred to as the Largest Coding Unit (LCU). A CTU has luminance and chrominance component parts, and each component part is called a Coding Tree Block (CTB). These different color components are not shown in Figure 1 here.

[0061] A CTU is typically sized 64 pixels × 64 pixels. Quadtree decomposition can be used to iteratively divide each CTU into smaller variable - sized Coding Units (CUs) 5.

[0062] A coding unit is the basic coding element and consists of two types of sub - units called Prediction Units (PUs) and Transform Units (TUs). The maximum size of a PU or TU is equal to the CU size. A prediction unit corresponds to a partition of the CU for pixel value prediction. Various different partitions of the CU into PUs are possible, as shown in 6, including a partition into 4 square PUs and two different partitions into 2 rectangular PUs. A transform unit is the basic unit for performing spatial transformation using DCT. A CU can be partitioned into TUs based on a quadtree representation 7.

[0063] Each strip is embedded in a Network Abstraction Layer (NAL) unit. Additionally, the coding parameters of a video sequence are stored in dedicated NAL units called parameter sets. In HEVC and H.264 / AVC, two types of parameter set NAL units are employed: First, the Sequence Parameter Set (SPS) NAL unit, which collects all the parameters that remain unchanged throughout the video sequence. Typically, it deals with the coding profile, the size of the video frames, and other parameters. Second, the Picture Parameter Set (PPS) NAL unit, which includes parameters that can change from one image (or frame) of the sequence to another. HEVC also includes a Video Parameter Set (VPS) NAL unit, which contains parameters that describe the overall structure of the bitstream. The VPS is a new type of parameter set defined in HEVC and applies to all layers of the bitstream. A layer can contain multiple temporal sub-layers, and all version 1 bitstreams are limited to a single layer. HEVC has certain hierarchical extensions for scalability and multi-view, and these extensions will allow multiple layers with a backward-compatible version 1 base layer.

[0064] In the current definition of Versatile Video Coding (VVC), there are three high-level possibilities for partitioning a picture: sub-picture, strip, and tile. Each has its own characteristics and usefulness. Partitioning into sub-pictures is for spatial extraction and / or merging of regions of the video. Partitioning into strips is based on a concept similar to that of previous standards and corresponds to sub-packets for video transmission (even though it can be used for other applications). Partitioning into tiles is conceptually an encoder parallelization tool as it splits the picture into independent coding regions of (almost) the same size of the picture. But this tool can also be used for other applications.

[0065] Since these three high-level available ways of picture partitioning can be used together, there are several modes for their use. As defined in the current draft specification of VVC, two modes for defining strips are defined. For the raster scan strip mode, a strip contains a complete sequence of tiles in the tile raster scan of the picture. This mode in the current VVC specification is shown in Figure 10 (a). As shown in the figure, the picture contains 18 by 12 luma CTUs shown partitioned into 12 strips and 3 raster scan strips.

[0066] For the second (rectangular strip mode), a strip contains several complete tiles from a rectangular region of the picture. This mode in the current VVC specification is shown in Figure 10 (b). In this example, the picture has 18 by 12 luma TUs shown partitioned into 24 tiles and 9 rectangular strips.

[0067] Figure 2An example of a data communication system that can implement one or more embodiments of the present invention is shown. The data communication system includes a transmitting device (in this case, server 201), which is operable to transmit data packets of a data stream via a data communication network 200 to a receiving device (in this case, client terminal 202). The data communication network 200 can be a wide area network (WAN) or a local area network (LAN). Such a network can be, for example, a wireless network (Wifi / 802.11a or b or g), an Ethernet network, an Internet network, or a hybrid network composed of several different networks. In a particular embodiment of the present invention, the data communication system can be a digital television broadcast system, in which the server 201 sends the same data content to multiple clients.

[0068] The data stream 204 provided by the server 201 can consist of multimedia data representing video and audio data. In some embodiments of the present invention, the audio and video data streams can be captured by the server 201 using a microphone and a camera, respectively. In some embodiments, the data stream can be stored on the server 201 or received by the server 201 from other data providers, or generated at the server 201. The server 201 is provided with an encoder for encoding the video and audio streams, particularly to provide a compressed bitstream for transmission, which is a more compact representation of the data presented as the input to the encoder.

[0069] To obtain a better ratio of the quality of the transmitted data to the amount of the transmitted data, the video data can be compressed, for example, according to the HEVC format or the H.264 / AVC format.

[0070] The client 202 receives the transmitted bitstream and decodes the reconstructed bitstream to reproduce the video image on a display device and reproduce the audio data using a speaker.

[0071] Although a streaming scenario is considered in the example of Figure 2 , it will be appreciated that in some embodiments of the present invention, data communication between the encoder and the decoder can be performed using, for example, a medium storage device (such as an optical disc, etc.).

[0072] In one or more embodiments of the present invention, the video image is transmitted together with data representing a compensation offset for the reconstructed pixels to be applied to the image to provide filtered pixels in the final image.

[0073] Figure 3 A processing device 300 configured to implement at least one embodiment of the present invention is schematically illustrated. The processing device 300 can be a device such as a microcomputer, a workstation, or a lightweight portable device.

[0074] The device 300 includes a communication bus 313, which is connected to:

[0075] - The central processing unit 311 represented as a CPU, such as a microprocessor, etc.;

[0076] - The read-only memory 306 represented as a ROM, which is used to store the computer program for implementing the present invention;

[0077] - The random access memory 312 represented as a RAM for storing the executable code of the method of the embodiments of the present invention, and registers suitable for recording variables and parameters required for implementing the method for encoding a digital image sequence and / or the method for decoding a bitstream according to the embodiments of the present invention; and

[0078] - The communication interface 302 connected to the communication network 303, through which digital data to be processed is transmitted or received.

[0079] Optionally, the device 300 may further include the following components:

[0080] - A data storage component 304 such as a hard disk, etc., which is used to store the computer program for implementing the method of one or more embodiments of the present invention and the data used or generated during the implementation of one or more embodiments of the present invention;

[0081] - A disk drive 305 for the disk 306, which is suitable for reading data from or writing data to the disk 306;

[0082] - A screen 309, which is used to display data by means of a keyboard 310 or any other indicating device and / or serves as a graphical interface for interacting with the user.

[0083] The device 300 can be connected to various peripheral devices such as a digital camera 320 or a microphone 308, etc., each of which is connected to an input / output card (not shown) to provide multimedia data to the device 300.

[0084] The communication bus provides communication and interoperability between the various elements included in or connected to the device 300. The representation of the bus is not restrictive, and in particular, the central processing unit can operably communicate instructions directly or by means of other elements of the device 300 to any element of the device 300.

[0085] The disc 306 can be replaced by any information medium such as a rewritable or non-rewritable compact disc (CD-ROM), ZIP disc, or memory card, and generally, by an information storage component readable by a microcomputer or microprocessor. The disc 306 may or may not be integrated into the device, may be removable, and is adapted to store one or more programs whose execution enables the implementation of a method for encoding a digital image sequence and / or a method for decoding a bitstream according to the present invention.

[0086] The executable code can be stored in the read-only memory 306, on the hard disk 304, or on a removable digital medium (such as, for example, the disc 306 as described above). According to a variant, the executable code of the program can be received via the interface 302 by means of the communication network 303 for storage in one of the storage components of the device 300 (such as the hard disk 304, etc.) before execution.

[0087] The central processing unit 311 is adapted to control and direct the execution of instructions or a part of the software code of one or more programs according to the present invention, the execution of instructions stored in one of the above-mentioned storage components. When powered on, one or more programs stored in non-volatile memory (for example, on the hard disk 304 or in the read-only memory 306) are transferred to the random access memory 312 (which then contains the executable code of one or more programs) and registers for storing variables and parameters necessary for implementing the present invention.

[0088] In this embodiment, the device is a programmable device that uses software to implement the present invention. However, alternatively, the present invention can be implemented in hardware (for example, in the form of an application-specific integrated circuit or ASIC).

[0089] Figure 4 Block diagram illustrating an encoder according to at least one embodiment of the present invention. The encoder is represented by connected modules, each module being adapted to implement at least one corresponding step of a method for encoding an image in an image sequence according to at least one embodiment of one or more embodiments of the present invention, for example, in the form of programming instructions executed by the CPU 311 of the device 300.

[0090] The encoder 400 receives the original sequence 401 of digital images i 0 to i n as input. Each digital image is represented by a set of samples (called pixels).

[0091] After implementing the encoding process, the encoder 400 outputs a bitstream 410. The bitstream 410 includes a plurality of coding units or strips, each strip including a strip header for transmitting the coded values of the coding parameters used for strip coding, and a strip body including coded video data.

[0092] Module 402 divides the input digital image i 0 to i n 401 into pixel blocks. The blocks correspond to parts of the image and can have variable sizes (e.g., 4×4, 8×8, 16×16, 32×32, 64×64, 128×128 pixels, and several rectangular block sizes can also be considered). An encoding mode is selected for each input block. Two families of encoding modes are provided: encoding modes based on spatial prediction encoding (intra-frame prediction) and encoding modes based on temporal prediction (inter-frame encoding, merge, skip). The possible encoding modes are tested.

[0093] Module 403 implements intra-frame prediction processing, where the block to be encoded is predicted by a predictor calculated based on adjacent pixels of the given block to be encoded. If intra-frame encoding is selected, the selected intra-frame predictor and an indication of the difference between the given block and its predictor are encoded to provide a residual.

[0094] Temporal prediction is implemented by motion estimation module 404 and motion compensation module 405. First, a reference image from the reference image set 416 is selected, and a part of the reference image (also referred to as a reference region or image part), which is the region closest to the given block to be encoded, is selected by motion estimation module 404. Then motion compensation module 405 uses the selected region to predict the block to be encoded. The difference between the selected reference region and the given block (also referred to as the residual block) is calculated by motion compensation module 405. The selected reference region is indicated by a motion vector.

[0095] Thus, in both cases (spatial and temporal prediction), the residual is calculated by subtracting the prediction from the original block.

[0096] In the intra-frame prediction implemented by module 403, the prediction direction is encoded. In temporal prediction, at least one motion vector is encoded. In the inter-frame prediction implemented by modules 404, 405, 416, 418, 417, at least one motion vector or data for identifying such a motion vector is encoded for temporal prediction.

[0097] If inter-frame prediction is selected, the information related to the motion vector and the residual block is encoded. To further reduce the bit rate, assuming that the motion is homogeneous, the motion vector is encoded by the difference relative to the motion vector predictor. The motion vector predictor in the set of motion information predictors is obtained by motion vector prediction and encoding module 417 from the motion vector field 418.

[0098] The encoder 400 further includes a selection module 406 that is configured to select an encoding mode by applying an encoding cost criterion such as a rate-distortion criterion or the like. To further reduce redundancy, a transform (such as DCT or the like) is applied to the residual block by a transform module 407, and then the obtained transformed data is quantized by a quantization module 408 and entropy encoded by an entropy encoding module 409. Finally, the encoded residual block of the currently encoded block is inserted into the bitstream 410.

[0099] The encoder 400 also decodes the encoded image to generate a reference image for motion estimation of subsequent images. This enables the encoder and decoder that receive the bitstream to have the same reference frames. An inverse quantization module 411 performs inverse quantization of the quantized data, followed by an inverse transform by an inverse transform module 412. The inverse intra prediction module 413 uses the prediction information to determine which predictor to use for a given block, and the inverse motion compensation module 414 actually adds the residual obtained by module 412 to the reference region obtained from the reference image set 416.

[0100] Then, post-filtering is applied by module 415 to filter the reconstructed pixel frame. In an embodiment of the present invention, an SAO loop filter is used, where a compensation offset is added to the pixel values of the reconstructed pixels of the reconstructed image.

[0101] Figure 5 A block diagram of a decoder 60 according to an embodiment of the present invention is shown. The decoder 60 can be used to receive data from an encoder. The decoder is represented by the connected modules, and each module is adapted to implement the corresponding steps of the method implemented by the decoder 60, for example, in the form of programming instructions to be executed by the CPU 311 of the device 300.

[0102] The decoder 60 receives a bitstream 600 including encoded units, and each encoded unit is composed of a header containing information related to the encoded parameters and a body containing the encoded video data. The structure of the bitstream in VVC is described in more detail below with reference to Figure 6 As illustrated with respect to Figure 4 For a given block, the encoded video data is entropy encoded on a predetermined number of bits, and the index of the motion vector predictor is encoded. The received encoded video data is entropy decoded by module 62. Then the residual data is dequantized by module 63, and then an inverse transform is applied by module 64 to obtain pixel values.

[0103] The mode data for indicating the encoding mode is also entropy decoded, and based on this mode, the encoded blocks of the image data are decoded as an intra type or an inter type.

[0104] In the case of the intra mode, the intra inverse prediction module 65 determines the intra predictor based on the intra prediction mode specified in the bitstream.

[0105] If the mode is inter-frame, motion prediction information is extracted from the bitstream to find the reference regions used by the encoder. The motion prediction information consists of a reference frame index and a motion vector residual. The motion vector predictor is added to the motion vector residual to obtain a motion vector by the motion vector decoding module 70.

[0106] The motion vector decoding module 70 applies motion vector decoding to each current block encoded by motion prediction. Once the index of the motion vector predictor for the current block has been obtained, the actual value of the motion vector associated with the current block can be decoded, and this actual value is used to apply inverse motion compensation by module 66. The reference image portion indicated by the decoded motion vector is extracted from the reference image 68 to apply inverse motion compensation 66. The motion vector field data 71 is updated with the decoded motion vector for inverse prediction of subsequent decoded motion vectors.

[0107] Finally, the decoded block is obtained. Post-filtering is applied by the post-filtering module 67. The decoder 60 finally provides the decoded video signal 69.

[0108] Figure 6 The organization of the bitstream in an exemplary VVC encoding system as described in JVET_Q2001-vD is shown.

[0109] The bitstream 61 according to the VVC encoding system consists of a sequence of syntax elements and encoded data. The syntax elements and the encoded data are placed into network abstraction layer (NAL) units 601 - 608. There are different NAL unit types. The network abstraction layer provides the ability to encapsulate the bitstream into different protocols such as RTP / IP (representing Real-Time Protocol / Internet Protocol), ISO base media file format, etc. The network abstraction layer also provides a framework for packet loss resilience.

[0110] The NAL units are divided into video coding layer (VCL) NAL units and non-VCL NAL units. The VCL NAL units contain the actual encoded video data. The non-VCL NAL units contain additional information. This additional information can be parameters required to decode the encoded video data or supplementary data that can enhance the usability of the decoded video data. The NAL unit 606 corresponds to a slice and constitutes the VCL NAL unit of the bitstream.

[0111] The different NAL units 601-605 correspond to different parameter sets, and these NAL units are non-VCL NAL units. The Decoder Parameter Set (DPS) NAL unit 301 contains parameters that are constant for a given decoding process. The Video Parameter Set (VPS) NAL unit 602 contains parameters defined for the entire video and thus the entire bitstream. The DPS NAL unit may define parameters that are more static than the parameters in the VPS. In other words, the parameters of the DPS change less frequently than the parameters of the VPS.

[0112] The Sequence Parameter Set (SPS) NAL unit 603 contains parameters defined for a video sequence. In particular, the SPS NAL unit may define the sub-picture layout of the video sequence and associated parameters. The parameters associated with each sub-picture specify the coding constraints applied to the sub-picture. In particular, it includes a flag indicating that the temporal prediction between sub-pictures is restricted to data from the same sub-picture. Another flag may enable or disable the loop filter across sub-picture boundaries.

[0113] The Picture Parameter Set (PPS) NAL unit 604, the PPS contains parameters defined for a picture or group of pictures. The Adaptive Parameter Set (APS) NAL unit 605 contains parameters for the loop filter, which is typically an Adaptive Loop Filter (ALF) or a shaper model (or a Luminance Mapping with Chroma Scaling (LMCS) model) or a scaling matrix used at the slice level.

[0114] The syntax of the PPS as proposed in the current version of VVC includes syntax elements that specify the size of the picture in terms of luminance samples and the partitioning of each picture into tiles and slices.

[0115] The PPS contains syntax elements that enable the determination of the slice positions in a frame. Since the sub-pictures form rectangular regions in the frame, the set of slices, tile parts, or tiles belonging to a sub-picture can be determined based on the parameter set NAL units. The PPS, like the APS, has an ID mechanism to limit the amount of transmission of the same PPS.

[0116] The main difference between the PPS and the picture header lies in its transmission. Compared with the PH that is systematically sent for each image, the PPS is typically sent for a group of pictures. Therefore, compared with the PH, the PPS contains parameters that can be constant for several pictures.

[0117] The bitstream may also contain Supplemental Enhancement Information (SEI) NAL units ( Figure 6(not shown in the figure). The occurrence period of these parameter sets in the bitstream is variable. The VPS defined for the entire bitstream can occur only once in the bitstream. In contrast, the APS defined for a slice can occur once for each slice in each picture. In fact, different slices can depend on the same APS, and thus there are usually fewer APSs than the slices in each picture. In particular, the APS is defined in the picture header. However, the ALF APS can be refined in the slice header.

[0118] The Access Unit Delimiter (AUD) NAL unit 607 separates two access units. An access unit is a set of NAL units, which may include one or more than one encoded picture with the same decoding timestamp. This optional NAL unit contains only one syntax element in the current VVC specification: pic_type, which indicates the slice_type value for all slices of the encoded pictures in the AU. If pic_type is set to equal 0, the AU contains only Intra slices. If it equals 1, it contains P and I slices. If it equals 2, it contains B, P, or Intra slices.

[0119] This NAL unit contains only one syntax element pic-type.

[0120] Table 1 Syntax AUD

[0121]

[0122] In JVET-Q2001-vD, pic-type is defined as follows:

[0123] "pic_type indicates that the slice_type value of all slices of the encoded pictures in the AU containing the AUD NAL unit is a member of the set listed in Table 2 for a given pic_type value. The value of pic_type in the bitstream of this version of the specification should be equal to 0, 1, or 2. Other values of pic_type are reserved for future use by ITUT|ISO / IEC. Decoders compliant with this version of the specification will ignore the reserved values of pic_type."

[0124] rbsp_trailing_bits() is a function that adds bits to align with the end of a byte. Therefore, after this function, the amount of the parsed bitstream is an integer number of bytes.

[0125] Table 2 Interpretation of pic_type

[0126] pic_type Possible slice_type values in AU 0 I 1 P, I 2 B, P, I

[0127] The PH NAL unit 608 is a picture header NAL unit that groups common parameters for a set of slices of an encoded picture. A picture may refer to one or more APSs to indicate AFL parameters, shaper models, and scaling matrices used by the slices of the picture.

[0128] The VCL NAL units 606 each contain a slice. A slice may correspond to an entire picture or sub-picture, a single block or multiple blocks or a fragment of a block. For example, Figure 6 a slice contains a number of blocks 620. A slice consists of a slice header 610 and a raw byte sequence payload RBSP 611, and the RBSP 611 contains encoded pixel data encoded as encoded blocks 640.

[0129] The syntax of the PPS as proposed in the current version of VVC includes syntax elements that specify the size of the picture in terms of luminance samples and the partitioning of each picture in terms of blocks and slices.

[0130] The PPS contains syntax elements that enable the determination of the slice positions in a frame. Since sub-pictures form rectangular regions in a frame, the set of slices, block parts, or blocks belonging to a sub-picture can be determined from the parameter set NAL units.

[0131] NAL unit slice

[0132] The NAL unit slice layer contains a slice header and slice data, as shown in Table 3.

[0133] Table 3 Slice layer syntax

[0134]

[0135] APS

[0136] The Adaptive Parameter Set (APS) NAL unit 605 is defined in Table 4 showing the syntax elements.

[0137] As depicted in Table 4, there are 3 possible types of APS given by the aps_params_type syntax element:

[0138] · ALF_AP: for ALF parameters

[0139] · LMCS_APS: for LMCS parameters

[0140] · SCALLING_APS: for scaling list related parameters

[0141] Table 4 Adaptive Parameter Set syntax

[0142]

[0143] The following discusses these three types of APS parameters in turn.

[0144] ALF APS

[0145] The ALF parameters are described in the adaptive loop filter data syntax elements (Table 5). First, four flags are dedicated to specifying whether the ALF filter is sent for luminance and / or for chrominance and whether CC-ALF (cross-component adaptive loop filtering) is enabled for the Cb and Cr components. If the luminance filter flag is enabled, another flag is decoded to know whether the cropping value (alf_luma_clip_flag) is signaled. Then, the number of signaled filters is decoded using the alf_luma_num_filters_signalled_minus1 syntax element. If necessary, the syntax element representing the ALF coefficient increment "alf_luma_coeff_delta_idx" is decoded for each enabled filter. Then, the absolute value and sign of each coefficient of each filter are decoded.

[0146] If the alf_luma_clip_flag is enabled, the cropping index of each coefficient of each enabled filter is decoded.

[0147] In the same way, the ALF chrominance coefficients are decoded when needed.

[0148] If CC-ALF is enabled for Cr or Cb, the number of filters is decoded (alf_cc_cb filters_signalled minusl or alf_cc_cr filters_signalled_minus1) and the relevant coefficients are decoded (alf_cc_cb_mapped_coeff_abs and alf_cc_cb_coeff_sign or, respectively, alf_cc_cr_mapped_coeff_abs and alf_cc_cr_coeff_sign).

[0149] Table 5 Adaptive Loop Filter Data Syntax

[0150]

[0151]

[0152]

[0153] LMCS syntax elements for both luminance mapping and chrominance scaling

[0154] Table 6 below gives all the LMCS syntax elements (LMCS_APS) encoded in the Adaptive Parameter Set (APS) syntax structure when the aps_params_type parameter is set to 1. Up to four LMCS APSs can be used in an encoded video sequence; however, for a given picture, only a single LMCS APS can be used.

[0155] These parameters are used to construct the forward and inverse mapping functions for luminance and the scaling function for chrominance.

[0156] Table 6 Luminance mapping with chroma scaling data syntax

[0157]

[0158]

[0159] Scaling list APS

[0160] The scaling list provides the possibility to update the quantization matrix used for quantization. In VVC, this scaling matrix is signaled in the APS as described in the scaling list data syntax element (Table 7 Scaling list data syntax). The first syntax element specifies whether the scaling matrix is used for the LFNST (Low Frequency Non-Separable Transform) tool based on the flag scaling_matrix_for_lfnst_disabled_flag. If the scaling list is used for the chroma component (scaling_list_chroma_present_flag), then the second one is specified. Then, the syntax elements required to decode and construct the scaling matrix (scaling_list_copy_mode_flag, scaling_list_pred_mode_flag, scaling_list_pred_id_delta, scaling_list_dc_coef, scaling_list_delta_coef) are decoded.

[0161] Table 7 Scaling list data syntax

[0162]

[0163]

[0164] Picture header

[0165] The picture header is sent at the start of each picture before the other slice data. This is very large compared to the previous headers in the previous drafts of the standard. A complete description of all these parameters can be found in JVET_Q2001-vD. Table 9 shows these parameters in the current picture header decoding syntax.

[0166] The relevant syntax elements that can be decoded involve:

[0167] · Whether to use the picture and reference frame

[0168] · The type of the picture

[0169] · Output frame

[0170] · The number of pictures

[0171] · Use sub - pictures (if needed)

[0172] · List of reference pictures (if needed)

[0173] · Color plane (if needed)

[0174] · Partition update (if the overwrite flag is enabled)

[0175] · Delta QP parameter (if needed)

[0176] · Motion information parameter (if needed)

[0177] · ALF parameter (if needed)

[0178] · SAO parameter (if needed)

[0179] · Quantization parameter (if needed)

[0180] · LMCS parameter (if needed)

[0181] · Scaling list parameter (if needed)

[0182] · Picture header extension (if needed)

[0183] · And so on

[0184] Picture "type"

[0185] The first flag is grd_or_irap_pic_flag, which indicates whether the current picture is a resynchronization picture (IRAP or GDR). If this flag is true, then decode gdr_pic_flag to know whether the current picture is an IRAP picture or a GDR picture.

[0186] Then decode ph_inter_slice_allowed_flag to identify the allowed inter - slice.

[0187] When they are allowed, decode the flag ph_infra_slice_allowed_flag to know whether intra - slice is allowed for the current picture.

[0188] Then decode the non_reference_picture_flag, the ph_pic_parameter_set_id indicating the PPS ID, and the ph_pic_order_cnt_lsb of the picture order count. The picture order count gives the number of the current picture.

[0189] If the picture is a GDR or IRAP picture, then decode the no_output_of_prior_pics_flag.

[0190] And if the picture is a GDR, then decode the recovery_poc_cnt. Then, if necessary, decode the ph_poc_msb_present_flag and the poc_msb_val.

[0191] ALF

[0192] After these parameters that describe important information about the current picture, if ALF is enabled at the SPS level and if ALF is enabled at the picture header level, then decode the set of ALF APS ID syntax elements. ALF is enabled at the SPS level due to the sps_alf_enabled_flag. And ALF is signaled at the picture header level since alf_info_in_ph_flag is equal to 1, otherwise (alf_info_in_ph_flag is equal to 0), ALF is signaled at the slice level.

[0193] The alf_info_in_ph_flag is defined as follows:

[0194] "alf_info_in_ph_flag being equal to 1 specifies that ALF information is present in the PH syntax structure and not present in the slice header that refers to a PPS that does not contain the PH syntax structure. alf_info_in_ph_flag being equal to 0 specifies that ALF information is not present in the PH syntax structure and may be present in the slice header that refers to a PPS that does not contain the PH syntax structure."

[0195] First, decode the ph_alf_enabled_present_flag to determine whether the ph_alf_enabled_flag should be decoded. If the ph_alf_enabled_present_flag is enabled, then ALF is enabled for all slices of the current picture.

[0196] If ALF is enabled, the pic_num_alf_aps_ids_luma syntax element is used to decode the amount of ALF APS IDs for luma. For each APS ID, the APS ID value for luma "ph_alf_aps_id_luma" is decoded.

[0197] For chroma, the syntax element ph_alf_chroma_idc is decoded to determine whether ALF is enabled for chroma, for Cr only, or for Cb only. If enabled, the ph_alf_aps_id_chroma syntax element is used to decode the value of the APS ID for chroma.

[0198] In this way, the APS IDs for the CC-ALF method are decoded if the Cb and / or Cr components require it.

[0199] LMCS

[0200] If LMCS is enabled at the SPS level, a set of LMCS APS ID syntax elements is decoded. First, the ph_lmcs_enabled_flag is decoded to determine whether LMCS is enabled for the current picture. If LMCS is enabled, the ID value ph_lmcs_aps_id is decoded. For chroma, only the ph_chroma_residual_scale_flag is decoded to enable or disable the method for chroma.

[0201] Scaling list

[0202] If the scaling list is enabled at the SPS level, a set of scaling list APS IDs is decoded. The ph_scaling_list_present_flag is decoded to determine whether the scaling matrix is enabled for the current picture. And then the value of the APS ID (ph_scaling_list_aps_id) is decoded.

[0203] Sub-picture

[0204] When the sub-picture parameters are enabled at the SPS and if signaling of the sub-picture ID is disabled, the sub-picture parameters are enabled. Some information about virtual boundaries is also included. For the sub-picture parameters, eight syntax elements are defined:

[0205] ·ph_virtual_boundaries_present_flag

[0206] ·ph_num_ver_virtual_boundaries

[0207] ·ph_virtual_boundaries_pos_x[i]

[0208] ·ph_num_hor_virtual_boundaries

[0209] ·ph_virtual_boundaries_pos_y[i]

[0210] Output flag

[0211] These sub-picture parameters are followed by pic_output_flag (if present).

[0212] Reference picture list

[0213] If the reference picture list is signaled in the picture header (due to rpl_info_in_ph_flag being equal to 1), then the parameters of the reference picture list ref_pic_lists() are decoded, which contain the following syntax elements:

[0214] ·rpl_sps_flag[]

[0215] ·rpl_idx[]

[0216] ·poc_lsb_lt[][]

[0217] ·delta_poc_msb_present_flag[][]

[0218] ·delta_poc_msb_cycle_lt[][]

[0219] Partition

[0220] If necessary, the set of partition parameters is decoded, and the set of partition parameters contains the following syntax elements:

[0221] ·partition_constraints_override_flag

[0222] ·ph_log2_diff_min_qt_min_cb_intra_slice_luma

[0223] ·ph_max_mtt_hierarchy_depth_intra_slice_luma

[0224] ·ph_log2_diff_max_bt_min_qt_intra_slice_luma

[0225] ·ph_log2_diff_max_tt_min_qt_intra_slice_luma

[0226] ·ph_log2_diff_min_qt_min_cb_intra_slice_chroma

[0227] ·ph_max_mtt_hierarchy_depth_intra_slice_chroma

[0228] ·ph_log2_diff_max_bt_min_qt_intra_slice_chroma

[0229] ·ph_log2_diff_max_tt_min_qt_intra_slice_chroma

[0230] ·ph_log2_diff_min_qt_min_cb_inter_slice

[0231] ·ph_max_mtt_hierarchy_depth_inter_slice

[0232] ·ph_log2_diff_max_bt_min_qt_inter_slice

[0233] ·ph_log2_diff_max_tt_min_qt_inter_slice

[0234] Weighted prediction

[0235] If the weighted prediction method is enabled at the PPS level and if the weighted prediction parameters are signaled in the picture header (wp_info_in_ph_flag equal to 1), then decode the weighted prediction parameters pred_weight_table().

[0236] When bidirectional prediction weighted prediction is enabled, pred_weight_table() contains the weighted prediction parameters for list L0 and list L1. As depicted in the pred_weight_table() syntax table (Table 8), when the weighted prediction parameters are sent in the picture header, the number of weights for each list is sent explicitly.

[0237] Table 8 Weighted prediction parameter syntax

[0238]

[0239]

[0240]

[0241] Incremental QP

[0242] When the picture is intra, if needed, decode ph_cu_qp_delta_subdiv_intra_slice and ph_cu_chroma_qp_offset_subdiv_intra_slice. And if inter slice strips are allowed, decode ph_cu_qp_delta_subdiv_inter_slice and ph_cu_chroma_qp_offset_subdiv_inter_slice when needed. Finally, decode the picture header extension syntax elements if needed.

[0243] Signal all parameters alf_info_in_ph_flag, rpl_info_in_ph_flag, qp_delta_info_in_ph_flag, sao_info_in_ph_flag, dbf_info_in_ph_flag, wp_info_in_ph_flag in the PPS.

[0244] Table 9 Picture Header Structure

[0245]

[0246]

[0247]

[0248]

[0249]

[0250]

[0251]

[0252] Slice Header

[0253] Send the slice header at the start of each slice. The slice header contains approximately 65 syntax elements. This is very large compared to the previous slice headers in earlier video coding standards. A complete description of all slice header parameters can be found in JVET-Q2001-vD. Table 10 shows these parameters in the current slice header decoding syntax.

[0254] Table 10 Partial Slice Header

[0255]

[0256]

[0257]

[0258]

[0259]

[0260] First, decode picture_header_in_slice_header_flag to know whether picture_header_structure() exists in the slice header.

[0261] Then, if necessary, decode slice_subpic_id to determine the sub-picture ID of the current slice. Then decode slice_address to determine the address of the current slice. If the current slice mode is the rectangular slice mode (rest_slice_flag equals 1) and if the number of slices in the current sub-picture is higher than 1, decode the slice address. If the current slice mode is the raster scan mode (rest_slice_flag equals 0) and if the number of blocks in the current picture is higher than 1 calculated based on the variables defined in the PPS, the slice address can also be decoded.

[0262] If the number of blocks in the current picture is greater than 1 and if the current slice mode is not the rectangular slice mode, decode num_tiles_in_slice_minus1. In the current VVC draft specification, num_tiles_in_slice_minus1 is defined as follows:

[0263] "num_tiles_in_slice_minus1 plus 1 (when present) specifies the number of blocks in the slice. The value of num_tiles_in_slice_minus1 shall be in the range of 0 to NumTilesInPic - 1 (inclusive of the end values)."

[0264] Then decode slice_type.

[0265] If ALF is enabled at the SPS level (sps_alf_enabled_flag) and if ALF is signaled in the slice header (alf_info_in_ph_flag equal to 0), then decode the ALF information. This includes the flag indicating that ALF is enabled for the current slice (slice_alf_enabled_flag). If enabled, decode the number of APS ALF IDs for luma (slice_num_alf_aps_ids_luma), and then decode the APS IDs (slice_alf_aps_id_luma[i]). Then, decode slice_alf_chroma_idc to know whether ALF is enabled for the chroma components and which chroma components are enabled. Then, if required, decode the APS IDs for chroma (slice_alf_aps_id_chroma). In the same way, if required, decode slice_cc_alf_cb_enabled_flag to know whether the CC ALF method is enabled. If the CC ALF is enabled and if the CC ALF is enabled for Cr and / or Cb, then decode the relevant APS IDs for Cr and / or Cb.

[0266] If the color planes are sent independently (separate_colour_plane_flag equal to 1), then decode the colour_plane_id.

[0267] When the reference picture list is not sent in the picture header (rpl_info_in_ph_flag equal to 0) and when the NAL unit is not an IDR or if the reference picture list is sent for an IDR picture (sps_idr_rpl_present_flag equal to 1), then decode the reference picture list parameters; these are similar to those in the picture header.

[0268] If the reference picture list is sent in the picture header (rpl_info_in_ph_flag equal to 1) or the NAL unit is not an IDR, or if the reference picture list is sent for an IDR picture (sps_idr_rpl_present_flag equal to 1), and if the number of references for at least one list is greater than 1, then decode the override flag num_ref_idx_active_override_flag.

[0269] If the flag is enabled, then decode the reference indices for each list.

[0270] If num_ref_idx_active_override_flag is enabled, decode the number num_ref_idx_active_minus1[i] of reference indices for each list “i” when needed. The number of reference index overrides for the current list shall be lower than or equal to the number of reference frame indices signaled in ref_pic_lists(). Thus, the override either reduces or does not reduce the maximum number of reference frames for each list.

[0271] When the slice type is not intra, and if needed, decode cabac_init_flag. If the reference picture lists are sent in the slice header and other conditions occur, decode slice_collocated_from_l0_flag and slice_collocated_ref_idx. These data are related to CABAC coding and collocated motion vectors.

[0272] In the same way, when the slice type is not intra, decode the parameters of weighted prediction pred_weight_table().

[0273] If the delta QP information is sent in the slice header (qp_delta_info_in_ph_flag equal to 0), decode slice_qp_delta. If needed, decode the syntax elements slice_cb_qp_offset, slice_cr_qp_offset, slice_joint_cbcr_qp_offset, cu_chroma_qp_offset_enabled_flag.

[0274] If SAO information is sent in the slice header (sao_info_in_ph_flag equal to 0) and if it is enabled at the SPS level (sps_sao_enabled_flag), decode the SAO enable flags for both luma and chroma: slice_sao_luma_flag, slice_sao_chroma_flag.

[0275] Then, if the deblocking filter parameters are signaled in the slice header (dbf_info_in_ph_flag equal to 0), decode the deblocking filter parameters.

[0276] Systematically decode the flag slice_ts_residual_coding_disabled_flag to know whether the transform skip residual coding method is enabled for the current slice.

[0277] If LMCS is enabled in the picture header (ph_lmcs_enabled_flag equals 1), then the flag slice_lmcs_enabled_flag is decoded.

[0278] In the same way, if the scaling list is enabled in the picture header (phpic_scaling_list_presentenabled_flag equals 1), then the flag slice_scaling_list_present_flag is decoded.

[0279] Then, if necessary, other parameters are decoded.

[0280] The picture header is within the slice header

[0281] In a specific signaling manner, as Figure 7 depicted, the picture header (708) can be signaled within the slice header (710). In this case, there is no NAL unit that contains only the picture header (608). NAL units 701 - 707 correspond to Figure 6 the corresponding NAL units 601 - 607 in Figure 6 Similarly, coded block group 720 and coded block 740 correspond to

[0282] The flag picture_header_in_slice_header_flag is defined as follows:

[0283] "picture_header_in_slice_header_flag equals 1 specifies the presence of the PH syntax structure in the slice header. picture_header_in_slice_header_flag equals 0 specifies the absence of the PH syntax structure in the slice header.

[0284] The requirement for bitstream conformance is that the value of picture_header_in_slice_header_flag shall be the same for all coded slices in the CLVS.

[0285] For bitstream conformance, when picture_header_in_slice_header_flag equals 1 for a coded slice, there shall be no VCL NAL unit with nal_unit_type equal to PH_NUT in the CLVS.

[0286] When picture_header_in_slice_header_flag equals 0, all coded slices in the current picture shall have picture_header_in_slice_header_flag equal to 0, and the current PU shall have a PH NAL unit.

[0287] picture_header_structure() contains the syntax elements of picture_rbsp() except for the filler bits rbsp_trailing_bits().

[0288] Streaming applications

[0289] Some streaming applications only extract certain parts of the bitstream. These extractions can be spatial (as sub - pictures) or temporal (sub - parts of the video sequence). Then, these extracted parts can be merged with other bitstreams. Other frames reduce the frame rate by only extracting some frames. Generally, the main purpose of these streaming applications is to use the maximum allowed bandwidth to produce the highest quality for the end - user.

[0290] In VVC, for frame rate reduction, the APS ID numbers have been restricted so that the new APS ID number of a frame cannot be used for frames in the upper layer in the temporal hierarchy. However, for streaming applications that extract parts of the bitstream, it is necessary to track the APS ID to determine which APSs should be retained for the sub - parts of the bitstream, because frames (due to IRAP) do not reset the numbering of APS IDs.

[0291] LMCS (Luminance Mapping with Chroma Scaling)

[0292] The Luminance Mapping with Chroma Scaling (LMCS) technique is a sample - value conversion method applied to blocks before applying loop filters in a video decoder such as VVC.

[0293] LMCS can be divided into two sub - tools. The first sub - tool is applied to luminance blocks, and the second sub - tool is applied to chroma blocks, as described below:

[0294] 1) The first sub-tool is the in-loop mapping of the luminance component based on an adaptive piecewise linear model. The in-loop mapping of the luminance component adjusts the dynamic range of the input signal by redistributing the codewords across the dynamic range to improve the compression efficiency. The luminance mapping utilizes a forward mapping function into the "mapping domain" and a corresponding inverse mapping function back to the "input domain".

[0295] 2) The second sub-tool is related to the chrominance component that applies luminance-dependent chrominance residual scaling. The chrominance residual scaling is designed to compensate for the interaction between the luminance signal and its corresponding chrominance signals. The chrominance residual scaling depends on the average of the reconstructed neighboring luminance samples above and / or to the left of the current block.

[0296] Similar to most other tools in a video encoder such as VVC, the LMCS can be enabled / disabled at the sequence level using an SPS flag. Whether the chrominance residual scaling is enabled is also signaled at the slice level. If the luminance mapping is enabled, an additional flag is signaled to indicate whether the luminance-dependent chrominance residual scaling is enabled. When the luminance mapping is not used, the luminance-dependent chrominance residual scaling is completely disabled. Additionally, for chrominance blocks of size less than or equal to 4, the luminance-dependent chrominance residual scaling is always disabled.

[0297] Figure 8 Illustrates the principle of the LMCS as described above for the luminance mapping sub-tool. Figure 8 The shaded blocks in are the new LMCS functional blocks, including the forward and inverse mapping of the luminance signal. It is important to note that when the LMCS is used, some decoding operations are applied in the "mapping domain". These operations are represented by the dashed blocks in this Figure 8 They typically correspond to inverse quantization, inverse transform, intra-luminance prediction, and the reconstruction step (which consists of adding the luminance prediction and the luminance residual). Conversely, Figure 8 the solid blocks in indicate the locations where the decoding processing is applied in the original (i.e., non-mapped) domain, and this includes loop filtering such as deblocking, ALF, and SAO, motion compensation prediction, and the storage of the decoded pictures as reference pictures (DPB).

[0298] Figure 9 Illustrates a figure similar to Figure 8 but this time it is for the chrominance scaling sub-tool of the LMCS tool. Figure 9 The shaded blocks in are the new LMCS functional blocks, which include the luminance-dependent chrominance scaling processing. However, in terms of chrominance, there are some important differences compared to the luminance case. Here, for chrominance samples, only inverse quantization and inverse transform represented by the blocks in the dashed line are performed in the "mapping domain". All other steps of intra-chrominance prediction, motion compensation, and loop filtering are performed in the original domain. As Figure 9 shown, for the luminance mapping, there is only the scaling processing and no forward and inverse processing.

[0299] Luminance mapping using a piecewise linear model

[0300] The luminance mapping sub-tool uses a piecewise linear model. This means that the piecewise linear model divides the input signal dynamic range into 16 equal sub-ranges, and for each sub-range, the number of codewords assigned to that range is used to represent its linear mapping parameter.

[0301] Semantics of luminance mapping

[0302] The syntax element lmcs_min_bin_idx specifies the minimum bin index used in the construction process of luminance mapping with chroma scaling (LMCS). The value of lmcs_min_bin_idx should be in the range of 0 to 15 (including the end values).

[0303] The syntax element lmcs_delta_max_bin_idx specifies the incremental value between 15 and the maximum bin index LmcsMaxBinIdx used in the construction process of luminance mapping with chroma scaling. The value of lmcs_delta_max_bin_idx should be in the range of 0 to 15 (including the end values). The value of LmcsMaxBinIdx is set to be equal to 15 - lmcs_delta_max_bin_idx. The value of LmcsMaxBinIdx should be greater than or equal to lmcs_min_bin_idx.

[0304] The syntax element lmcs_delta_cw_prec_minus1 plus 1 specifies the number of bits used to represent the syntax lmcs_delta_abs_cw[i].[[]]

[0305] The syntax element lmcs_delta_abs_cw[i] specifies the absolute incremental codeword value of the i-th bin.

[0306] The syntax element lmcs_delta_sign_cw_flag[i] specifies the sign of the variable lmcsDeltaCW[i]. When lmcs_delta_sign_cw_flag[i] does not exist, it is inferred to be equal to 0.

[0307] LMCS intermediate variable calculation for luminance mapping

[0308] To apply the forward and inverse luminance mapping processes, some intermediate variables and data arrays are required.

[0309] First, the variable OrgCW is derived as follows:

[0310] OrgCW = (1 << BitDepth) / 16

[0311] Then, the variable lmcsDeltaCW[i] (where i = lmcs_min_bin_idx…LmcsMaxBinIdx) is calculated as follows:

[0312] lmcsDeltaCW[i] = (1 - 2 * lmcs_delta_sign_cw_flag[i]) * lmcs_delta_abs_cw[i]

[0313] The new variable lmcsCW[i] is derived as follows:

[0314] - For i = 0…lmcs_min_bin_idx - 1, lmcsCW[i] is set to be equal to 0.

[0315] - For i = lmcs_min_bin_idx…LmcsMaxBinIdx, the following is applied:

[0316] lmcsCW[i] = OrgCW + lmcsDeltaCW[i]

[0317] The value of lmcsCW[i] should be in the range of (OrgCW >> 3) to (OrgCW << 3 - 1) (including the end values).

[0318] - For i = LmcsMaxBinIdx + 1…15, lmcsCW[i] is set to be equal to 0.

[0319] The variable InputPivot[i] (where i = 0…16) is derived as follows:

[0320] InputPivot[i] = i * OrgCW

[0321] The variables LmcsPivot[i] (where i = 0…16), the variables ScaleCoeff[i] and InvScaleCoeff[i] (where i = 0…15) are calculated as follows:

[0322]

[0323] Forward luminance mapping

[0324] As Figure 8 shown, when LMCS is applied to luminance, the luminance remapped samples called predMapSamples[i][j] are obtained from the predicted samples predSamples[i][j].

[0325] predMapSamples[i][j] is calculated as follows:

[0326] First, calculate the index idxY from the predicted sample predSamples[i][j] at position (i, j).

[0327] idxY = predSamples[i][j] >> Log2(OrgCW)

[0328] Then, derive predMapSamples[i][j] as follows using the intermediate variables idxY, LmcsPivot[idxY], and InputPivot[idxY] with partial 0s:

[0329] predMapSamples[i][j] = LmcsPivot[idxY]

[0330] +(ScaleCoeff[idxY] * (predSamples[i][j] - InputPivot[idxY])+(1 << 10)) >> 11

[0331] Luminance reconstruction sample

[0332] Obtain the reconstruction process from the predicted luminance sample predMapSample[i][j] and the residual luminance sample resiSamples[i][j].

[0333] Simply obtain the reconstructed luminance picture sample recSamples[i][j] by adding predMapSample[i][j] to resiSamplei[i][j] as follows:

[0334] recSamples[i][j] = Clip1(predMapSamples[i][j]+resiSamples[i][j]])

[0335] In the above relationship, the Clip1 function is a clipping function to ensure that the reconstructed sample is between 0 and 1 << BitDepth - 1.

[0336] Inverse luminance mapping

[0337] When applying the inverse luminance mapping according to Figure 8 apply the following operations to each sample recSample[i][j] of the current block being processed:

[0338] First, calculate the index idxY from the reconstructed sample recSamples[i][j] at position (i, j).

[0339] idxY = recSamples[i][j] >> Log2(OrgCW)

[0340] The inverse mapped luminance sample invLumaSample[i][j] is derived as follows:

[0341] invLumaSample[i][j] =

[0342] InputPivot[idxYInv] + (InvScaleCoeff[idxYInv] *

[0343] (recSample[i][j] - LmcsPivot[idxYInv]) + (1 << 10)) >> 11

[0344] Then a clipping operation is performed to obtain the final sample:

[0345] finalSample[i][j] = Clip1(invLumaSample[i][j])

[0346] Chroma scaling

[0347] LMCS semantics for chroma scaling

[0348] The syntax element lmcs_delta_abs_crs in Table 6 specifies the absolute codeword value of the variable lmcsDeltaCrs. The value of lmcs_delta_abs_crs shall be in the range of 0 to 7 (inclusive of the end values). When not present, lmcs_delta_abs_crs is inferred to be equal to 0.

[0349] The syntax element lmcs_delta_sign_crs_flag specifies the sign of the variable lmcsDeltaCrs. When not present, lmcs_delta_sign_crs_flag is inferred to be equal to 0.

[0350] LMCS intermediate variable calculation for chroma scaling

[0351] To apply the chroma scaling process, some intermediate variables are required.

[0352] The variable lmcsDeltaCrs is derived as follows:

[0353] lmcsDeltaCrs = (1 - 2 * lmcs_delta_sign_crs_flag) * lmcs_delta_abs_crs

[0354] The variable ChromaScaleCoeff[i] (where i = 0...15) is derived as follows:

[0355]

[0356] Chroma scaling processing

[0357] In the first step, the variable invAvgLuma is derived to calculate the average luminance value of the reconstructed luminance samples around the current corresponding chroma block. The average luminance is calculated from the left luminance block and the upper luminance block surrounding the corresponding chroma block.

[0358] If no samples are available, the variable invAvgLuma is set as follows:

[0359] invAvgLuma = 1 << (BitDepth - 1)

[0360] Based on the intermediate array LmcsPivot[] of part 0, the variable idxYInv is then derived as follows:

[0361]

[0362] The variable varScale is derived as follows:

[0363] varScale = ChromaScaleCoeff[idxYInv]

[0364] When applying the transform to the current chroma block, the reconstructed chroma picture sample array recSamples is derived as follows:

[0365] recSamples[i][j] = Clip1(predSamples[i][j] +

[0366] Sign(resiSamples[i][j]) * ((Abs(resiSamples[i][j]) * varScale + (1 << 10)) >> 11))

[0367] If the transform has not been applied to the current block yet, the following is applied:

[0368] recSamples[i][j] = Clip1(predSamples[i][j])

[0369] Encoder considerations

[0370] The basic principle of the LMCS encoder is to first allocate more codewords to those ranges of the dynamic range segments that have lower variance than the average variance. In an alternative concept of this, the main objective of LMCS is to allocate fewer codewords to those dynamic range segments that have higher variance than the average variance. In this way, the smooth regions of the picture will be encoded with more codewords than the average, and vice versa.

[0371] All parameters of the LMCS tool stored in the APS are determined at the encoder side (see Table 6). The LMCS encoder algorithm is based on the evaluation of local luminance variance, and the determination of LMCS parameters is optimized according to the above basic principle. Then optimization is carried out to obtain the best PSNR metric for the final reconstructed samples of a given block.

[0372] Embodiment

[0373] Avoid slice address syntax elements when not needed

[0374] In one embodiment, when signaling the picture header in the slice header, even if the number of tiles is greater than 1, it is inferred that the slice_address syntax element is equal to the value 0. Table 11 shows this embodiment.

[0375] The advantage of this embodiment is that when the picture header is in the slice header, the slice address is not parsed, which reduces the bit rate, especially for low-latency and low-bit-rate applications, and reduces the parsing complexity of some implementations when signaling the picture in the slice header.

[0376] In the embodiment, this is only applied to the raster scan slice mode (rect_slice_flag equal to 0). This reduces the parsing complexity of some implementations.

[0377] Table 11 shows the modified partial slice header

[0378]

[0379] Avoid sending the number of tiles in a slice when not needed

[0380] In one embodiment, when sending the picture header in the slice header, the number of tiles in the slice is not sent. Table 12 shows this embodiment, where when the flag picture_header_in_slice_header_flag is set to equal 1, the num_tiles_in_slice_minus1 syntax element is not sent. The advantage of this embodiment is that the bit rate is reduced, especially for low-latency and low-bit-rate applications, because there is no need to send the number of tiles.

[0381] In an embodiment, this only applies to the raster scan strip mode (rect_slice_flag equal to 0). This reduces the parsing complexity for some implementations.

[0382] Table 12 shows the modified partial strip header

[0383]

[0384] Predicted by the PPS value NumTilesInPic (semantics)

[0385] In an additional embodiment, when the picture header is sent in the strip header, it is inferred that the number of tiles in the current strip is equal to the number of tiles in the picture. This can be set by adding the following sentence in the semantics of the syntax element num_tiles_in_slice_minus1: "When not present, the variable num_tiles_in_slice_minus1 is set to be equal to NumTilesInPic - 1".

[0386] Where the variable NumTilesInPic gives the maximum number of tiles in the picture. This variable is calculated based on the syntax elements sent in the PPS.

[0387] The number of tiles is set before the strip address and the unnecessary transmission of slice_address is avoided

[0388] In an embodiment, the syntax element dedicated to the number of tiles in the strip is sent before the strip address, and its value is used to know whether the strip address needs to be decoded. More precisely, the number of tiles in the strip is compared with the number of tiles in the picture to know whether the strip address needs to be decoded. In fact, if the number of tiles in the strip is equal to the number of tiles in the picture, it is ensured that the current picture contains only one strip.

[0389] In an embodiment, this only applies to the raster scan strip mode (rect_slice_flag equal to 0). This reduces the parsing complexity for some implementations.

[0390] Table 13 shows this embodiment. Where if the value of the syntax element num_tiles_in_slice_minus1 is equal to the variable NumTilesInPic minus 1, the syntax element slice_address is not decoded. When num_tiles_in_slice_minus1 is equal to the variable NumTilesInPic minus 1, it is inferred that slice_address is equal to 0.

[0391] Table 13 shows the modified partial strip header

[0392]

[0393] The advantage of this embodiment is that when the condition is set to true, the bit rate is reduced and the parsing complexity is reduced because the strip address is not sent.

[0394] In one embodiment, when the picture header is sent in the strip header, the syntax element indicating the number of blocks in the current strip is not decoded, and it is inferred that the number of blocks in the strip is equal to 1. And when the number of blocks in the strip is equal to the number of blocks in the picture, it is inferred that the strip address is equal to 0, and the related syntax element is not decoded. Table 14 shows this embodiment.

[0395] This increases the bit rate reduction obtained by the combination of these two embodiments.

[0396] Table 14 shows the modified partial strip header

[0397]

[0398] Remove the unnecessary condition NumTilesInPic > 1

[0399] In one embodiment, when the raster scan strip mode is enabled, it is not necessary to test the condition that the number of tiles in the current picture actually needs to be greater than 1 to decode the syntax elements slice_address and / or the number of blocks in the current strip. Specifically, when the number of tiles in the current picture is equal to 1, it is inferred that the rect_slice_flag value is equal to 1. Therefore, the raster scan strip mode cannot be enabled in this case. Table 15 shows this embodiment.

[0400] This embodiment reduces the parsing complexity of the strip header.

[0401] Table 15 shows the modified partial strip header

[0402]

[0403] In one embodiment, when the picture header is sent in the strip header and when the raster scan strip mode is enabled, the syntax element indicating the number of blocks in the current strip is not decoded, and it is inferred that the number of blocks in the strip is equal to 1. And when the number of blocks in the strip is equal to the number of blocks in the picture and when the raster scan strip mode is enabled, it is inferred that the strip address is equal to 0, and the related syntax element slice_address is not decoded. Table 16 shows this embodiment.

[0404] The advantages are reduced bit rate and reduced parsing complexity.

[0405] Table 16 shows the modified partial strip header

[0406]

[0407] Implement

[0408] Figure 11 FIG. 8 shows systems 191, 195 according to embodiments of the present invention, which include at least one of an encoder 150 or a decoder 100 and a communication network 199. According to an embodiment, system 195 is configured to process and provide content to a user (e.g., video and audio content for display / output or streaming of video / audio content), and the user accesses decoder 100, for example, through a user interface of a user terminal including decoder 100 or a user terminal communicable with decoder 100. Such a user terminal may be a computer, a mobile phone, a tablet, or any other type of device capable of providing / displaying (the provided / streamed) content to the user. System 195 obtains / receives a bitstream 101 via communication network 199 (in the form of a continuous stream or signal (e.g., while displaying / outputting earlier video / audio)). According to an embodiment, system 191 is configured to process content and store the processed content, e.g., video and audio content processed for display / output / streaming at a later time. System 191 obtains / receives content including an original image sequence 151, which is received and processed by encoder 150 (including filtering using a deblocking filter according to the present invention), and encoder 150 generates a bitstream 101 that will be transmitted to decoder 100 via communication network 199. Then, bitstream 101 is transmitted to decoder 100 in a variety of ways. For example, it may be pre-generated by encoder 150 and stored as data in a storage device in communication network 199 (e.g., on a server or cloud storage device) until the user requests the content (i.e., the bitstream data) from the storage device, at which time the data is transmitted / streamed from the storage device to decoder 100. System 191 may also include a content providing device for providing / streaming to the user content information (e.g., the title of the content and other meta / storage location data for identifying, selecting, and requesting the content) of the content stored in the storage device (e.g., by transmitting data for a user interface to be displayed on the user terminal), and for receiving and processing a user's request for the content such that the requested content can be transmitted / streamed from the storage device to the user terminal. Alternatively, encoder 150 generates bitstream 101 and transmits / streams it directly to decoder 100 when the user requests the content. Then, decoder 100 receives bitstream 101 (or signal) and filters it using a deblocking filter according to the present invention to obtain / generate a video signal 109 and / or an audio signal, and then the user terminal uses the video signal 109 and / or the audio signal to provide the requested content to the user.

[0409] Any step of the method / process according to the present invention or the functions described herein can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the steps / functions can be stored as one or more instructions or code or programs or computer-readable media on one or more hardware-based processing units or transmitted via one or more hardware-based processing units and executed by one or more hardware-based processing units, such as programmable computing machines, which can be a PC (“personal computer”), DSP (“digital signal processor”), circuits, circuitry, processors and memories, general microprocessors or central processing units, microcontrollers, ASICs (“application specific integrated circuits”), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Thus, the term “processor” as used herein can refer to any one of the foregoing structures or any other structure suitable for implementing the techniques described herein.

[0410] Embodiments of the present invention can also be implemented by various apparatuses or devices, including wireless handsets, integrated circuits (ICs) or JC collections (e.g., chip sets). Various components, modules or units are described herein to illustrate the functional aspects of the apparatuses / devices configured to perform these embodiments, but need not necessarily be implemented by different hardware units. Rather, the various modules / units can be combined in a codec hardware unit or provided by a collection of interoperating hardware units, including one or more processors in conjunction with suitable software / firmware.

[0411] Embodiments of the present invention can be implemented by a computer that reads and executes computer-executable instructions (e.g., one or more programs) recorded on a storage medium to perform one or more of the modules / units / functions in the above embodiments and / or a system or device including one or more processing units or circuits for performing one or more of the functions in the above embodiments, and can be implemented by a method performed by the computer of the system or device, e.g., reading and executing computer-executable instructions from a storage medium to perform one or more of the functions in the above embodiments and / or controlling one or more processing units or circuits to perform one or more of the functions in the above embodiments. The computer can include a single computer or a network of individual processing units to read and execute the computer-executable instructions. The computer-executable instructions can be provided to the computer, for example, via a network or a tangible storage medium from a computer-readable medium such as a communication medium. The communication medium can be a signal / bitstream / carrier wave. The tangible storage medium is a "non-transitory computer-readable storage medium" which can include, for example, one or more of a hard disk, random access memory (RAM), read-only memory (ROM), storage devices of a distributed computing system, optical discs (e.g., compact disc (CD), digital versatile disc (DVD) or Blu-ray disc (BD) TM ), flash memory devices, memory cards, etc. At least some of the steps / functions can also be implemented in hardware by a machine or a dedicated component such as an FPGA ("field programmable gate array") or an ASIC ("application specific integrated circuit").

[0412] Figure 12FIG. 0 is a schematic block diagram of a computing device 2000 for implementing one or more embodiments of the present invention. The computing device 2000 may be a device such as a microcomputer, a workstation, or a light portable device. The computing device 2000 includes a communication bus connected to the following: - a central processing unit (CPU) 2001, such as a microprocessor; - a random access memory (RAM) 2002 for storing executable code of the method of the embodiments of the present invention and registers adapted to record variables and parameters required for implementing a method for encoding or decoding at least a part of an image according to an embodiment of the present invention, the storage capacity of which may be extended, for example, by an optional RAM connected to an expansion port; - a read-only memory (ROM) 2003 for storing a computer program for implementing the embodiments of the present invention; - a network interface (NET) 2004, which is generally connected to a communication network through which digital data to be processed is transmitted or received. The network interface (NET) 2004 may be a single network interface or consist of a set of different network interfaces (for example, wired and wireless interfaces, or different types of wired or wireless interfaces). Under the control of a software application running in the CPU 2001, data packets are written to the network interface for transmission or read from the network interface for reception; - a user interface (UI) 2005, which may be used to receive input from a user or display information to the user; - a hard disk (HD) 2006, which may be provided as a mass storage device; - an input / output module (IO) 2007, which may be used to receive / send data from / to an external device (such as a video source or a display). The executable code may be stored in the ROM 2003, on the HD 2006, or on a removable digital medium such as a disk. According to a variant, the executable code of the program may be received via the NET 2004 by means of a communication network and stored in one of the storage components (such as the HD 2006, etc.) of the computing device 2000 before being executed. The CPU 2001 is adapted to control and direct the execution of instructions or portions of software code of one or more programs according to embodiments of the present invention, the instructions being stored in one of the aforementioned storage components. For example, after power-on, the CPU 2001 is capable of executing those instructions related to the software application from the main RAM memory 2002 after loading the instructions from the program ROM 2003 or the HD 2006. Such a software application, when executed by the CPU 2001, causes the steps of the method according to the present invention to be performed.

[0413] It should also be understood that, according to other embodiments of the present invention, a decoder according to the above embodiments is provided in a user terminal such as a computer, a mobile phone (cellular phone), a tablet, or any other type of device capable of providing / displaying content to a user (e.g., a display device). According to yet another embodiment, an encoder according to the above embodiments is provided in an image capture device, which further includes a camera, a video camera, or a network camera (e.g., a closed-circuit television or video surveillance camera) for capturing and providing content for the encoder to encode. The following refers to Figure 13 and 14 Two such examples are provided.

[0414] Network camera

[0415] Figure 13 FIG. is an illustration of a network camera system 2100 including a network camera 2102 and a client device 2104.

[0416] The network camera 2102 includes an imaging unit 2106, an encoding unit 2108, a communication unit 2110, and a control unit 2112.

[0417] The network camera 2102 and the client device 2104 are interconnected via a network 200 to enable communication with each other.

[0418] The imaging unit 2106 includes a lens and an image sensor (e.g., a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS)), and captures an image of an object and generates image data based on the image. The image can be a still image or a video image.

[0419] The encoding unit 2108 encodes the image data by using the encoding method described above.

[0420] The communication unit 2110 of the network camera 2102 transmits the encoded image data encoded by the encoding unit 2108 to the client device 2104.

[0421] In addition, the communication unit 2110 receives commands from the client device 2104. The commands include commands for setting parameters for the encoding of the encoding unit 2108.

[0422] The control unit 2112 controls other units in the network camera 2102 according to the commands received by the communication unit 2110.

[0423] The client device 2104 includes a communication unit 2114, a decoding unit 2116, and a control unit 2118.

[0424] The communication unit 2114 of the client device 2104 transmits commands to the network camera 2102.

[0425] In addition, the communication unit 2114 of the client device 2104 receives the encoded image data from the network camera 2102.

[0426] The decoding unit 2116 decodes the encoded image data by using the decoding method described above.

[0427] The control unit 2118 of the client device 2104 controls other units in the client device 2104 according to user operations or commands received by the communication unit 2114.

[0428] The control unit 2118 of the client device 2104 controls the display device 2120 to display the image decoded by the decoding unit 2116.

[0429] The control unit 2118 of the client device 2104 also controls the display device 2120 to display a GUI (Graphical User Interface) for specifying values of parameters of the network camera 2102 (including parameters for encoding by the encoding unit 2108).

[0430] The control unit 2118 of the client device 2104 also controls other units in the client device 2104 according to user operation inputs to the GUI displayed on the display device 2120.

[0431] The control unit 2118 of the client device 2104 controls the communication unit 2114 of the client device 2104 according to user operation inputs to the GUI displayed on the display device 2120, so as to transmit a command for specifying values of parameters of the network camera 2102 to the network camera 2102.

[0432] Smartphone

[0433] Figure 14 is a diagram illustrating the smartphone 2200.

[0434] The smartphone 2200 includes a communication unit 2202, a decoding unit 2204, a control unit 2206, a display unit 2208, an image recording device 2210, and a sensor 2212.

[0435] The communication unit 2202 receives the encoded image data via the network 200.

[0436] The decoding unit 2204 decodes the encoded image data received by the communication unit 2202.

[0437] The decoding unit 2204 decodes the encoded image data by using the decoding method described above.

[0438] The control unit 2206 controls other units in the smart phone 2200 according to user operations or commands received by the communication unit 2202.

[0439] For example, the control unit 2206 controls the display unit 2208 to display the image decoded by the decoding unit 2204.

[0440] Although the present invention has been described with reference to the embodiments, it should be understood that the present invention is not limited to the disclosed embodiments. Those skilled in the art will understand that various changes and modifications can be made without departing from the scope of the present invention defined by the appended claims. All features disclosed in this specification (including any appended claims, abstract and drawings), and / or all steps of any method or process disclosed, may be combined in any combination, except for at least some combinations of such features and / or steps that are mutually exclusive. Unless otherwise expressly stated, each feature disclosed in this specification (including any appended claims, abstract and drawings) may be replaced by alternative features for the same, equivalent or similar purposes. Therefore, unless otherwise expressly stated, each disclosed feature is only an example of a general series of equivalent or similar features.

[0441] It should also be understood that any result of the above comparison, determination, evaluation, selection, execution, performance or consideration (e.g., a selection made during an encoding or filtering process) may be indicated in the data in the bitstream (e.g., a flag or data indicating the result) or determinable / inferable from the data in the bitstream, such that the indicated or determined / inferred result can be used in the process instead of actually performing the comparison, determination, evaluation, selection, execution, performance or consideration, e.g., during a decoding process.

[0442] In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. The mere fact that different features are recited in mutually different dependent claims does not indicate that these features cannot be used advantageously in combination.

[0443] The reference signs appearing in the claims are for illustration only and should not limit the scope of the claims.

Claims

1. A method for decoding video data from a bitstream, the bitstream including video data corresponding to one or more strips, wherein, each strip can include one or more blocks, wherein the bitstream includes a picture header and a strip header, the picture header including syntax elements to be used when decoding one or more strips, the strip header including syntax elements to be used when decoding a strip, and wherein the method includes: parsing the syntax elements; decoding weighted prediction parameters from the picture header according to the value of a flag in a picture parameter set, wherein when the value is 1, the flag indicates that the weighted prediction parameters can be present in the picture header; in the case where a strip includes multiple blocks, if a parsed second syntax element indicates the presence of a picture header in the strip header, omitting the parsing of a first syntax element indicating the address of the strip; and decoding the video data from the bitstream using the parsed syntax elements.

2. The method according to claim 1, wherein, the omission is to be performed when a raster scan strip mode is to be used for decoding a strip.

3. The method according to claim 1 or 2, wherein, if the second syntax element indicates the presence of the picture header in the strip header, omitting the parsing of a third syntax element representing the result of subtracting 1 from the number of blocks in the strip, and inferring the value of the third syntax element to be equal to 0 regardless of the number of blocks in the strip.

4. A method for encoding video data into a bitstream, the bitstream including the video data corresponding to one or more strips, wherein, each strip can include one or more blocks, wherein the bitstream includes a picture header and a strip header, the picture header including syntax elements to be used when decoding one or more strips, the strip header including syntax elements to be used when encoding a strip, and the method includes: determining one or more syntax elements for encoding the video data; encoding weighted prediction parameters in the picture header according to the value of a flag in a picture parameter set, wherein when the value of the flag is 1, the flag indicates that the weighted prediction parameters can be present in the picture header; in the case where a strip includes multiple blocks, if a second syntax element indicates the presence of a picture header in the strip header, omitting the encoding of a first syntax element indicating the address of the strip; and encoding the video data using one or more syntax elements.

5. The method according to claim 4, wherein, the omission is to be performed when a raster scan strip mode is to be used for encoding a strip.

6. The method according to claim 4 or 5, wherein, If the second syntax element indicates the presence of the picture header in the strip header, then the encoding of the third syntax element representing the result of subtracting 1 from the number of blocks in the strip is omitted, and the value of the third syntax element is inferred to be equal to 0 regardless of the number of blocks in the strip.

7. A method of encoding video data into a bitstream, the bitstream including the video data corresponding to one or more strips, wherein, each strip can include one or more blocks, wherein the bitstream includes a picture header and a strip header, the picture header including syntax elements to be used when decoding one or more strips, and the strip header including syntax elements to be used when encoding a strip, the bitstream is constrained such that: in the case where the bitstream includes a syntax element having a value indicating that the picture includes a plurality of blocks and the bitstream includes a syntax element indicating signaling of the picture header in the strip header, the bitstream further includes a syntax element indicating that a syntax element indicating the address of the strip will not be parsed, the method includes encoding the video data using the syntax elements, and the method includes: encoding weighted prediction parameters in the picture header according to the value of a flag in a picture parameter set, wherein when the value of the flag is 1, the flag indicates that the weighted prediction parameters can be present in the picture header.

8. A decoder for decoding video data from a bitstream, the decoder being configured to perform the method according to any one of claims 1 to 3.

9. An encoder for encoding video data into a bitstream, the encoder being configured to perform the method according to any one of claims 4 to 7.

10. A computer program product comprising a computer program which, when executed, causes the method according to any one of claims 1 to 7 to be performed.

11. A computer-readable storage medium storing a computer program which, when executed, causes the method according to any one of claims 1 to 7 to be performed.

Images