Encoder, decoder, and corresponding method

Abstract

A method for decoding a coded video bitstream, implemented by a video decoder, is disclosed. [Solution] The method includes the steps of: determining that the coded video sequence (CVS) of the coded video bitstream includes a video coding layer (VCL) network abstraction layer (NAL) unit having a progressive decode refresh (GDR) network abstraction layer (NAL) unit type (GDR_NUT), wherein the VCL NAL unit having the GDR_NUT includes a GDR picture; initiating decoding of the CVS in the GDR picture; and generating an image according to the decoded CVS.

Description

[Technical Field]

[0001] Technical field Generally, this disclosure describes techniques that support gradual decoding refresh in video coding. More specifically, this disclosure permits progressive intra refresh, which enables random access without the need to use intra random access point (IRAP) pictures. [Background technology]

[0002] Even the amount of video data required to depict a relatively short video can be considerable, which can cause difficulties when data is streamed or otherwise transmitted over communication networks with limited bandwidth. Therefore, video data is generally compressed before being transmitted over modern telecommunication networks. Video size can also be an issue when video is stored in storage, as memory resources may be limited. Video compressors often use software and / or hardware at the source to encode video data before transmission or storage, thereby reducing the amount of data required to represent the digital video image. The compressed data is then received at the destination by a video decompressor that decodes the video data. Due to limited network resources and the ever-increasing demand for higher video quality, improved compression and decompression techniques that improve the compression ratio with little to no sacrifice of image quality are desirable. [Overview of the Initiative]

[0003] The first aspect relates to a method for decoding a coded video bitstream, implemented by a video decoder. The method includes the steps of: determining by the video decoder that the coded video sequence (CVS) of the coded video bitstream includes a video coding layer (VCL) network abstraction layer (NAL) unit having a gradual decoding refresh (GDR) network abstraction layer (NAL) unit type (GDR_NUT), wherein the VCL NAL unit having the GDR_NUT includes a GDR picture; initiating decoding of the CVS in the GDR picture by the video decoder; and generating an image according to the decoded CVS by the video decoder.

[0004] This method provides a technique that allows progressive intra-refresh, enabling random access without the need to use intra-random access point (IRAP) pictures. A video coding layer (VCL) network abstraction layer (NAL) unit has a progressive decode refresh (GDR) network abstraction layer (NAL) unit type (GDR_NUT) to indicate that the VCL NAL unit contains a GDR picture. By using a GDR picture instead of an IRAP picture, a smoother, more consistent bitrate may be achieved, for example, due to the size of the GDR picture relative to the size of the IRAP picture, which allows for reduced end-to-end delay (i.e., latency). Thus, the coder / decoder (also known as "codec") in video coding is improved compared to current codecs. As a practical matter, the improved video coding process provides a better user experience when video is sent, received, and / or viewed.

[0005] In the first implementation of the method using the first aspect itself, the GDR picture is the first picture in CVS.

[0006] In the second implementation of the method, either by the first aspect itself or by any preceding implementation of the first aspect, the GDR picture is the first picture in the GDR period.

[0007] In the third implementation of the method, either the first aspect itself or any preceding implementation of the first aspect, the GDR picture has a temporal identifier (ID) equal to zero.

[0008] In the fourth implementation of the method, either the first aspect itself or any preceding implementation of the first aspect, an access unit containing a VCL NAL unit having GDR_NUT is referred to as a GDR access unit.

[0009] In a fifth implementation of the method, either the first aspect itself or any preceding implementation of the first aspect, GDR_NUT indicates to the video decoder that a VCL NAL unit having GDR_NUT contains a GDR picture.

[0010] The second aspect relates to a method for encoding a video bitstream, implemented by a video encoder. This method includes: the steps of: the video encoder determining random access points for a video sequence; the video encoder encoding a gradual decoding refresh (GDR) picture into video coding layer (VCL) network abstraction layer (NAL) units having a gradual decoding refresh (GDR) network abstraction layer (NAL) unit type (GDR_NUT) at the random access points for the video sequence; the video encoder generating a bitstream containing the video sequence having the GDR picture in the VCL NAL units having the GDR_NUT at the random access points; and the video encoder storing the bitstream for transmission to a video decoder.

[0011] This method provides a technique that allows progressive intra-refresh, enabling random access without the need to use intra-random access point (IRAP) pictures. A video coding layer (VCL) network abstraction layer (NAL) unit has a progressive decode refresh (GDR) network abstraction layer (NAL) unit type (GDR_NUT) to indicate that the VCL NAL unit contains a GDR picture. By using a GDR picture instead of an IRAP picture, a smoother, more consistent bitrate may be achieved, for example, due to the size of the GDR picture relative to the size of the IRAP picture, which allows for reduced end-to-end delay (i.e., latency). Thus, the coder / decoder (also known as "codec") in video coding is improved compared to current codecs. As a practical matter, the improved video coding process provides a better user experience when video is sent, received, and / or viewed.

[0012] In the first implementation of the method based on the second aspect itself, the GDR picture is the first picture in CVS.

[0013] In the second implementation of the method, either the second aspect itself or any preceding implementation of the second aspect, the GDR picture is the first picture in the GDR period.

[0014] In the third implementation of the method, either the second aspect itself or any preceding implementation of the second aspect, the GDR picture has a temporal identifier (ID) equal to zero.

[0015] In a fourth implementation of the method, either the second aspect itself or any preceding implementation of the second aspect, a VCL NAL unit having GDR_NUT is referred to as a GDR access unit.

[0016] In a fifth implementation of the method, either the second aspect itself or any preceding implementation of the second aspect, GDR_NUT indicates to the video decoder that a VCL NAL unit having GDR_NUT contains a GDR picture.

[0017] The third aspect relates to a decoding device. The decoding device includes a receiver configured to receive a coded video bitstream; a memory coupled to the receiver storing instructions; and a processor coupled to the memory, the processor being configured to execute the instructions and cause the decoding device to perform the following steps: determine that the coded video sequence (CVS) of the coded video bitstream includes a video coding layer (VCL) network abstraction layer (NAL) unit having a gradual decoding refresh (GDR) network abstraction layer (NAL) unit type (GDR_NUT), wherein the VCL NAL unit having the GDR_NUT includes a GDR picture; begin decoding the CVS in the GDR picture; and generate an image according to the decoded CVS.

[0018] The decoding device provides a technique that allows progressive intra-refresh, enabling random access without the need to use intra-random access point (IRAP) pictures. A Video Coding Layer (VCL) Network Abstraction Layer (NAL) unit has a Progressive Decode Refresh (GDR) Network Abstraction Layer (NAL) unit type (GDR_NUT) to indicate that the VCL NAL unit contains a GDR picture. By using a GDR picture instead of an IRAP picture, a smoother, more consistent bitrate may be achieved, for example, due to the size of the GDR picture relative to the size of the IRAP picture, which allows for reduced end-to-end delay (i.e., latency). Thus, the coder / decoder (also known as a "codec") in video coding is improved compared to current codecs. As a practical matter, the improved video coding process provides a better user experience when video is sent, received, and / or viewed.

[0019] In the first implementation of the decoding device according to the third aspect itself, the decoding device further includes a display configured to show the generated image.

[0020] The fourth aspect relates to an encoding device. The encoding device includes: a memory storing instructions; a processor coupled to the memory, the processor configured to execute the instructions and cause the encoding device to: determine random access points for a video sequence; encode a gradual decoding refresh (GDR) picture into video coding layer (VCL) network abstraction layer (NAL) units having gradual decoding refresh (GDR) network abstraction layer (NAL) units (GDR_NUT) at the random access points for the video sequence; and generate a bitstream containing the video sequence having the GDR picture in the VCL NAL units having the GDR_NUT at the random access points; and a transmitter coupled to the processor, configured to transmit the bitstream to a video decoder.

[0021] The encoding device provides a technique that allows progressive intra-refresh, enabling random access without the need to use intra-random access point (IRAP) pictures. A video coding layer (VCL) network abstraction layer (NAL) unit has a progressive decode refresh (GDR) network abstraction layer (NAL) unit type (GDR_NUT) to indicate that the VCL NAL unit contains a GDR picture. By using a GDR picture instead of an IRAP picture, a smoother, more consistent bitrate may be achieved, for example, due to the size of the GDR picture relative to the size of the IRAP picture, which allows for reduced end-to-end delay (i.e., latency). Thus, the coder / decoder (also known as a "codec") in video coding is improved compared to current codecs. As a practical matter, the improved video coding process provides a better user experience when video is sent, received, and / or viewed.

[0022] In the first implementation of the encoding device according to the fourth aspect itself, the memory stores the bitstream before the transmitter transmits the bitstream to the video decoder.

[0023] A fifth aspect relates to a coding apparatus. The coding apparatus includes: a receiver configured to receive a picture to be encoded or a bitstream to be decoded; a transmitter coupled to the receiver, the transmitter configured to transmit the bitstream to a decoder or to transmit a decoded image to a display; a memory coupled to at least one of the receiver or the transmitter, the memory configured to store instructions; and a processor coupled to the memory, the processor configured to execute instructions stored in the memory to perform any of the methods disclosed herein.

[0024] The coding device provides a technique that allows progressive intra-refresh, enabling random access without the need to use intra-random access point (IRAP) pictures. A video coding layer (VCL) network abstraction layer (NAL) unit has a progressive decode refresh (GDR) network abstraction layer (NAL) unit type (GDR_NUT) to indicate that the VCL NAL unit contains a GDR picture. By using a GDR picture instead of an IRAP picture, a smoother, more consistent bitrate may be achieved, for example, due to the size of the GDR picture relative to the size of the IRAP picture, which allows for reduced end-to-end delay (i.e., latency). Thus, the coder / decoder (also known as a "codec") in video coding is improved compared to current codecs. As a practical matter, the improved video coding process provides a better user experience when video is sent, received, and / or viewed.

[0025] The sixth aspect relates to a system, which includes an encoder and a decoder communicating with the encoder, wherein the encoder or the decoder includes a decoding device, encoding device, or coding device disclosed herein.

[0026] The system provides a technique that allows progressive intra-refresh, enabling random access without the need to use intra-random access point (IRAP) pictures. A Video Coding Layer (VCL) Network Abstraction Layer (NAL) unit has a Progressive Decode Refresh (GDR) Network Abstraction Layer (NAL) unit type (GDR_NUT) to indicate that the VCL NAL unit contains a GDR picture. By using a GDR picture instead of an IRAP picture, for example, a smoother and more consistent bitrate may be achieved due to the size of the GDR picture relative to the size of the IRAP picture, which allows for reduced end-to-end delay (i.e., latency). Thus, the coder / decoder (also known as a "codec") in video coding is improved compared to current codecs. As a practical matter, the improved video coding process provides a better user experience when video is sent, received, and / or viewed.

[0027] The seventh aspect relates to coding means, which include: a receiving means configured to receive a picture to be encoded or a bitstream to be decoded; a transmitting means coupled to the receiving means, the transmitting means configured to transmit the bitstream to a decoding means or to transmit a decoded image to a display means; a storage means coupled to at least one of the receiving means or the transmitting means, the storage means configured to store instructions; and a processing means coupled to the storage means, the processing means configured to execute instructions stored in the storage means to perform any of the methods disclosed herein.

[0028] The coding means provides a technique that allows progressive intra-refresh, enabling random access without the need to use intra-random access point (IRAP) pictures. A video coding layer (VCL) network abstraction layer (NAL) unit has a progressive decode refresh (GDR) network abstraction layer (NAL) unit type (GDR_NUT) to indicate that the VCL NAL unit contains a GDR picture. By using a GDR picture instead of an IRAP picture, for example, a smoother and more consistent bitrate may be achieved due to the size of the GDR picture relative to the size of the IRAP picture, which allows for reduced end-to-end delay (i.e., latency). Thus, the coder / decoder (also known as a "codec") in video coding is improved compared to current codecs. As a practical matter, the improved video coding process provides a better user experience when video is sent, received, and / or viewed. [Brief explanation of the drawing]

[0029] For a more complete understanding of this disclosure, please refer to the following brief description, which will be considered in conjunction with the accompanying drawings and detailed description. Here, similar reference numerals represent similar parts.

[0030] [Figure 1] This is a block diagram illustrating an example coding system that can utilize GDR technology.

[0031] [Figure 2] A block diagram illustrating a video encoder that can implement GDR technology.

[0032] [Figure 3] A block diagram showing an example of a video decoder that can implement GDR technology.

[0033] [Figure 4] This represents the relationship between intra-random access point (IRAP) pictures and the first and subsequent pictures in the decoding and presentation order.

[0034] [Figure 5] This demonstrates a progressive decoding and refreshing technique.

[0035] [Figure 6] This is a schematic diagram illustrating the search for undesirable movement patterns.

[0036] [Figure 7] The present disclosure shows a bitstream illustrating a incremental decode-refresh technique according to one embodiment of this disclosure.

[0037] [Figure 8] This is an embodiment of a method for decoding a coded video bitstream.

[0038] [Figure 9] This is an embodiment of a method for encoding a video bitstream.

[0039] [Figure 10] This is a schematic diagram of a video coding device.

[0040] [Figure 11] This is a schematic diagram of an embodiment of the coding means. [Modes for carrying out the invention]

[0041] Figure 1 is a block diagram illustrating an exemplary coding system 10 that can utilize the video coding technology described herein. As shown in Figure 1, the coding system 10 includes a source device 12 that provides encoded video data to be later decoded by a destination device 14. Specifically, the source device 12 may provide the video data to the destination device 14 via a computer-readable medium 16. The source device 12 and destination device 14 may include any of the wide range of devices, including desktop computers, notebook (e.g., laptop) computers, tablet computers, set-top boxes, telephone terminals such as so-called "smart" phones, so-called "smart" pads, televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some cases, the source device 12 and destination device 14 may be equipped for wireless communication.

[0042] The destination device 14 may receive the encoded video data to be decoded via a computer-readable medium 16. The computer-readable medium 16 may include any type of medium or device that can move the encoded video data from the source device 12 to the destination device 14. For example, the computer-readable medium 16 may include a communication medium that allows the source device 12 to directly transmit the encoded video data to the destination device 14 in real time. The encoded video data may be modulated according to a communication standard, such as a wireless communication protocol, and transmitted to the destination device 14. The communication medium may include any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include routers, switches, base stations, or other equipment that may be useful to facilitate communication from the source device 12 to the destination device 14.

[0043] In some examples, the encoded data may be output to a storage device via the output interface 22. Similarly, the encoded data may be accessed from the storage device via the input interface. The storage device may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, Blu-ray disc, digital video disc (DVD), compact disc read-only memory (CD-ROM), flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In a further example, the storage device may correspond to a file server or another intermediate storage device that can store the encoded video generated by the source device 12. The destination device 14 can access the stored video data from the storage device via streaming or download. The file server may be any type of server that can store encoded video data and send that encoded video data to the destination device 14. Exemplary file servers include a web server (e.g., for a website), a File Transfer Protocol (FTP) server, a network-attached storage (NAS) device, or a local disk drive. The destination device 14 can access the encoded video data through any standard data connection, including an internet connection. This may include wireless channels (e.g., Wi-Fi connection), wired connections (e.g., digital subscriber line (DSL), cable modem, etc.), or a combination of both suitable for accessing encoded video data stored on a file server. Transmission of the encoded video data from the storage device may be streaming transmission, download transmission, or a combination thereof.

[0044] The technology of this disclosure is not necessarily limited to wireless applications or settings. The technology may be applied to video coding supporting any of a variety of multimedia applications, such as wireless television broadcasting, cable television transmission, satellite television transmission, internet streaming video transmission, such as Dynamic Adaptive Streaming (DASH) over HTTP, encoded digital video on a data storage medium, decoding of digital video stored on a data storage medium, or other applications. In some examples, the coding system 10 may be configured to support one-way or two-way video transmission to support applications such as video streaming, video playback, video broadcasting, and / or video phone calls.

[0045] In the example shown in Figure 1, the source device 12 includes a video source 18, a video encoder 20, and an output interface 22. The destination device 14 includes an input interface 28, a video decoder 30, and a display device 32. According to this disclosure, the video encoder 20 of the source device 12 and / or the video decoder 30 of the destination device 14 may be configured to apply techniques for video coding. In other examples, the source device and destination device may include other components or arrangements. For example, the source device 12 may receive video data from an external video source, such as an external camera. Similarly, the destination device 14 may interface with an external display device rather than including an integrated display device.

[0046] The coding system 10 shown in Figure 1 is merely an example. The technique for video coding can be performed by any digital video encoding and / or decoding device. The technique of this disclosure is generally performed by a video coding device, but the technique may also be performed by a video encoder / decoder, typically called a “codec”. Furthermore, the technique of this disclosure may also be performed by a video preprocessor. The video encoder and / or decoder may be a graphics processing unit (GPU) or a similar device.

[0047] The source device 12 and destination device 14 are merely examples of such coding devices, where the source device 12 generates coded video data for transmission to the destination device 14. In some examples, the source device 12 and destination device 14 can operate in a substantially symmetrical manner, so that each of the source device 12 and destination device 14 includes video encoding and decoding components. Thus, the coding system 10 can support one-way or two-way video transmission between video devices 12, 14 for, for example, video streaming, video playback, video broadcasting, or video phone calls.

[0048] The video source 18 of the source device 12 may include a video capture device such as a video camera, a video archive containing previously captured video, and / or a video feed interface for receiving video from a video content provider. As a further alternative, the video source 18 may generate computer graphics-based data as source video, or a combination of live video, archived video, and computer-generated video.

[0049] In some cases, when the video source 18 is a video camera, the source device 12 and the destination device 14 can form a so-called camera phone or video phone. However, as stated above, the technology described herein may be applicable to video coding in general and may be applicable to wireless and / or wired applications. In each case, captured, pre-captured, or computer-generated video may be encoded by the video encoder 20. The encoded video information may then be output to a computer-readable medium 16 via the output interface 22.

[0050] The computer-readable medium 16 may include temporary media such as wireless broadcast or wired network transmission, or storage media such as hard disks, flash drives, compact discs, digital video discs, Blu-ray discs, or other computer-readable media. In some examples, a network server (not shown) may receive encoded video data from a source device 12 and provide the encoded video data to a destination device 14, for example, via network transmission. Similarly, a computer in a media manufacturing facility, such as a disc stamping facility, may receive encoded video data from a source device 12 and produce discs containing the encoded video data. Thus, the computer-readable medium 16 may be understood to include one or more computer-readable media in various forms in various examples.

[0051] The input interface 28 of the destination device 14 receives information from the computer-readable medium 16. The information on the computer-readable medium 16 may include syntactic information defined by the video encoder 20, which is also used by the video decoder 30, and includes syntactic elements that describe the features and / or processing of blocks and / or other coding units, such as a group of pictures (GOP). The display device 32 displays the decoded video data to the user and may include any of the various display devices, such as a cathode ray tube (CRT), liquid crystal display (LCD), plasma display, organic light-emitting diode (OLED) display, or other types of display devices.

[0052] The video encoder 20 and video decoder 30 may operate in accordance with a video coding standard such as the High Efficiency Video Coding (HEVC) standard currently under development, or they may conform to the HEVC Test Model (HM). Alternatively, the video encoder 20 and video decoder 30 may operate in accordance with other proprietary or industry standards, such as the International Telecommunication Union Telecommunication Standardization Sector (ITU-T) H.264 standard, also known as the Motion Picture Expert Group (MPEG)-4, Part 10, Advanced Video Coding (AVC), or extensions of such standards. However, the technology of this disclosure is not limited to any particular coding standard. Other examples of video coding standards include MPEG-2 and ITU-T H.263. Although not shown in Figure 1, in some respects, the video encoder 20 and video decoder 30 may be integrated with an audio encoder and decoder, respectively, and may include a suitable multiplexer-demultiplexer (MUX-DEMUX) unit or other hardware and software to handle the encoding of both audio and video in a common data stream or separate data streams. Where applicable, the MUX-DEMUX unit may conform to the ITU H.223 Multiplexer Protocol or other protocols such as the User Datagram Protocol (UDP).

[0053] The video encoder 20 and video decoder 30 may each be implemented as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. Where the technology is partially implemented in software, the device may store instructions for the software on a suitable non-temporary computer-readable medium and execute those instructions in hardware using one or more processors to perform the technology of this disclosure. Each of the video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, any of which may be integrated as part of a combined encoder / decoder (codec) within the respective device. A device including the video encoder 20 and / or video decoder 30 may include an integrated circuit, a microprocessor, and / or a wireless communication device such as a cellular telephone.

[0054] Figure 2 is a block diagram showing an example of a video encoder 20 capable of implementing video coding techniques. The video encoder 20 can perform intracoding and intercoding of video blocks within a video slice. Intracoding relies on spatial prediction to reduce or eliminate spatial redundancy in video within a given video frame or picture. Intercoding relies on temporal prediction to reduce or eliminate temporal redundancy in video within adjacent frames or pictures of a video sequence. Intra-mode (I-mode) can refer to any of several spatial-based coding modes. Inter-mode, such as one-way (also known as single prediction) prediction (P-mode) or bi-prediction (also known as bi-prediction) (B-mode), can refer to any of several temporal-based coding modes.

[0055] As shown in Figure 2, the video encoder 20 receives the current video block in the video frame to be encoded. In the example in Figure 2, the video encoder 20 includes a mode selection unit 40, a reference frame memory 64, an adder 50, a transformation unit 52, a quantization unit 54, and an entropy coding unit 56. The mode selection unit 40 further includes a motion compensation unit 44, a motion estimation unit 42, an intra-prediction (also known as internal prediction) unit 46, and a splitting unit 48. For video block reconstruction, the video encoder 20 also includes an inverse quantization unit 58, an inverse transformation unit 60, and an adder 62. A deblocking filter (not shown in Figure 2) may also be included to filter block boundaries to remove blocky artifacts from the reconstructed video. If desired, the deblocking filter typically filters the output of the adder 62. In addition to the deblocking filter, additional filters (in-loop or after-loop) may also be used. Such a filter is not shown for brevity, but if desired, the output of adder 50 may be filtered (as an in-loop filter).

[0056] During the encoding process, the video encoder 20 receives video frames or slices to be coded. The frames or slices may be divided into multiple video blocks. The motion estimation unit 42 and the motion compensation unit 44 perform inter-predictive coding of the received video blocks for one or more blocks within one or more reference frames to provide temporal predictions. The intra-predictive unit 46 may, alternatively, perform intra-predictive coding of the received video blocks for one or more neighboring blocks within the same frame or slice as the block to be coded to provide spatial predictions. The video encoder 20 may perform multiple coding passes, for example, to select an appropriate coding mode for each block of video data.

[0057] Furthermore, the splitting unit 48 may divide blocks of video data into subblocks based on an evaluation of a previous splitting method in a previous coding pass. For example, the splitting unit 48 may first divide a frame or slice into the largest coding unit (LCU), and then divide each LCU into subcoding units (subCUs) based on rate-distortion analysis (e.g., rate-distortion optimization). The mode selection unit 40 can further generate a quadtree data structure that shows the division of LCUs into subCUs. The leaf nodes CU of the quadtree may include one or more prediction units (PUs) and one or more transformation units (TUs).

[0058] This disclosure uses the term “block” to refer to any CU, PU, ​​or TU in the context of HEVC, or any similar data structure in the context of other standards (e.g., macroblocks and their subblocks in H.264 / AVC). A CU includes a coding node, PUs, and TUs associated with the coding node. The size of a CU corresponds to the size of the coding node and is square. The size of a CU ranges from 8x8 pixels to the size of a tree block of up to 64x64 pixels or more. Each CU may contain one or more PUs and one or more TUs. Syntactic data associated with a CU may, for example, describe the partitioning of the CU into one or more PUs. The partitioning mode may differ depending on whether the CU is skipped or direct mode encoded, intra-predictive mode encoded, or inter-predictive (also known as mutual predictive) mode encoded. PUs may be partitioned into a non-square shape. Syntactic data associated with a CU may, for example, describe the partitioning of the CU into one or more TUs according to a quadtree. TUs may be square or non-square (e.g., rectangular) in shape.

[0059] The mode selection unit 40 may select one of the coding modes, for example, intra or inter, based on the error result, and provides the resulting intra or intercoded block to the adder 50 which generates residual block data, and to the adder 62 which reconstructs the encoded block for use as a reference frame. The mode selection unit 40 also provides syntactic elements such as motion vectors, intra-mode indicators, division information, and other such syntactic information to the entropy coding unit 56.

[0060] The motion estimation unit 42 and the motion compensation unit 44 may be highly integrated, but are shown separately for conceptual purposes. The motion estimation performed by the motion estimation unit 42 is a process that generates motion vectors to estimate the motion of a video block. The motion vectors may, for example, represent the displacement of the PU of a video block in the current video frame or picture relative to a predicted block (or other coding unit) in a reference frame relative to the current block (or other coding unit) being coded in the current frame. The predicted block is a block that is found to closely match the coded block with respect to pixel differences, and the match may be determined by the sum of absolute differences (SAD), the sum of squared differences (SSD), or other difference metrics. In some examples, the video encoder 20 can calculate integer-fine pixel position values ​​of a reference picture stored in the reference frame memory 64. For example, the video encoder 20 can interpolate values ​​of a quarter-pixel position, an eighth-picture position, or other fractional picture position of the reference picture. Therefore, the motion estimation unit 42 can perform motion search for all pixel positions and fractional pixel positions, and output motion vectors with fractional pixel accuracy.

[0061] The motion estimation unit 42 calculates motion vectors for the PU of a video block in the interconnected slice by comparing the position of the PU with the position of a predicted block in a reference picture. The reference picture may be selected from a first reference picture list (list 0) or a second reference picture list (list 1), each of which identifies one or more reference pictures stored in the reference frame memory 64. The motion estimation unit 42 sends the calculated motion vectors to the entropy coding unit 56 and the motion compensation unit 44.

[0062] Motion compensation performed by the motion compensation unit 44 may include fetching or generating a predicted block based on a motion vector determined by the motion estimation unit 42. Again, the motion estimation unit 42 and the motion compensation unit 44 may be functionally integrated in some examples. Upon receiving a motion vector for the PU of the current video block, the motion compensation unit 44 can locate the predicted block pointed to by the motion vector in one of the reference picture lists. The adder 50 forms a residual video block by subtracting the pixel values ​​of the predicted block from the pixel values ​​of the current video block to be coded to form a pixel difference value, as described later. Generally, the motion estimation unit 42 performs motion estimation for the lumen component, and the motion compensation unit 44 uses the motion vector calculated based on the lumen component for both the chroma and lumen components. The mode selection unit 40 can also generate syntactic elements related to video blocks and video slices for use by the video decoder 30 when decoding the video block of the video slice.

[0063] The intra-prediction unit 46 may intra-predict the current block as an alternative to the inter-prediction performed by the motion estimation unit 42 and motion compensation unit 44 as described above. Specifically, the intra-prediction unit 46 may determine the intra-prediction mode to use to encode the current block. In some examples, the intra-prediction unit 46 may encode the current block using different intra-prediction modes, for example, between separate encoding passes, and the intra-prediction unit 46 (or, in some examples, the mode selection unit 40) can select an appropriate intra-prediction mode to use from the tested modes.

[0064] For example, the intra-prediction unit 46 can calculate rate-distortion values ​​for various tested intra-prediction modes using rate-distortion analysis and select the intra-prediction mode with the best rate-distortion characteristics among the tested modes. Rate-distortion analysis generally determines the amount of distortion (or error) between an encoded block and the original unencoded block encoded to produce the encoded block, as well as the bit rate (i.e., number of bits) used to produce the encoded block. The intra-prediction unit 46 can calculate a ratio from the distortion and rate for various encoded blocks to determine the intra-prediction mode that exhibits the best rate-distortion value for that block.

[0065] In addition, the intra-prediction unit 46 may be configured to code depth blocks of the depth map using a depth modeling mode (DMM). The mode selection unit 40 can determine, for example, using rate-distortion optimization (RDO), whether an available DMM mode yields better coding results than the intra-prediction mode and other DMM modes. Data for the texture image corresponding to the depth map may be stored in the reference frame memory 64. The motion estimation unit 42 and the motion compensation unit 44 may be configured to inter-predict depth blocks of the depth map.

[0066] After selecting an intra-prediction mode for a block (for example, one of the conventional intra-prediction modes or DMM modes), the intra-prediction unit 46 can provide the entropy coding unit 56 with information indicating the selected intra-prediction mode for that block. The entropy coding unit 56 can encode the information indicating the selected intra-prediction mode. The video encoder 20 may include configuration data in the transmit bitstream. The configuration data may include multiple intra-prediction mode index tables and multiple modified intra-prediction mode index tables (also called codeword mapping tables), definitions of encoding contexts for various blocks, and instructions for the most likely intra-prediction mode, intra-prediction mode index table, and modified intra-prediction mode index table to use for each context.

[0067] The video encoder 20 forms a residual video block by subtracting predicted data from the mode selection unit 40 from the original video block to be coded. The adder 50 represents one or more components that perform this subtraction operation.

[0068] The transformation processing unit 52 applies a transformation, such as a discrete cosine transform (DCT) or a conceptually similar transformation, to the residual block to generate a video block containing residual transformation coefficient values. The transformation processing unit 52 may also perform other transformations conceptually similar to the DCT. Wavelet transforms, integer transforms, subband transforms, or other types of transformations can also be used.

[0069] The conversion processing unit 52 applies the conversion to the residual block to generate a block of residual conversion coefficients. The conversion may convert the residual information from the pixel value domain to the conversion domain, for example, the frequency domain. The conversion processing unit 52 may send the resulting conversion coefficients to the quantization unit 54. The quantization unit 54 quantizes the conversion coefficients to further reduce the bit rate. The quantization process may reduce the bit depth associated with some or all of the coefficients. The degree of quantization may be modified by adjusting the quantization parameters. In some examples, the quantization unit 54 may then perform a scan of the matrix containing the quantized conversion coefficients. Alternatively, the entropy coding unit 56 may perform the scan.

[0070] After quantization, the entropy coding unit 56 entropy codes the quantized transformation coefficients. For example, the entropy coding unit 56 may perform context-adaptive variable-length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), stochastic interval partitioning entropy (PIPE) coding, or other entropy coding techniques. In the case of context-based entropy coding, the context may be based on neighborhood blocks. Following entropy coding by the entropy coding unit 56, the encoded bitstream may be transmitted to another device (e.g., a video decoder 30) or archived for later transmission or retrieval.

[0071] The inverse quantization unit 58 and the inverse transform unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct the residual blocks in the pixel region, for example, for later use as reference blocks. The motion compensation unit 44 can compute a reference block by adding the residual blocks to a predicted block of one of the frames in the reference frame memory 64. The motion compensation unit 44 may also apply one or more interpolation filters to the reconstructed residual blocks to compute pixel values ​​finer than integers for use in motion estimation. The adder 62 adds the reconstructed residual blocks to the motion-compensated predicted blocks generated by the motion compensation unit 44 to produce a reconstructed video block for storage in the reference frame memory 64. The reconstructed video block may be used by the motion estimation unit 42 and the motion compensation unit 44 as a reference block for intercoding blocks in subsequent video frames.

[0072] Figure 3 is a block diagram showing an example of a video decoder 30 that can implement video coding technology. In the example in Figure 3, the video decoder 30 includes an entropy decoding unit 70, a motion compensation unit 72, an intra-prediction unit 74, an inverse quantization unit 76, an inverse transform unit 78, a reference frame memory 82, and an adder 80. In some examples, the video decoder 30 can perform a decoding path that is roughly the reverse of the encoding path described with respect to the video encoder 20 (Figure 2). The motion compensation unit 72 may generate prediction data based on motion vectors received from the entropy decoding unit 70, while the intra-prediction unit 74 may generate prediction data based on an intra-prediction mode indicator received from the entropy decoding unit 70.

[0073] During the decoding process, the video decoder 30 receives an encoded video bitstream from the video encoder 20, representing the video blocks and associated syntactic elements of the encoded video slice. The entropy decoding unit 70 of the video decoder 30 decodes the bitstream and generates quantized coefficients, motion vectors or intra-predictive mode indicators, and other syntactic elements. The entropy decoding unit 70 transfers the motion vectors and other syntactic elements to the motion compensation unit 72. The video decoder 30 may receive syntactic elements at the video slice level and / or video block level.

[0074] When a video slice is coded as an intra-coded (I) slice, the intra-prediction unit 74 may generate prediction data for the video block of the current video slice based on the signaled intra-prediction mode and data from a block decoded before the current frame or picture. When a video frame is coded as an inter-coded (e.g., B, P, or GPB) slice, the motion compensation unit 72 generates prediction blocks for the video block of the current video slice based on motion vectors and other syntactic elements received from the entropy decoding unit 70. The prediction blocks may be generated from one of the reference pictures in one of the reference picture lists. The video decoder 30 can construct the reference frame lists, List 0 and List 1, using default construction techniques based on reference pictures stored in the reference frame memory 82.

[0075] The motion compensation unit 72 determines prediction information for the video blocks of the current video slice by parsing motion vectors and other syntactic elements, and uses this prediction information to generate prediction blocks for the current video block being decoded. For example, using some of the received syntactic elements, the motion compensation unit 72 determines the prediction mode used to code the video blocks of the video slice (e.g., intra-prediction or inter-prediction), the inter-prediction slice type (e.g., B-slice, P-slice, or GPB-slice), construction information for one or more of the reference picture lists for the slice, motion vectors for each inter-encoded video block of the slice, the inter-prediction status for each intercoded video block of the slice, and other information for decoding the video blocks in the current video slice.

[0076] The motion compensation unit 72 may also perform interpolation based on an interpolation filter. The motion compensation unit 72 can use the interpolation filter used by the video encoder 20 during video block encoding to calculate interpolated values ​​for pixels finer than an integer in the reference block. In this case, the motion compensation unit 72 may determine the interpolation filter used by the video encoder 20 from the received syntactic elements and use the interpolation filter to generate the prediction block.

[0077] Data for the texture image corresponding to the depth map may be stored in the reference frame memory 82. The motion compensation unit 72 may be configured to interpret the depth blocks of the depth map.

[0078] With the above in mind, video compression techniques perform spatial (intra-picture) prediction and / or temporal (inter-picture) prediction to reduce or eliminate redundancy inherent in video sequences. For block-based video coding, a video slice (i.e., a video picture or part of a video picture) may be divided into video blocks, such video blocks may also be referred to as tree blocks, coding tree blocks (CTBs), coding tree units (CTUs), coding units (CUs), and / or coding nodes. A video block in an intra-coded (I) slice of a picture is encoded using spatial prediction with respect to a reference sample in a neighboring block within the same picture. A video block in an intercoded (P or B) slice of a picture may use spatial prediction with respect to a reference sample in a neighboring block within the same picture, or temporal prediction with respect to a reference sample in another reference picture. A picture may be referred to as a frame, and a reference picture may be referred to as a reference frame.

[0079] Spatial or temporal predictions provide predicted blocks for the blocks to be coded. Residual data represents the pixel difference between the original blocks to be coded and the predicted blocks. Intercoded blocks are coded according to motion vectors pointing to the reference sample blocks forming the predicted blocks and residual data showing the difference between the blocks to be coded and the predicted blocks. Intracoded blocks are coded according to the intracoding mode and residual data. For further compression, the residual data may be transformed from a pixel domain to a transformation domain to give residual transformation coefficients, which may then be quantized. The quantized transformation coefficients may initially be placed in a two-dimensional array and scanned to generate a one-dimensional vector of transformation coefficients, and entropy coding may be applied to achieve further compression.

[0080] Image and video compression has experienced rapid growth, giving rise to various coding standards. Such video coding standards include Advanced Video Coding (AVC), also known as ITU-T H.261, ISO / IEC MPEG-1 Part 2, ITU-T H.262 or ISO / IEC MPEG-2 Part 2, ITU-T H.263, ISO / IEC MPEG-4 Part 2, ITU-T H.264 or ISO / IEC MPEG-4 Part 10, and High Efficiency Video Coding (HEVC), also known as ITU-T H.265 or MPEG-H Part 2. AVC includes extensions such as Scalable Video Coding (SVC), Multiview Video Coding (MVC), Multiview Video Coding + Depth (MVC+D), and 3D AVC (3D-AVC). HEVC includes extensions such as Scalable HEVC (SHVC), Multiview HEVC (MV-HEVC), and 3D HEVC (3D-HEVC).

[0081] There is also a new video coding standard called Versatile Video Coding (VVC) being developed by the Joint Video Expert Team (JVET) of ITU-T and ISO / IEC. While there are several working drafts of the VVC standard, here we will refer specifically to one working draft (WD) of VVC, namely B. Bross, J. Chen, and S. Liu, "Versatile Video Coding (Draft 4)," JVET-M1001-v5, 13th JVET Meeting, January 2019 (VVC Draft 4).

[0082] The description of the technology disclosed herein is based on Versatile Video Coding (VVC), a video coding standard under development by the Joint Video Expert Team (JVET) of the ITU-T and ISO / IEC. However, this technology is also applicable to other video codec specifications.

[0083] Figure 4 is a representation 400 of the relationship between IRAP pictures 402 and leading pictures 404 and subsequent pictures 406 in the decoding order 408 and presentation order 410. In some embodiments, IRAP pictures 402 are called clean random access (CRA) pictures or instantaneous decoder refresh (IDR) pictures with random access decodable leading (RADL) pictures. In HEVC, IDR pictures, CRA pictures, and Broken Link Access (BLA) pictures are all considered IRAP pictures 402. For VVC, it was agreed at the 12th JVET meeting in October 2018 that IRAP pictures may include both IDR and CRA pictures.

[0084] As shown in Figure 4, the leading picture 404 (for example, pictures 2 and 3) follows IRAP picture 402 in the decoding order 408, but precedes IRAP picture 402 in the presentation order 410. The succeeding picture 406 follows IRAP picture 402 in both the decoding order 408 and the presentation order 410. Although two leading pictures 404 and one succeeding picture 406 are depicted in Figure 4, those skilled in the art will understand that in practical applications, more or fewer leading pictures 404 and / or succeeding pictures 406 may exist in the decoding order 408 and the presentation order 410.

[0085] In Figure 4, the first picture 404 is divided into two types: random access skipped lading (RASL) and RADL. If decoding begins with IRAP picture 402 (e.g., picture 1), the RADL picture (e.g., picture 3) can be properly decoded, but the RASL picture (e.g., picture 2) cannot. Therefore, the RASL picture is discarded. In light of the distinction between RADL and RASL pictures, the type of the first picture associated with an IRAP picture should be identified as either RADL or RASL for efficient and proper coding. In HEVC, if both RASL and RADL pictures exist, for RASL and RADL pictures associated with the same IRAP picture, the RASL picture is constrained to precede the RADL picture in presentation order 410.

[0086] The IRAP picture 402 provides two important functions / benefits. First, the presence of the IRAP picture 402 indicates that the decoding process can start from that picture. This function allows for random access functionality, where the decoding process starts at that location in the bitstream, rather than necessarily at the beginning of the bitstream, as long as the IRAP picture 402 is present in that location. Second, the presence of the IRAP picture 402 refreshes the decoding process, thereby coding pictures that begin with the IRAP picture 402, excluding RASL pictures, without referencing the previous picture. As a result, the presence of the IRAP picture 402 in the bitstream prevents any errors that might occur during the decoding of pictures coded before the IRAP picture 402 from propagating to the IRAP picture 402 and the pictures that follow it in the decoding order 408.

[0087] IRAP picture 402 provides important functionality but comes with a penalty to compression efficiency. The presence of IRAP picture 402 causes a surge in bitrate. This penalty to compression efficiency is due to two reasons. Firstly, since IRAP picture 402 is an intra-predicted picture, the picture itself requires relatively more bits to represent when compared to other pictures that are inter-predicted pictures (e.g., leading picture 404, following picture 406). Secondly, the presence of IRAP picture 402 disrupts the temporal prediction (this is because the decoder refreshes the decoding process, one of which is to remove the previous reference picture in the decode picture buffer (DPB)). Therefore, with IRAP picture 402, coding the picture that follows IRAP picture 402 in decoding order 408 is less efficient (i.e., requires more bits to represent) because there are fewer reference pictures for inter-predicted coding.

[0088] Among the picture types considered to be IRAP Picture 402, the IDR picture in HEVC has different signal transmission and derivation compared to other picture types. Some of the differences are as follows:

[0089] For signal transmission and derivation of the Picture Sequence Count (POC) value of an IDR picture, the most significant bit (MSB) portion of the POC is not derived from the previous key picture, but is simply set to equal to 0.

[0090] Regarding the signaling information required for reference picture management, the slice header of an IDR picture does not contain any information that needs to be signaled to assist in reference picture management. For other picture types (i.e., CRA, Trailing, Temporal Sub-layer Access, TSA, etc.), information such as the reference picture set (RPS) or other forms of similar information (e.g., a reference picture list) described below is required for the reference picture marking process (i.e., the process of determining the state of reference pictures in the decode picture buffer (DPB), whether they are used for reference or not). However, for IDR pictures, such information does not need to be signaled because the presence of the IDR indicates that the decoding process should simply mark all reference pictures in the DPB as not being used for reference.

[0091] In HEVC and VVC, IRAP picture 402 and leading picture 404 may each be contained within a single Network Abstraction Layer (NAL) unit. A collection of NAL units is sometimes called an access unit. IRAP picture 402 and leading picture 404 are given different NAL unit types and are therefore easily identifiable by system-level applications. For example, a video splicer needs to understand the coded picture type without having to understand the excessive detail of syntactic elements in the coded bitstream, and in particular needs to distinguish leading picture 404 from subsequent picture 406, including distinguishing IRAP picture 402 from non-IRAP pictures and determining RASL and RADL pictures. Subsequent picture 406 is associated with IRAP picture 402 and is the picture that comes after IRAP picture 402 in presentation order 410. A picture may come after a particular IRAP picture 402 in decoding order 408 and may come before any other IRAP picture 402 in decoding order 408. To achieve this, assigning their own NAL unit types to IRAP picture 402 and leading picture 404 would be helpful for such applications.

[0092] For HEVC, the NAL unit type of IRAP pictures includes the following: BLA with leading picture (BLA_W_LP): A NAL unit of a Broken Link Access (BLA) picture, where one or more leading pictures may follow in the decoding order. BLA with RADL (BLA_W_RADL): A NAL unit of a BLA picture that may be followed by one or more RADL pictures in the decoding order, but not by RASL pictures. BLA without a leading picture (BLA_N_LP): A NAL unit of a BLA picture where the leading picture does not follow in the decoding order. IDR with RADL (IDR_W_RADL): NAL unit of IDR picture where one or more RADL pictures may follow in the decoding order, but no RASL pictures follow. IDR without a leading picture (IDR_N_LP): NAL units of IDR pictures where the leading picture does not follow the decoded picture. CRA: A NAL unit of a clean random access (CRA) picture that may be followed by a leading picture (i.e., a RASL picture or a RADL picture, or both). RADL: NAL units for RADL pictures RASL: NAL unit for RASL pictures.

[0093] For VVC, the NAL unit types for IRAP picture 402 and leading picture 404 are as follows: IDR with RADL (IDR_W_RADL): NAL unit of IDR picture where one or more RADL pictures may follow in the decoding order, but no RASL pictures follow. IDR without a leading picture (IDR_N_LP): NAL units of IDR pictures where the leading picture does not follow the decoded picture. CRA: A NAL unit of a clean random access (CRA) picture that may be followed by a leading picture (i.e., a RASL picture or a RADL picture, or both). RADL: NAL units for RADL pictures RASL: NAL unit for RASL pictures.

[0094] Progressive intra-refresh / gradual decode-refresh will be discussed below.

[0095] For low-latency applications, it is preferable to avoid coding pictures as IRAP pictures (e.g., IRAP picture 402) because their bitrate requirements are relatively higher compared to non-IRAP (i.e., P / B) pictures, resulting in greater latency / delay. However, completely avoiding the use of IRAP may not be possible for all low-latency applications. For example, conversational applications such as multi-party teleconferencing need to provide regular points where new users can join the teleconferencing.

[0096] To provide access to a bitstream that allows new users to join a multi-party remote conferencing application, one possible strategy is to use Progressive Intra Refresh (PIR) technology instead of using IRAP pictures to avoid peaks in the bitrate. PIR is sometimes also called gradual decoding refresh (GDR). The terms PIR and GDR may be used interchangeably in this disclosure.

[0097] Figure 5 shows a progressive decode refresh (GDR) technique 500. As shown, the GDR technique 500 is drawn using a GDR picture 502, one or more subsequent pictures 504, and a recovery point picture 506 in a coded video sequence 508 of a bitstream. In one embodiment, the GDR picture 502, the subsequent pictures 504, and the recovery point picture 506 can define a GDR period in the CVS 508. The CVS 508 is a series of pictures (or a portion thereof) beginning with the GDR picture 502 and includes all pictures (or a portion thereof) up to the next GDR picture (but not including the next GDR picture), or up to the end of the bitstream. A GDR period is a series of pictures beginning with the GDR picture 502 and includes all pictures up to the recovery point picture 506, including the recovery point picture 506.

[0098] As shown in Figure 5, the GDR technique 500 or principle operates on a series of pictures, beginning with GDR picture 502 and ending with recovery point picture 506. GDR picture 502 includes a refresh / clean region 510 containing blocks coded using intra-prediction (i.e., intra-prediction blocks) and an unrefreshed / dirty region 512 containing blocks coded using inter-prediction (i.e., inter-prediction blocks).

[0099] A subsequent picture 504 immediately adjacent to the GDR picture 502 includes a refresh / clean region 510 having a first portion 510A coded using intra-prediction and a second portion 510B coded using inter-prediction. The second portion 510B is coded, for example, by referencing the refresh / clean region 510 of the preceding picture within the GDR period of CVS 508. As shown in the figure, the refresh / clean region 510 of the subsequent picture 504 expands as the coding process moves or progresses in a consistent direction (e.g., from left to right), correspondingly contracting the unrefreshed / dirty region 512. Finally, a recovery point picture 506 containing only the refresh / clean region 510 is obtained from the coding process. Notably, as will be discussed further below, the second portion 510B of the refresh / clean region 510 coded as an inter-prediction block may simply refer to the refresh / clean region 510 in the reference picture.

[0100] In HEVC, the GDR technique 500 shown in Figure 5 was non-normatively supported using Recovery Point Supplemental Improvement (SEI) messages and Region Refresh Information SEI messages. These two SEI messages do not define how GDR is performed. Rather, they simply provide a mechanism for indicating the first and last pictures in a GDR period (i.e., provided by the Recovery Point SEI message) and the regions to be refreshed (i.e., provided by the Region Refresh Information SEI message).

[0101] In practice, GDR technique 500 is implemented by using two techniques together. These two techniques are constraint intra prediction (CIP) and encoder constraints on motion vectors. CIP can be used for the purposes of GDR, in particular, to code areas that are coded only as intra-predicted blocks (e.g., the first portion 510A of the refresh / clean area 510). This is because CIP allows areas that do not use samples from unrefreshed areas (e.g., the unrefreshed / dirty area 512) to be used for reference. However, the use of CIP causes a severe degradation in coding performance because the constraints on intra-blocks must apply not only to intra-blocks in the refreshed area but also to all intra-blocks in the picture. Encoder constraints on motion vectors restrict the encoder from using any samples in the reference picture that are located outside the refresh area. Such constraints cause suboptimal motion search.

[0102] Figure 6 is a schematic diagram showing an undesirable motion search 600 when encoder constraints are used to support GDR. As shown in the figure, the motion search 600 draws the current picture 602 and the reference picture 604. The current picture 602 and the reference picture 604 contain a refreshed region 606 coded with intra-prediction, a refreshed region 608 coded with inter-prediction, and an unrefreshed region 608, respectively. The refreshed region 604, the refreshed region 606, and the unrefreshed region 608 are similar to the first portion 510A of the refresh / clean region 510, the second portion 510B of the refresh / clean region 510, and the unrefreshed / dirty region 512 in Figure 5.

[0103] During the motion search process, the encoder is constrained or prevented from selecting any motion vector 610 that leads to any of the samples in reference block 612 located outside the refresh region 606. This occurs even if reference block 612 provides the best rate-distortion cost criterion when predicting the current block 614 in the current picture 602. Thus, Figure 6 illustrates the reason for the inoptimality in motion search 600 when encoder constraints are used to support GDR.

[0104] JVET contributions JVET-K0212 and JVET-L0160 describe an implementation of GDR based on the use of CIP and an encoder-constrained approach. The implementation can be summarized as follows: an intra-prediction mode is enforced for coding units on a column basis; constrained intra-prediction is enabled to ensure reconstruction of intra-CUs; motion vectors are constrained to point within the refresh area, taking into account an additional margin (e.g., 6 pixels) to avoid error diffusion about filters; and previous reference pictures are removed when re-looping intra-columns.

[0105] JVET's contribution, JVET-M0529, proposed a normative method for indicating that a picture is the first and last in a GDR period. The proposed idea works as follows:

[0106] A new NAL unit with a NAL unit type recovery point indication is defined as a non-video coding layer (VCL) NAL unit. The NAL unit payload contains syntactic elements that specify information that can be used to derive the POC value of the last picture in the GDR period. An access unit containing a non-VCL NAL unit with a type recovery point indication is called a recovery point begin (RBP) access unit (AU), and pictures within an RBP access unit are called RBP pictures. The decoding process can be initiated from an RBP AU. When decoding is initiated from an RBP AU, all pictures in the GDR period except the last picture are not output.

[0107] Some of the problems with existing GDR designs will be discussed.

[0108] Existing designs / approaches to support GDR have at least the following problems:

[0109] The normative method for defining GDR in JVET-M0529 has the following problems: The proposed method does not describe how GDR is performed. Instead, the proposed method only provides some signaling to indicate the first and last pictures in a GDR period. A new non-VCL NAL unit is required to indicate the first and last pictures in a GDR period. This is redundant because the information contained in the recovery point indication (RPI) NAL unit can simply be included in the tile group header of the first picture in a GDR period. Furthermore, the proposed method cannot describe which areas in a picture are refreshed and which are not refreshed during a GDR period.

[0110] The GDR approach described in JVET-K0212 and JVET-L0160 has the following problems. Firstly, there is the use of CIP. To prevent any samples from the unrefreshed region from being used for spatial reference, it is necessary to code the refresh region by intra-prediction with several constraints. When CIP is used, coding is picture-based, which means that all intra-blocks in the picture must also be coded as CIP intra-blocks. This results in performance degradation. Furthermore, if the samples of the reference block related to the motion vector are not entirely within the refresh region in the reference picture, the encoder is prevented from selecting the best motion vector by using encoder constraints that limit the motion search. Also, the refresh region coded by intra-prediction only is not CTU size. Instead, the refresh region can be smaller than the CTU size, and may even be the minimum CU size. This unnecessarily complicates the implementation as it may require block-level instructions.

[0111] Disclosed herein is a technique for supporting progressive decode refresh (GDR) in video coding. The disclosed technique allows progressive intra-refresh to enable random access without the need to use intra-random access point (IRAP) pictures. As will be further described below, a video coding layer (VCL) network abstraction layer (NAL) unit has a progressive decode refresh (GDR) network abstraction layer (NAL) unit type (GDR_NUT) to indicate that the VCL NAL unit contains a GDR picture. That is, GDR_NUT directly indicates to the video decoder that the VCL NAL unit having GDR_NUT contains a GDR picture. By using GDR pictures instead of IRAP pictures, a smoother and more consistent bitrate can be achieved, for example, due to the size of the GDR picture relative to the size of the IRAP picture. This allows for a reduction in end-to-end delay (i.e., latency). Therefore, the coder / decoder (also known as "codec") in video coding will be improved compared to current codecs. As a practical matter, the improved video coding process will provide users with a better user experience when video is sent, received, and / or viewed.

[0112] To address one or more of the issues discussed above, this disclosure discloses the following aspects. Each aspect may be applied individually, and some of them may be applied in combination.

[0113] 1) A VCL NAL unit with type GDR_NUT is defined.

[0114] a. A picture with a NAL unit type GDR_NUT is referred to as a GDR picture, i.e., the first picture in the GDR period.

[0115] b. A GDR picture has a temporal ID equal to 0.

[0116] c. An access unit containing a GDR picture is called a GDR access unit. As mentioned above, an access unit is a collection of NAL units. Each NAL unit can contain a single picture.

[0117] 2) A coded video sequence (CVS) may begin with a GDR access unit.

[0118] 3) A GDR access unit is the first access unit in CVS if any of the following is true:

[0119] a. The GDR access unit is the first access unit in the bitstream.

[0120] b. The GDR access unit comes immediately after the end-of-sequence (EOS) access unit.

[0121] c. The GDR access unit comes immediately after the end-of-bitstream (EOB) access unit.

[0122] d. A decoder flag called NoIncorrectPicOutputFlag is associated with the GDR picture, and the value of this flag is set to 1 (i.e., true) by an entity outside the decoder.

[0123] 4) If the GDR picture is the first access unit in CVS, the following applies:

[0124] a. All reference pictures in the DPB are marked "Not for Reference."

[0125] b. The POC MSB of the picture is set to equal to 0.

[0126] c. The GDR picture and all pictures that follow the GDR picture in the output order will not be output (i.e., will be marked as "not needed for output") until the last picture in the GDR period (except for the last picture in the GDR period).

[0127] 5) A flag specifying whether GDR is enabled is signaled in the sequence-level parameter set (for example, in SPS).

[0128] a. The flag may also be specified as gdr_enabled_flag.

[0129] b. If the flag is equal to 1, the GDR picture may exist in CVS. Otherwise, if the flag is equal to 0, GDR is not enabled and the GDR picture does not exist in CVS.

[0130] 6) Information that can be used to derive the POC value of the last picture during the GDR period is transmitted in the tile group header of the GDR picture.

[0131] a. This information is transmitted as the delta POC between the last picture and the GDR picture during the GDR period. This information can be transmitted using a syntactic element specified as recovery_point_cnt.

[0132] b. The presence of the syntactic element recovery_point_cnt in the tile group header may be conditional based on the value of the gdr_enabled flag and the NAL unit type of the picture. That is, the flag exists only if gdr_enabled_flag is equal to 1 and the nal_unit_type of the NAL unit containing the tile group is GDR_NUT.

[0133] 7) A flag is transmitted in the tile group header to indicate whether or not the tile group is part of the refresh area.

[0134] a. The flag may also be specified as refreshed_region_flag.

[0135] b. The existence of the flag may be conditional on the value of gdr_enabled_flag and whether the picture containing the tile group is within the GDR period. Thus, the flag exists only if all of the following are true:

[0136] i. The value of gdr_enabled_flag is equal to 1.

[0137] ii. The current picture's POC is greater than or equal to the POC value of the last GDR picture (if the current picture is a GDR picture, the last GDR picture is the current picture), and less than the POC of the last picture during the GDR period.

[0138] c. If the flag is not present in the tile group header, the flag's value is assumed to be equal to 1.

[0139] 8) All tile groups where refreshed_region_flag is equal to 1 cover a bound region. Similarly, all tile groups where refreshed_region_flag is equal to 0 also cover a bound region.

[0140] 9) A tile group with refreshed_region_flag can be of type I (i.e., intra-tile group) or B or P (i.e., inter-tile group).

[0141] 10) Each picture from the GDR picture to the last picture in the GDR period contains at least one tile group in which refreshed_region_flag is equal to 1.

[0142] 11) The GDR picture contains at least one tile group where refreshed_region_flag is equal to 1 and tile_group_type is equal to I (i.e., intra-tile group).

[0143] 12) If gdr_enabled_flag is equal to 1, information about rectangular tile groups, i.e., the number of tile groups and their addresses, may be signaled in either the Picture Parameter Set (PPS) or the Tile Group Header. To do this, a flag is signaled in the PPS to indicate whether rectangular tile group information is present in the PPS. This flag may be called rect_tile_group_in_pps_flag. This flag may be constrained to be equal to 1 when gdr_enabled_flag is equal to 1.

[0144] a. In one alternative, instead of signaling whether rectangular tile group information exists in the PPS, a more general flag can be signaled in the PPS that specifies whether tile group information (i.e., any type of tile group, such as rectangular tile groups or raster scan tile groups) exists in the PPS.

[0145] 13) If tile group information is not present in the PPS, it may be further constrained that there is no signaling of explicit tile group identifier (ID) information. Explicit tile group ID information includes signaled_tile_group_id_flag, signaled_tile_group_id_length_minus1, and tile_group_id[i].

[0146] 14) A flag is signaled to indicate whether loop filtering operation across the boundary between the refreshed and unrefreshed areas within the picture is permitted.

[0147] a. This flag is signaled in PPS and is called loop_filter_across_refreshed_region_enabled_flag.

[0148] b. The existence of loop_filter_across_refreshed_region_enabled_flag may be conditional on the value of loop_filter_across_tile_enabled_flag. If loop_filter_across_tile_enabled_flag is equal to 0, loop_filter_across_refreshed_region_enabled_flag does not need to exist, and its value is presumed to be equal to 0.

[0149] c. In one alternative, the flag may be signaled in the tile group header, and its presence may be conditional on the value of refreshed_region_flag. That is, the flag exists only when the value of refreshed_region_flag is equal to 1.

[0150] 15) If a tile group is indicated to be a refresh region and loop filters crossing the refresh region are not permitted, the following applies:

[0151] a. Edge unblocking at tile group boundaries is not performed if the neighboring tile group sharing the edge is a tile group that has not been refreshed.

[0152] b. The sample adaptive offset (SAO) process for blocks at the boundaries of tile groups does not use any samples from outside the refresh area boundary.

[0153] c. The adaptive loop filtering (ALF) process for blocks at the boundaries of tile groups does not use any samples from outside the refresh region boundary.

[0154] 16) If gdr_enabled_flag is equal to 1, each picture is associated with a variable to determine the boundaries of the refresh area within the picture. These variables may be called as follows:

[0155] a. The position of the left boundary of the refresh area within the picture is given by PicRefreshedLeftBoundaryPos.

[0156] b. The position of the right boundary of the refresh area within the picture, PicRefreshedRightBoundaryPos.

[0157] c. The position of the top boundary of the refresh area within the picture is given by PicRefreshedTopBoundaryPos.

[0158] d. The position of the lower boundary of the refresh area within the picture is given by PicRefreshedBotBoundaryPos.

[0159] 17) The refresh region boundary within the picture may be derived. The refresh region boundary of the picture is updated by the decoder after the tile group header is parsed and the value of the tile group's refreshed_region_flag is equal to 1.

[0160] 18) In one alternative to solution 17, the boundaries of the refreshed areas within the picture are explicitly signaled in each tile group of the picture.

[0161] a. A flag indicating whether the picture to which the tile group belongs contains an unrefreshed area may be signaled. If it is specified that the picture does not contain an unrefreshed area, refreshed boundary information is not signaled and can simply be assumed to be equal to the picture boundary.

[0162] 19) For the current picture, the refresh area boundary is used in the in-loop filtering process as follows:

[0163] a. For the deblocking process, determine whether the edges need to be deblocked in order to determine the edges in the refresh region.

[0164] b. For the SAO process, the boundaries of the refresh region are determined so that a clipping process can be applied to avoid using samples from the unrefreshed region when loop filtering across the refresh region is not permitted.

[0165] c. For the ALF process, the boundaries of the refresh region are determined so that a clipping process can be applied to avoid using samples from the unrefreshed region when loop filters crossing the refresh region are not permitted.

[0166] 20) For motion compensation processes, information regarding refresh region boundaries, particularly in reference pictures, is used as follows: If the current block in the current picture is in a tile group with a refreshed_region_flag equal to 1, and the reference block is in a reference picture that contains an unrefreshed region, then the following applies:

[0167] a. The motion vector from the current block to its reference picture is clipped by the boundary of the refresh region within that reference picture.

[0168] b. For the fractional interpolation filter on the samples in the reference picture, it is clipped by the boundaries of the refresh region in the reference picture.

[0169] A detailed description of embodiments of this disclosure is provided. This description is in relation to the base text, JVET-M1001-v5, a contribution from JVET. That is, only the differences are described, and text in the base text not mentioned below remains unchanged. Modified text in relation to the base text is italicized (underlined in the text).

[0170] A definition is given.

[0171] 3.1 Clean Random Access (CRA) Picture: An IRAP picture where each VCL NAL unit has a nal_unit_type equal to CRA_NUT.

[0172] Note - A CRA picture does not refer to any other picture for interpretation in its decoding process and may be the first picture in the bitstream in decoding order, or it may appear later in the bitstream. A CRA picture may have an associated RADL or RASL picture. CRA picture equals 1 NoIncorrectPicOutputFlag If this is present, the associated RASL picture will not be output by the decoder. This is because the RASL picture may not be decodeable, as it may contain references to pictures that do not exist in the bitstream.

[0173] 3.2 Coding Video Sequence (CVS): In decoded order, equal to 1 NoIncorrectPicOutputFlag IRAP access unit with Or a GDR access unit with NoIncorrectPicOutputFlag equal to 1 And then, the following, equal to 1 NoIncorrectPicOutputFlag IRAP access unit with Or a GDR access unit with NoIncorrectPicOutputFlag equal to 1IRAP access units consisting of zero or more access units that are not equal to 1 and have a NoIncorrectPicOutputFlag equal to 1 Or a GDR access unit with NoIncorrectPicOutputFlag equal to 1 A coding video sequence (CVS) that includes all subsequent access units, up to any subsequent access unit, but excluding that particular subsequent access unit.

[0174] Note 1 - IRAP access units may be IDR access units or CRA access units. NoIncorrectPicOutputFlag The value is equal to 1 for each IDR access unit and each CRA access unit that has HandleCraAsCvsStartFlag equal to 1, which is the first access unit in the bitstream in decode order, which is the first access unit following the end of the sequence NAL unit in decode order, or which is equal to 1.

[0175] Note 2 - NoIncorrectPicOutputFlag is equal to 1 for each GDR access unit that is the first access unit in the bitstream in decode order, the first access unit following the end of a sequence NAL unit in decode order, or has a HandleGdrAsCvsStartFlag equal to 1.

[0176] 3.3 Progressive Decode Refresh (GDR) Access Unit: An access unit in which the coding picture is a GDR picture.

[0177] 3.4 Gradual Decode Refresh (GDR) Picture: A picture in which each VCL NAL unit has a nal_unit_type equal to GDR_NUTS.

[0178] 3.5 Random Access Skip First (RASL) Picture: A coded picture where each VCL NAL unit has a nal_unit_type equal to RASL_NUT.

[0179] Note - All RASL pictures are the first picture of their associated CRA picture. (Associated CRA picture equals 1) NoIncorrectPicOutputFlag If present, RASL pictures will not be output and may not be able to decode correctly. This is because RASL pictures may contain references to pictures that do not exist in the bitstream. RASL pictures are not used as reference pictures for the decoding process of non-RASL pictures. If present, all RASL pictures come before all subsequent pictures of the same associated CRA picture in the decoding order.

[0180] Syntax and semantics of raw byte sequence payload (RBSP) for sequence parameter sets [Table 1]

[0181] A gdr_enabled_flag equal to 1 indicates that GDR pictures may exist in the coded video sequence. A gdr_enabled_flag equal to 0 indicates that GDR pictures do not exist in the coded video sequence.

[0182] RBSP syntax and semantics of picture parameter sets [Table 2]

[0183] A value of rect_tile_group_info_in_pps_flag equal to 1 indicates that rectangular tile group information is transmitted in PPS. A value of rect_tile_group_info_in_pps_flag equal to 0 indicates that rectangular tile group information is not transmitted in PPS.

[0184] The bitstream compatibility requirement is that the value of rect_tile_group_info_in_pps_flag must be equal to 0 if the value of gdr_enabled_flag in an active SPS is equal to 0.

[0185] A value of 1 for `loop_filter_across_refreshed_region_enabled_flag` indicates that in-loop filtering may be performed across the boundaries of tile groups with a `refreshed_region_flag` equal to 1 in a picture referencing a PPS. A value of 0 for `loop_filter_across_refreshed_region_enabled_flag` indicates that in-loop filtering will not be performed across the boundaries of tile groups with a `refreshed_region_flag` equal to 1 in a picture referencing a PPS. In-loop filtering operations include unblocking filters, sample-adaptive offset filters, and adaptive loop filtering operations. If not present, the value of `loop_filter_across_refreshed_region_enabled_flag` is assumed to be equal to 0.

[0186] A signaled_tile_group_id_flag equal to 1 indicates that the tile group ID for each tile group is signaled. A signaled_tile_group_index_flag equal to 0 indicates that the tile group ID is not signaled. Does not exist In this case, the value of signalled_tile_group_index_flag is presumed to be equal to 0.

[0187] Adding 1 to signalled_tile_group_length_minus1 specifies the number of bits used to represent the syntactic elements tile_group_id[i] and tile_group_address, if present, in the tile group header. The value of signalled_tile_group_index_length_minus1 is in the range of 0 to 15 (inclusive). If it does not exist, the value of signalled_tile_group_index_length_minus1 is estimated as follows:

[0188] If rect_tile_group_info_in_pps_flag is equal to 1, then Ceil(Log2(num_tile_groups_in_pic_minus1+1))-1.

[0189] Otherwise, use Ceil(Log2(NumTilesInPic))-1.

[0190] General tile, group, and header syntax and semantics [Table 3]

[0191] `tile_group_address` specifies the tile address of the first tile in a tile group. If it does not exist, the value of `tile_group_address` is assumed to be equal to 0.

[0192] If rect_tile_group_flag is equal to 0, the following applies: tile_group_address is the tile ID specified by formula 6-7. The length of tile_group_address is Ceil(Log2(NumTilesInPic)) bits. The value of tile_group_address should be in the range of 0 to NumTilesInPic-1 (including both ends).

[0193] Otherwise, if rect_tile_group_flag is equal to 1 and rect_tile_group_info_in_pps is equal to 0, the following applies: `tile_group_address` is the tile index of the tile located in the top-left corner of the i-th tile group. The length of tile_group_address is signed_tile_group_index_length_minus1+1 bits. If signelled_tile_group_id_flag is equal to 0, the value of tile_group_address will be in the range from 0 to NumTilesInPic-1 (including both ends). Otherwise, the value of tile_group_address will be from 0 to 2 (signalled_tile_group_index_length_minus1+1) The range is -1 (including both endpoints).

[0194] Otherwise (rect_tile_group_flag is equal to 1 and rect_tile_group_in_pps is equal to 1), The following applies: tile_group_address is the tile group ID of the tile group. The length of tile_group_address is signed_tile_group_index_length_minus1+1 bits. If signaled_tile_group_id_flag is equal to 0, the value of tile_group_address is in the range of 0 to num_tile_groups_in_pic_minus1 (including both ends). Otherwise, the value of tile_group_address is from 0 to 2 (signalled_tile_group_index_length_minus1+1) The range is -1 (including both endpoints).

[0195] `bottom_right_tile_id` specifies the tile index of the tile located in the bottom right corner of the tile group. If `single_tile_per_tile_group_flag` is equal to 1, `bottom_right_tile_id` is assumed to be equal to `tile_group_address`. The length of the `bottom_right_tile_id` syntax element is `Ceil(Log2(NumTilesInPic))` bits.

[0196] The variables NumTilesInCurrTileGroup, which specify the number of tiles in the current tile group; TopLeftTileIdx, which specifies the tile index of the top-left tile in the tile group; BottomRightTileIdx, which specifies the tile index of the bottom-right tile in the tile group; and TgTileIdx[i], which specifies the tile index of the i-th tile in the current tile group, are derived as follows: [Table 4]

[0197] `recovery_poc_cnt` specifies the recovery point of the decoded picture in output order. In CVS, if there is a picture picA that follows the current picture (i.e., the GDR picture) in decode order and has a PicOrderCntVal equal to the sum of the current picture's PicOrderCntVal and the value of `recovery_poc_cnt`, then picture picA is called the recovery point picture. Otherwise, the first picture in output order with a PicOrderCntVal greater than the sum of the current picture's PicOrderCntVal and the value of `recovery_poc_cnt` is called the recovery point picture. The recovery point picture does not precede the current picture in decode order. All decoded pictures in output order are shown as correct or nearly correct in content starting at the output order position of the recovery point picture. The value of `recovery_poc_cnt` is in the range of -MaxPicOrderCntLsb / 2 to MaxPicOrderCntLsb / 2-1 (including both ends).

[0198] The value of RecoveryPointPocVal is derived as follows:

[0199] RecoveryPointPocVal=PicOrderCntVal+recovery_poc_cnt.

[0200] A refreshed_region_flag equal to 1 indicates that decoding the tile group will produce correct reconstructed sample values ​​regardless of the NoIncorrectPicOutputFlag value of the associated GDR. A refreshed_region_flag equal to 0 indicates that decoding the tile group may produce incorrect reconstructed sample values ​​if it starts with an associated GDR that has a NoIncorrectPicOutputFlag equal to 1. If it does not exist, the value of refreshed_region_flag is assumed to be equal to 1.

[0201] Note x - The current picture itself can be a GDR picture with a NoIncorrectPicOutputFlag equal to 1.

[0202] The refreshed boundary of a tile group is derived as follows: [Table 5]

[0203] Meaning of NAL unit headers [Table 6]

[0204] ...

[0205] If nal_unit_type is equal to GDR_NUT, the coded tile group belongs to a GDR picture and TemporalId is equal to 0.

[0206] The order of access units and their association with CVS are discussed.

[0207] A bitstream conforming to this specification (i.e., JVET contribution JVET-M1001-v5) includes one or more CVSs.

[0208] A CVS contains one or more access units. The order of NAL units and coded pictures, and their association with access units, are described in Section 7.4.2.4.4.

[0209] The first unit of access in CVS is one of the following:

[0210] • An IRAP access unit with a NoBrokenPictureOutputFlag equal to 1.

[0211] A GDR access unit with NoIncorrectPicOutputFlag equal to 1.

[0212] The bitstream compliance requirement, if present, is that the next access unit following the end of a sequence NAL unit or an access unit containing the end of a bitstream NAL unit is one of the following:

[0213] • IRAP access unit. This can be an IDR access unit or a CRA access unit.

[0214] • GDR access unit.

[0215] 8.1.1 The decoding process of the coded picture is discussed.

[0216] …

[0217] If the current picture is an IRAP picture, the following applies.

[0218] · If the current picture is an IDR picture, the first picture in the bitstream in decode order, or the first picture following the end of the sequence NAL unit in decode order, the variable NoIncorrectPicOutputFlag is set equal to 1.

[0219] · Otherwise, if some external means (e.g., user input) not specified in this specification is available to set the variable HandleCraAsCvsStartFlag to the value of the current picture, the variable HandleCraAsCvsStartFlag is set equal to the value provided by the external means, and the variable NoIncorrectPicOutputFlag is set equal to HandleCraAsCvsStartFlag.

[0220] · Otherwise, the variable HandleCraAsCvsStartFlag is set equal to 0, and the variable NoIncorrectPicOutputFlag is set equal to 0.

[0221] If the current picture is a GDR picture, the following applies:

[0222] If the current picture is a GDR picture, the first picture in the bitstream in decoding order, or the first picture following the end of a sequence NAL unit in decoding order, the variable NoIncorrectPicOutputFlag is set to equal 1.

[0223] If, otherwise, any external means not specified in this specification is available to set the variable HandleGdrAsCvsStartFlag to a value for the current picture, the variable HandleGdrAsCvsStartFlag will be set to the value provided by the external means, and the variable NoIncorrectPicOutputFlag will be set to HandleGdrAsCvsStartFlag.

[0224] Otherwise, the variable HandleGdrAsCvsStartFlag is set to 0 and the variable NoIncorrectPicOutputFlag is set to 0.

[0225] ]>…

[0226] The decoding process works as follows for the current picture, CurrPic:

[0227] 1. Decoding of NAL units is specified in Section 8.2.

[0228] 2. The process in Section 8.3 defines the following decoding process using the tile group header layer and syntactic elements above it.

[0229] The variables and functions related to the picture order count are derived as specified in Section 8.3.1. This should only be called for the first tile group of the picture.

[0230] At the start of the decoding process for each tile group of non-IDR pictures, the decoding process for constructing reference picture lists as defined in Section 8.3.2 is invoked to derive reference picture list 0 (RefPicList[0]) and reference picture list 1 (RefPicList[1]).

[0231] The decoding process for reference picture marking in Section 8.3.3 is invoked. Here, the reference picture may be marked as "Not for reference" or "For long-term reference". This is invoked only for the first tile group of the picture.

[0232] The PicOutputFlag is set as follows:

[0233] • PictureOutputFlag is set to 0 if any of the following conditions are true:

[0234] • The current picture is a RASL picture, and the NoIncorrectPicOutputFlag of the associated IRAP picture is equal to 1.

[0235] • gdr_enabled_flag is equal to 1, and the current picture is a GDR picture with NoIncorrectPicOutputFlag equal to 1.

[0236] The gdr_enabled_flag is equal to 1, the current picture contains one or more tile groups with refreshed_region_flag equal to 0, and the NoBrokenPictureOutputFlag of the associated GDR picture is equal to 1.

[0237] · Otherwise, PicOutputFlag is set equal to 1.

[0238] 3. The various decoding processes are called to code tree units, scaling, conversion, in-loop filtering, etc.

[0239] 4. After all tile groups of the current picture have been decoded, the current decoded picture is marked as "used for short-term reference".

[0240] The decoding process for picture order count is discussed.

[0241] The output of this process is PicOrderCntVal, which is the picture order count of the current picture.

[0242] Each coded picture is associated with a picture order count variable indicated as PicOrderCntVal.

[0243] If the current picture is 1 not an IRAP picture with NoIncorrectPicOutputFlag equal to Or a GDR picture with NoIncorrectPicOutputFlag equal to 1 then the variables prevPicOrderCntLsb and prevPicOrderCntMsb are derived as follows.

[0244] · Let prevTid0Pic be the picture immediately preceding in decoding order that has a TemporalId equal to 0 and is not a RASL or RADL picture.

[0245] · The variable prevPicOrderCntLsb is set equal to the tile_group_pic_order_cnt_lsb of prevTid0Pic.

[0246] · The variable prevPicOrderCntMsb is set equal to the PicOrderCntMsb of prevTid0Pic.

[0247] The current picture variable PicOrderCntMsb is derived as follows:

[0248] • The current picture is equal to 1. NoIncorrectPicOutputFlag IRAP picture with Or a GDR picture with NoIncorrectPicOutputFlag equal to 1 If this is the case, PicOrderCntMsb will be set to equal to 0.

[0249] Otherwise, PicOrderCntMsb is derived as follows: [Table 7] PicOrderCntVal is derived as follows: PicOrderCntVal=PicOrderCntMsb+tile_group_pic_order_cnt_lsb (8-2)

[0250] Note 1 - Equivalent to 1 NoIncorrectPicOutputFlag All IRAP pictures that have this property have a PicOrderCntVal equal to tile_group_pic_order_cnt_lsb. This is because for IRAP pictures with NoIncorrectPicOutputFlag equal to 1, PicOrderCntMsb is set to equal to 0.

[0251] Note 1 - All GDR pictures with a NoIncorrectPicOutputFlag equal to 1 have a PicOrderCntVal equal to tile_group_pic_order_cnt_lsb. This is because for GDR pictures with a NoIncorrectPicOutputFlag equal to 1, PicOrderCntMsb is set to equal to 0.

[0252] The value of PicOrderCntVal is -2 31 From 2 31 The range is -1 (including both endpoints).

[0253] If the current picture is a GDR picture, the value of LastGDRPocVal is set to be equal to PicOrderCntVal.

[0254] The decoding process for the refresh boundary position of the picture is discussed.

[0255] This process is only invoked if gdr_enabled_flag is equal to 1.

[0256] This process is called after the parsing of the tile group header is complete.

[0257] The output of this process is the boundary position of the refresh area of ​​the current picture, which is called PicRefreshedLeftBoundaryPos, PicRefreshedRightBoundaryPos, PicRefreshedTopBoundaryPos, and PicRefreshedBotBoundaryPos.

[0258] Each coded picture is associated with a set of refresh region boundary position variables, which are represented as PicOrderCntVal.

[0259] PicRefreshedLeftBoundaryPos, PicRefreshedRightBoundaryPos, PicRefreshedTopBoundaryPos, and PicRefreshedBotBoundaryPos are derived as follows:

[0260] If the tile group is the first tile group received in the current picture with a refreshed_region_flag equal to 1, the following applies: PicRefreshedLeftBoundaryPos=TGRefreshedLeftBoundary PicRefreshedRightBoundaryPos=TGRefreshedRightBoundary PicRefreshedTopBoundaryPos=TGRefreshedTopBoundary PicRefreshedBotBoundaryPos=TileGroupBotBoundary Otherwise, if refreshed_region_flag is equal to 1, the following applies: PicRefreshedLeftBoundaryPos=TGRefreshedLeftBoundary < PicRefreshedLeftBoundaryPos ? TGRefreshedLeftBoundary : PicRefreshedLeftBoundaryPos PicRefreshedRightBoundaryPos=TGRefreshedRightBoundary > PicRefreshedRightBoundaryPos ? TGRefreshedRightBoundary : PicRefreshedRightBoundaryPos PicRefreshedTopBoundaryPos=TGRefreshedTopBoundary < PicRefreshedTopBoundaryPos ? TGRefreshedTopBoundary : RefreshedRegionTopBoundaryPos PicRefreshedBotBoundaryPos=TileGroupBotBoundary > PicRefreshedBotBoundaryPos ? TileGroupBotBoundary : PicRefreshedBotBoundaryPos

[0261] The decoding process for constructing a reference picture list is discussed. ...

[0262] Equal to 1 NoIncorrectPicOutputFlag IRAP picture with Or a GDR picture with NoIncorrectPicOutputFlag equal to 1 For each current picture that is not a bitstream, the bitstream compatibility requirement is that the value of maxPicOrderCnt - minPicOrderCnt is less than MaxPicOrderCntLsb / 2. ...

[0263] Decoding process for reference picture markings ... The current picture is an IRAP picture with NoIncorrectPicOutputFlag equal to 1. Or a GDR picture with NoIncorrectPicOutputFlag equal to 1 If this is the case, all referenced pictures (if any) in the DPB will now be marked as "Not for Reference". ...

[0264] A derivation process for predicting temporal luma motion vectors is discussed. ... The variable `currCb` specifies the current ruma coding block at the ruma location (xCb, yCb).

[0265] The variables mvLXCol and availableFlagLXCol are derived as follows: If tile_group_temporal_mvp_enabled_flag is equal to 0, both components of mvLXCol are set to equal to 0, and availableFlagLXCol is set to equal to 0. Otherwise (tile_group_temporal_mvp_enabled_flag is equal to 1), the following ordered steps apply: 1. The copositional motion vector in the lower right is derived as follows: [Table 8]

[0266] If yCb>>CtbLog2SizeY is equal to yColBr>>CtbLog2Size, and yColBr is in the range from topBoundaryPos to botBoundaryPos (including both ends), and xColBr is in the range from leftBoundaryPos to rightBoundaryPos (including both ends), then the following applies.

[0267] The variable colCb specifies a rumaccoding block that covers the modified position given by ((xColBr>>3)<<3,(yColBr>>3)<<3) within the copositional picture specified by ColPic.

[0268] · The luma position (xColCb, yColCb) is set equal to the sample at the upper left of the luma coding block at the co-location specified by colCb with respect to the sample at the upper left of the picture at the co-location specified by ColPic.

[0269] · The derivation process for the co-located motion vector specified in Section 8.5.2.12 is called with currCb, colCb, (xColCb, yColCb), refIdxLX, and sbFlag (set to 0) as inputs, and the outputs are assigned to mvLXCol and availableFlagLXCol.

[0270] · Otherwise, both components of mvLXCol are set equal to 0, and availableFlagLXCol is set equal to 0.

[0271] 2. … The luma sample bilinear interpolation process is discussed.

[0272] The inputs to this process are as follows: · The luma position in full sample units (xInt L , yInt L ) · The luma position in fractional sample units (xFrac L , yFrac L ) · The luma reference sample array refPicLX L • Refreshed area boundaries of the reference picture: PicRefreshedLeftBoundaryPos, PicRefreshedTopBoundaryPos, PicRefreshedRightBoundaryPos, and PicRefreshedBotBoundaryPos …

[0273] The luma position in full sample units (xInt i , yInt i ) is derived as follows for i = 0..1:

Table 9

[0274] The Luma sample 8-tap interpolation filtering process is discussed.

[0275] The inputs to this process are as follows: • Luma position in complete sample units (xInt L ,yInt L ) • Luma position in fractional sample units (xFrac L yFrac L ) • Luma reference sample sequence refPicLX L , • Reference sample: padVal[dir] - Specifies padding direction and amount, a list of padding values ​​of 0 and 1. • Refreshed area boundaries of the reference picture: PicRefreshedLeftBoundaryPos, PicRefreshedTopBoundaryPos, PicRefreshedRightBoundaryPos, and PicRefreshedBotBoundaryPos ...

[0276] Luma position in complete sample units (xInt i ,yInt i The following can be derived for i=0 to 7. [Table 10] ...

[0277] The chroma-sample interpolation process is discussed.

[0278] The inputs to this process are as follows: · Chroma position (xInt) C ,yInt C ) Chroma position in units of 1 / 32 fractional samples (xFrac C yFrac C ) • Chroma reference sample sequence refPicLX C • Refreshed area boundaries of the reference picture: PicRefreshedLeftBoundaryPos, PicRefreshedTopBoundaryPos, PicRefreshedRightBoundaryPos, and PicRefreshedBotBoundaryPos ...

[0279] The variable xOffset is set to equal to (sps_ref_wraparound_offset_minus1+1)*MinCbSizeY) / SubWidthC.

[0280] Chroma position at the complete sample level (xInt i ,yInt i The following can be derived for i=0..3: [Table 11]

[0281] The deblocking filter process is discussed.

[0282] General process ...

[0283] The unblocking filter process applies to all coding subblock edges and transform block edges in the picture, except for the following types of edges: • Edges at the boundary of the picture • If all of the following conditions are met, the edge matches the boundary on top of tile group tgA. • gdr_enabled_flag is equal to 1 • loop_filter_across_refreshed_region_enabled_flag is equal to 0 • The edge in question coincides with the lower boundary of tile group tgB, and the value of tgB's refreshed_region_flag is different from the value of tgA's refreshed_region_flag. • If all of the following conditions are met, the edge matches the left border of tile group tgA. • gdr_enabled_flag is equal to 1 • loop_filter_across_refreshed_region_enabled_flag is equal to 0 • The edge in question coincides with the right boundary of tile group tgB, and the value of tgB's refreshed_region_flag is different from the value of tgA's refreshed_region_flag. • If loop_filter_cross_tiles_enabled_flag is equal to 0, edges that match the tile boundaries Edges that match the top or left border of a tile group that has a tile_group_loop_filter_across_tile_groups_enabled_flag equal to 0 or a tile_group_deblocking_filter_disabled_flag equal to 1. Edges within tile groups that have a tile_group_deblocking_filter_disabled_flag equal to 1 • Edges that do not correspond to the 8x8 sample grid boundaries of the components being considered. • Edges in the chroma component where both sides of the edge use interpretation. - Edges of a chroma transformation block, not edges of the associated transformation unit. Edges that span rumor transformation blocks of coding units with an IntraSubPartitionsSplit value that is not equal to ISP_NO_SPLIT.

[0284] A deblocking filter process in a certain direction is discussed. ... For each coding unit with coding block width log2CbW, coding block height log2CbH, and the position of the top-left sample of the coding block (xCb, yCb), if edgeType is equal to EDGE_VER and xCb % 8 is equal to 0, or if edgeType is equal to EDGE_HOR and yCb % 8 is equal to 0, the edge is filtered by the following ordered steps:

[0285] 1. The coding block width nCbW is set to 1 << log2CbW, and the coding block height nCbH is set to 1 << log2CbH.

[0286] 2. The variable filterEdgeFlag is derived as follows. · When edgeType is equal to EDGE_VER and one or more of the following conditions are true, filterEdgeFlag is set to 0: · The left boundary of the current coding block is the left boundary of the picture. · The left boundary of the current coding block is the left boundary of the tile and loop_filter_across_tiles_enabled_flag is equal to 0. · The left boundary of the current coding block is the left boundary of the tile group and tile_group_loop_filter_across_tile_groups_enabled_flag is equal to 0. • The left boundary of the current coding block is the left boundary of the current tile group, and all of the following conditions are met: • gdr_enabled_flag is equal to 1 • loop_filter_across_refreshed_region_enabled_flag is equal to 0 - There exists a tile group that shares a border with the left border of the current tile group, and the value of its refreshed_region_flag is different from the value of the refreshed_region_flag of the current tile group.

[0287] · Otherwise, when edgeType is equal to EDGE_HOR and one or more of the following conditions are true, the variable filterEdgeFlag is set to 0: · The upper boundary of the current coding block is the upper boundary of the picture. · The upper boundary of the current coding block is the upper boundary of the tile and loop_filter_across_tiles_enabled_flag is equal to 0. · The upper boundary of the current coding block is the upper boundary of the tile group and tile_group_loop_filter_cross_tile_groups_enabled_flag is equal to 0. • The boundary above the current coding block is the boundary above the current tile group, and all of the following conditions are met: • gdr_enabled_flag is equal to 1. • loop_filter_across_refreshed_region_enabled_flag is equal to 0. - There is a tile group that shares a boundary with the current tile group, and the value of its refreshed_region_flag is different from the value of the current tile group's refreshed_region_flag.

[0288] Otherwise, filterEdgeFlag is set to 1.

[0289] Once the tiles are integrated, the syntax is applied.

[0290] 3. All elements of the two-dimensional (nCbW)×(nCbH) array edgeFlags are initialized to be equal to zero.

[0291] The CTB modification process for SAO is discussed. ...

[0292] All sample positions (xS) of i=0..nCtbSw-1 and j=0..nCtbSh-1 i ,yS j ) and (xY i ,yY j Regarding ), the coding unit pcm_loop_filter_disabled_flag, pcm_flag[xY i ][yY j ] and cu_transquant_bypass_flag Depending on the value of , the following applies: ·…

[0293] Modify the highlighted section by relying on future decision transformation / quantization bypasses. Otherwise, if SaoTypeIdx[cIdx][rx][ry] is equal to 2, the following ordered steps apply: 1. The values ​​of hPos[k] and vPos[k] for k=0..1 are defined in Table 8-18 based on SaoEoClass[cIdx][rx][ry]. 2. The variable edgeIdx is derived as follows: • Corrected sample position (xS ik' ,yS jk') and (xY ik' ,yY jk' ) is derived as follows: (xS ik' ,yS jk' )=(xS i +hPos[k],ySj+vPos[k]) (8-1128) (xY ik' ,yY jk' )=(cIdx==0) ? (xS ik' ,yS jk' ) : (xS ik' *SubWidthC,yS jk' *SubHeightC) (8-1129) · All sample positions (xS) of k=0..1 ik' ,yS jk' ) and (xY ik' ,yY jk' If one or more of the following conditions are true for ), edgeIdx is set to equal to 0. ·Position(xS ik' ,yS jk' The sample is outside the picture boundary. • gdr_enabled_flag is equal to 1, loop_filter_across_refreshed_region_enabled_flag is equal to 0, current tile group's refreshed_region_flag is equal to 1, position (xS ik' ,yS jk' The refreshed_region_flag of the tile group containing the sample in ) is equal to 0. ·Position(xS ik' ,yS jk' The samples of ) belong to different tile groups, and one of the following two conditions is true: ·MinTbAddrZs[xY ik' >>MinTbLog2SizeY][yY jk' >>MinTbLog2SizeY] is MinTbAddrZs[xY i >>MinTbLog2SizeY][yY j >>Smaller than MinTbLog2SizeY, sample recPicture[xS i ][yS j The tile_group_loop_filter_across_tile_groups_enabled_flag for the tile group to which ] belongs is equal to 0. ·MinTbAddrZs[xY i >>MinTbLog2SizeY][yY j >>MinTbLog2SizeY] is MinTbAddrZs[xY ik' >>MinTbLog2SizeY][yY jk' >>Smaller than MinTbLog2SizeY, sample recPicture[xS ik' ][yS jk' The tile_group_loop_filter_across_tile_groups_enabled_flag for the tile group to which ] belongs is equal to 0. · loop_filter_across_tiles_enabled_flag is equal to 0, position (xS ik' ,yS jk' The sample of ) belongs to a different tile.

[0294] If tiles without tile groups are included, correct the highlighted section. Otherwise, edgeIdx is derived as follows: The following applies: edgeIdx=2+Sign(recPicture[xS i ][yS j ]-recPicture[xS i +hPos[0]][yS j +vPos[0]])+ Sign(recPicture[xS i ][yS j ]-recPicture[xS i +hPos[1]][yS j +vPos[1]]) (8-1130) If edgeIdx is equal to 0, 1, or 2, edgeIdx is modified as follows: edgeIdx=(edgeIdx==2)? 0 : (edgeIdx+1) (8-1131)

[0295] 3. Modified picture sample array saoPicture[xS i ][yS j ] is derived as follows:

[0296] saoPicture[xS i ][yS j ]=Clip3(0,(1< <bitDepth)-1,recPicture[xS i ][yS j ]+ SaoOffsetVal[cIdx][rx][ry][edgeIdx]) (8-1132)

[0297] A coding tree block filtering process for luma samples for ALF is discussed. ...

[0298] To derive the filtered and reconstructed ruma sample alfPictureL[x][y], each reconstructed ruma sample in the current ruma coding tree block recPictureL[x][y] is filtered as follows, with x,y=0..CtbSizeY-1: ·… • The position (h) of each corresponding luma sample (x,y) in the given sequence recPicture of the luma sample. x ,v y ) is derived as follows: If gdr_enabled_flag is equal to 1, loop_filter_across_refreshed_region_enabled_flag is equal to 0, and refreshed_region_flag for tile group tgA containing the lumen sample at position (x,y) is equal to 1, then the following applies: ·Position(h x ,h y If ) is located in another tile group tgB and the refreshed_region_flag of tgB is equal to 0, then the variables leftBoundary, rightBoundary, topBoundary, and botBoundary are set to equal TGRefreshedLeftBoundary, TGRefreshedRightBoundary, TGRefreshedTopBoundary, and TGRefreshedBotBoundary, respectively. Otherwise, the variables leftBoundary, rightBoundary, topBoundary, and botBoundary are set to equal PicRefreshedLeftBoundaryPos, PicRefreshedRightBoundaryPos, PicRefreshedTopBoundaryPos, and PicRefreshedBotBoundaryPos, respectively. h x =Clip3(leftBoundary,rightBoundary,xCtb+x) (8-1140) v y =Clip3(topBoundary,botBoundary,yCtb+y) (8-1141) Otherwise, the following applies: h x=Clip3(0,pic_width_in_luma_samples - 1,xCtb + x) (8 - 1140) v y =Clip3(0,pic_height_in_luma_samples - 1,yCtb + y) (8 - 1141) ·…

[0299] The process of deriving the ALF transposition for luma samples and the filter index is discussed. …

[0300] For each of the corresponding luma samples (x, y) in a given array recPicture of luma samples, the position (h x ,v y ) is derived as follows: If gdr_enabled_flag is equal to 1, loop_filter_across_refreshed_region_enabled_flag is equal to 0, and refreshed_region_flag for tile group tgA containing the lumen sample at position (x,y) is equal to 1, then the following applies: ·Position(h x ,h y If ) is located in another tile group tgB and the refreshed_region_flag of tgB is equal to 0, then the variables leftBoundary, rightBoundary, topBoundary, and botBoundary are set to equal TGRefreshedLeftBoundary, TGRefreshedRightBoundary, TGRefreshedTopBoundary, and TGRefreshedBotBoundary, respectively. Otherwise, the variables leftBoundary, rightBoundary, topBoundary, and botBoundary are set to equal PicRefreshedLeftBoundaryPos, PicRefreshedRightBoundaryPos, PicRefreshedTopBoundaryPos, and PicRefreshedBotBoundaryPos, respectively. h x =Clip3(leftBoundary,rightBoundary,x) (8-1140) v y =Clip3(topBoundary,botBoundary,y) (8-1141) · Otherwise, the following applies: h x =Clip3(0,pic_width_in_luma_samples - 1,x) (8 - 1145) v y =Clip3(0,pic_height_in_luma_samples - 1,y) (8 - 1146) The coding tree block filtering process for chroma samples is discussed. …

[0301] To derive the filtered and reconstructed chroma samples alfPicture[x][y], each reconstructed chroma sample in the current chroma coding tree block recPicture[x][y] is filtered as follows, with x,y = 0..ctbSizeC-1: • The position (h) of each corresponding chromatic sample (x,y) in the given sequence recPicture of the chromatic sample. x ,v y ) is derived as follows: If gdr_enabled_flag is equal to 1, loop_filter_across_refreshed_region_enabled_flag is equal to 0, and refreshed_region_flag for tile group tgA containing the lumen sample at position (x,y) is equal to 1, then the following applies: ·Position(h x ,h y If ) is located in another tile group tgB and the refreshed_region_flag of tgB is equal to 0, then the variables leftBoundary, rightBoundary, topBoundary, and botBoundary are set to equal TGRefreshedLeftBoundary, TGRefreshedRightBoundary, TGRefreshedTopBoundary, and TGRefreshedBotBoundary, respectively. Otherwise, the variables leftBoundary, rightBoundary, topBoundary, and botBoundary are set to equal PicRefreshedLeftBoundaryPos, PicRefreshedRightBoundaryPos, PicRefreshedTopBoundaryPos, and PicRefreshedBotBoundaryPos, respectively. h x =Clip3(leftBoundary / SubWidthC,rightBoundary / SubWidthC,xCtbC+x) (8-1140) v y =Clip3(topBoundary / SubWidthC,botBoundary / SubWidthC,yCtbC+y) (8-1141) Otherwise, the following applies: h x =Clip3(0,pic_width_in_luma_samples / SubWidthC-1,xCtb+x) (8-1177) v y =Clip3(0,pic_height_in_luma_samples / SubHeightC-1,yCtb+y) (8-1178)

[0302] Figure 7 shows a video bitstream 750 configured to implement a progressive decode-refresh (GDR) technique 700 according to one embodiment of the present disclosure. The GDR technique 700 may be similar to the GDR technique 500 in Figure 5. As used herein, the video bitstream 750 may be referred to as a coded video bitstream, a bitstream, or a variation thereof. As shown in Figure 7, the bitstream 750 includes a sequence parameter set (SPS) 752, a picture parameter set (PPS) 754, a slice header 756, and image data 758.

[0303] SPS 752 contains data common to all pictures in a sequence of pictures (SOP). In contrast, PPS 754 contains data common to the entire picture. The slice header 756 contains information about the current slice, such as the slice type and which of the reference pictures is used. SPS 752 and PPS 754 may also be commonly referred to as parameter sets. SPS 752, PPS 754, and slice header 756 are types of network abstraction layer (NAL) units. A NAL unit is a syntactic structure that contains instructions for the type of data to follow (e.g., coded video data). NAL units are classified into video coding layer (VCL) and non-VCL NAL units. VCL NAL units contain data representing the values ​​of samples within a video picture, while non-VCL NAL units contain any relevant additional information, such as parameter sets (essential header data applicable to many VCL NAL units) and supplemental enhancement information (other supplemental data that may enhance the usefulness of the decoded video signal but is not necessary for decoding the timing information and the values ​​of samples within the video picture). Those skilled in the art will understand that bitstream 750 may contain other parameters and information in practical applications.

[0304] The image data 758 in Figure 7 includes data related to an image or video to be encoded or decoded. The image data 758 may simply be referred to as the payload or data carried within the bitstream 750. In one embodiment, the image data 758 includes a CVS 708 which includes a GDR picture 702, one or more subsequent pictures 704, and a recovery point picture 706. In one embodiment, the GDR picture 702, subsequent pictures 704, and recovery point picture 706 may define a GDR period within the CVS 708.

[0305] As shown in Figure 7, the GDR technique 700 or principle operates on a series of pictures beginning with GDR picture 702 and ending with recovery point picture 706. GDR picture 702 includes a refresh / clean area 710 containing blocks coded entirely using intra-prediction (i.e., intra-predicted blocks) and an unrefreshed / dirty area 712 containing blocks coded entirely using inter-prediction (i.e., inter-predicted blocks).

[0306] The subsequent picture 704, immediately adjacent to the GDR picture 702, includes a refresh / clean region 710 having a first portion 710A coded using intra-prediction and a second portion 710B coded using inter-prediction. The second portion 710B is coded, for example, by referencing the refresh / clean region 710 of the preceding picture within the GDR period of CVS 708. As shown in the figure, the refresh / clean region 710 of the subsequent picture 704 expands as the coding process moves or progresses in a consistent direction (e.g., from left to right), correspondingly shrinking the unrefreshed / dirty region 712. Finally, a recovery point picture 706 containing only the refresh / clean region 710 is obtained from the coding process. The second portion 710B of the refresh / clean region 710, coded as an inter-prediction block, may simply referencing the refresh / clean region 710 in the reference picture.

[0307] As shown in Figure 7, the GDR picture 702, the subsequent picture 704, and the recovery point picture 706 within the CVS 708 are each contained within their own VCL NAL unit 730. The set of VCL NAL units 730 within the CVS 708 may be referred to as an access unit.

[0308] A NAL unit 730 containing the GDR picture 702 in CVS 708 has a GDR NAL unit type (GDR_NUT). That is, in one embodiment, a NAL unit 730 containing the GDR picture 702 in CVS 708 has its own unique NAL unit type for subsequent pictures 704 and recovery point picture 706. In one embodiment, GDR_NUT allows the bitstream to begin with the GDR picture 702, rather than the bitstream having to begin with an IRAP picture. Specifying the VCL NAL unit 730 for the GDR picture 702 as GDR_NUT can indicate, for example, to the decoder that the first VCL NAL unit 730 in CVS 708 contains the GDR picture 702.

[0309] In some embodiments, GDR picture 702 is the first picture in CVS 708. In some embodiments, GDR picture 702 is the first picture in a GDR period. In some embodiments, GDR picture 702 has a temporal identifier (ID) equal to zero. The temporal ID is a value or number that identifies the position or order of the picture relative to other pictures. In some embodiments, an access unit containing a VCL NAL unit 730 having GDR_NUT is designated as a GDR access unit. In some embodiments, GDR picture 702 is a code slice of another (e.g., a larger) GDR picture. That is, GDR picture 702 may be part of a larger GDR picture.

[0310] Figure 8 shows an embodiment of method 800 for decoding a coded video bitstream, implemented by a video decoder (e.g., video decoder 30). Method 800 may be performed after the decoded bitstream has been received directly or indirectly from a video encoder (e.g., video encoder 20). Method 800 improves the decoding process because it allows for incremental intra-refreshes to enable random access without the need to use IRAP pictures. By using GDR pictures instead of IRAP pictures, for example, a smoother and more consistent bitrate may be achieved due to the size of the GDR picture relative to the size of the IRAP picture, which allows for reduced end-to-end delay (i.e., latency). Thus, as a practical matter, the codec performance is improved, which leads to a better user experience.

[0311] In block 802, the video decoder determines that the coded video sequence (CVS) of the coded video bitstream includes a video coding layer (VCL) network abstraction layer (NAL) unit having a gradual decoding refresh (GDR) network abstraction layer (NAL) unit type (GDR_NUT). The VCL NAL unit having the GDR_NUT includes a GDR picture.

[0312] In some embodiments, a GDR picture is the first picture in a CVS. In some embodiments, a GDR picture is the first picture in a GDR period. In some embodiments, a GDR picture has a temporal identifier (ID) equal to zero. A temporal ID of zero may indicate that a GDR picture is, for example, the first picture in a CVS or a GDR period. In some embodiments, an access unit containing a VCL NAL unit having GDR_NUT is designated as a GDR access unit. In some embodiments, a GDR picture is a coded slice of another GDR picture.

[0313] In block 804, the video decoder begins decoding the CVS in the GDR picture.

[0314] In block 806, the video decoder generates an image according to the decoded CVS. The image may then be displayed for the user of an electronic device (e.g., a smartphone, tablet, laptop, personal computer, etc.).

[0315] Figure 9 shows one embodiment of method 900 for encoding a video bitstream, implemented by a video encoder (e.g., video encoder 20). Method 900 may be performed when a picture (e.g., from a video) is encoded into a video bitstream and then transmitted to a video decoder (e.g., video decoder 30). Method 900 improves the encoding process by allowing progressive intra-refresh to enable random access without the need to use IRAP pictures. By using GDR pictures instead of IRAP pictures, for example, a smoother and more consistent bitrate may be achieved due to the size of the GDR picture relative to the size of the IRAP picture, which allows for reduced end-to-end delay (i.e., latency). Thus, as a practical matter, the codec performance is improved, which leads to a better user experience.

[0316] In block 902, the video encoder determines a random access point for the video sequence. In block 904, at the random access point for the video sequence, the video encoder encodes a gradual decoding refresh (GDR) picture into video coding layer (VCL) network abstraction layer (NAL) units having a gradual decoding refresh (GDR) network abstraction layer (NAL) unit type (GDR_NUT).

[0317] In some embodiments, a GDR picture is the first picture in a CVS. In some embodiments, a GDR picture is the first picture in a GDR period. In some embodiments, a GDR picture has a temporal identifier (ID) equal to zero. A temporal ID of zero may indicate that a GDR picture is, for example, the first picture in a CVS or a GDR period. In some embodiments, an access unit containing a VCL NAL unit having GDR_NUT is designated as a GDR access unit. In some embodiments, a GDR picture is a coded slice of another GDR picture.

[0318] In block 906, the video encoder generates a bitstream containing the video sequence having the GDR picture in the VCL NAL unit having the GDR_NUT at the random access point.

[0319] In block 908, the bitstream is stored for transmission to the video decoder. The video bitstream is also called the coded video bitstream or encoded video bitstream. The video encoder can transmit the bitstream to the video decoder. Once received by the video decoder, the encoded video bitstream can be decoded (for example, as described above) to generate or produce an image for display to the user on the display or screen of an electronic device (for example, a smartphone, tablet, laptop, personal computer, etc.).

[0320] Figure 10 is a schematic diagram of a video coding device 1000 (for example, a video encoder 20 or a video decoder 30) according to one embodiment of the present disclosure. The video coding device 1000 is suitable for implementing the disclosed embodiments as described herein. The video coding device 1000 has an inlet port 1010 and a receiver unit (Rx) 1020 for receiving data, a processor, logic unit, or central processing unit (CPU) 1030 for processing the data, a transmitter unit (Tx) 1040 and an exit port 1050 for transmitting the data, and a memory 1060 for storing the data. The video coding device 1000 may also have optical-to-electrical (OE) components and electrical-to-optical (EO) components coupled to the inlet port 1010, the receiver unit 1020, the transmitter unit 1040, and the exit port 1050 for the input and output of optical or electrical signals.

[0321] The processor 1030 is implemented by hardware and software. The processor 1030 may be implemented as one or more CPU chips, cores (e.g., a multi-core processor), field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), and digital signal processors (DSPs). The processor 1030 communicates with the inlet port 1010, the receiver unit 1020, the transmitter unit 1040, the exit port 1050, and the memory 1060. The processor 1030 has a coding module 1070. The coding module 1070 implements the embodiments disclosed above. For example, the coding module 1070 implements, processes, prepares, or provides various codec functions. Thus, by including the coding module 1070, the functionality of the video coding device 1000 is substantially improved so that the video coding device 1000 can perform conversions to different states. Alternatively, the coding module 1070 may be implemented as an instruction stored in memory 1060 and executed by processor 1030.

[0322] The video coding device 1000 may also include an input and / or output (I / O) device 1080 for communicating data with the user. The I / O device 1080 may include output devices such as a display for showing video data and speakers for outputting audio data. The I / O device 1080 may also include input devices such as a keyboard, mouse, or trackball, and / or corresponding interfaces for interacting with such output devices.

[0323] Memory 1060 includes one or more disks, tape drives, and solid-state drives and may be used as an overflow data storage device to store the program when it is selected for execution and to store instructions and data read during the execution of the program. Memory 1060 may be volatile and / or non-volatile and may be read-only memory (ROM), random-access memory (RAM), tertiary associative memory (TCAM), and / or static random-access memory (SRAM).

[0324] Figure 11 is a schematic diagram of one embodiment of the coding means 1100. In one embodiment, the coding means 1100 is implemented in a video coding device 1102 (for example, a video encoder 20 or a video decoder 30). The video coding device 1102 includes a receiving means 1101. The receiving means 1101 is configured to receive a picture to encode or a bitstream to decode. The video coding device 1102 includes a transmitting means 1107 coupled to the receiving means 1101. The transmitting means 1107 is configured to transmit a bitstream to a decoder or transmit a decoded picture to a display means (for example, one of the I / O devices 1080).

[0325] The video coding device 1102 includes a storage means 1103. The storage means 1103 is coupled to at least one of the receiving means 1101 or the transmitting means 1107. The storage means 1103 is configured to store instructions. The video coding device 1102 also includes a processing means 1105. The processing means 1105 is coupled to the storage means 1103. The processing means 1105 is configured to execute instructions stored in the storage means 1103 in order to carry out the method disclosed herein. Furthermore, the steps of the exemplary methods described herein do not necessarily have to be performed in the order described, and the order of the steps of such methods should be understood to be merely illustrative. Similarly, additional steps may be included in such methods, and certain steps may be omitted or combined in a manner consistent with the various embodiments of this disclosure.

[0326] While several embodiments are provided in this disclosure, it should be understood that the disclosed systems and methods can be embodied in many other specific forms without departing from the spirit or scope of this disclosure. The examples of this application should be considered illustrative, not restrictive, and their intent is not limited to the details given herein. For example, different elements or components may be combined or integrated into another system, or certain features may be omitted or not implemented.

[0327] Furthermore, technologies, systems, subsystems, and methods described and illustrated as discrete or separate in various embodiments may be combined with or integrated with other systems, modules, technologies, or methods without departing from the scope of this disclosure. Other items shown or discussed as being coupled to one another, directly coupled to one another, or communicating with one another may be indirectly coupled or communicated through some interface, device, or intermediate component, electrically, mechanically, or otherwise. Other examples of modifications, substitutions, and alterations can be identified by those skilled in the art and can be made without departing from the spirit and scope disclosed herein.

Claims

1. A method for obtaining a bitstream, The stage of acquiring the video sequence; The step of encoding the aforementioned video sequence into a bitstream The bitstream includes an access unit, the access unit includes at least one video coding layer (VCL) network abstraction layer (NAL) unit, the at least one VCL NAL unit includes a coded progressive decode refresh (GDR) picture, each of the at least one VCL NAL unit has a GDR NAL unit type (GDR_NUT), and a syntactic element specifying a delta picture sequential count (POC) between the GDR picture and the recovery point picture associated with the GDR picture is present in the header of the GDR picture. method.

2. The method according to claim 1, wherein the header is a slice header.

3. The method according to claim 1, wherein the syntactic element is represented as recovery_poc_cnt.

4. The aforementioned recovery point picture is: A picture that follows the aforementioned GDR picture and has a POC value greater than the sum of the POC value of the aforementioned GDR picture and the value of recovery_poc_cnt, or The first picture in the output order that has a POC value greater than the sum of the POC value of the aforementioned GDR picture and the value of the aforementioned recovery_poc_cnt, The method according to claim 3.

5. The method according to claim 1, wherein a flag for specifying that a GDR picture exists in the bitstream is signaled in a sequence-level parameter set in the bitstream.

6. The method according to claim 1, wherein the temporal identifier (ID) associated with the GDR picture is equal to 0.

7. A processing circuit configured to perform the method described in any one of claims 1 to 6, or One or more processors; and A computer-readable storage medium coupled to the processor and storing a program for execution by the processor, A device wherein, when executed by the processor, the program enables the device to perform the method described in any one of claims 1 to 6. device.

8. A computer-readable storage medium storing a computer program that, when executed on a computer or processor, enables the computer or processor to perform the method described in any one of claims 1 to 6.

9. A computer-readable storage medium having the bitstream obtained according to the method described in any one of claims 1 to 6.

10. A device having a transmitter and a memory, wherein the memory is configured to store a bitstream obtained according to any one of claims 1 to 6, and the transmitter is configured to transmit the bitstream.

Images