Method for recovery point process for video coding and related apparatus

Abstract

A method of decoding a picture from a bitstream is provided. The method includes identifying a recovery point in the bitstream according to the recovery point indication. The recovery point specifies a starting position in the bitstream for decoding the set of pictures. The method also includes decoding the recovery point indication to obtain a set of decoded syntax elements. The method also includes deriving information for generating a set of unavailable reference pictures from the set of decoded syntax elements prior to the decoder parsing any encoded picture data. The method further includes generating a set of unavailable reference pictures based on the derived information. The method further includes decoding the set of pictures after generating the set of unavailable reference pictures. A method performed by an encoder is also provided.

Description

[0001] This application is a divisional application of the Chinese patent application "Method and related device for recovery point process of video encoding" (application number: 202080019901.9) with an application date of March 10, 2020. Technical Field

[0002] The present disclosure generally relates to a method of encoding a recovery point indication having information about how to generate an unavailable reference picture into a bitstream and a method of decoding a set of pictures from the bitstream. The present disclosure also relates to an encoder configured to encode the recovery point indication and a decoder configured to decode the set of pictures. Background Art

[0003] High Efficiency Video Coding (HEVC) and Versatile Video Coding (VVC) will now be discussed. HEVC is a block-based video codec standardized by ITU-T and MPEG that utilizes both temporal and spatial prediction. Spatial prediction can be achieved using intra (I) prediction from within the current picture. Temporal prediction can be achieved using unidirectional (P) or bidirectional (B) intra prediction at the block level based on previously decoded reference pictures. In the encoder, the difference between the original pixel data and the predicted pixel data (called the residual) can be transformed to the frequency domain, quantized, and then entropy encoded before being sent along with prediction parameters (such as prediction mode and motion vectors, which are also entropy encoded). The decoder can perform entropy decoding, inverse quantization, and inverse transformation to obtain the residual, which can then be added to the intra prediction or inter prediction to reconstruct the picture.

[0004] MPEG and ITU-T are developing a successor to HEVC within the Joint Video Exploration Team (JVET). The name of this video codec under development is VVC.

[0005] The components of an image will now be discussed. A video sequence consists of a series of images, where each image consists of one or more components. Each component can be described as a two-dimensional rectangular array of sample values. An image in a video sequence contains three components; a luma component Y, where the sample values ​​are luma values, and two chroma components Cb and Cr, where the sample values ​​are chroma values. The chroma components can be 1 / 2 the size of the luma components in each dimension to save bits of compression. For example, the luma component of a high-definition image can be sized 1920×1080, and the chroma components can each have a size of 960×540. Components are sometimes also referred to as color components.

[0006] Blocks and units will now be discussed. A block is a two-dimensional array of samples. In video coding, each component can be partitioned into blocks, and the coded video bitstream includes a series of coded blocks. In video coding, an image can be partitioned into units covering a specific area of ​​the image. Each unit includes all blocks from all components that make up that specific area, and each block belongs to one unit. Macroblocks in H.264 and Coding Units (CUs) in HEVC are examples of units.

[0007] Blocks may alternatively be described as two-dimensional arrays that apply the transforms used in encoding. These blocks may be referred to as "transform blocks". Alternatively, blocks may be described as two-dimensional arrays that apply a single prediction mode. These blocks may be referred to as "prediction blocks". In the present disclosure, the term "block" is not limited to one of these descriptions, but the description herein may apply to either "transform blocks" or "prediction blocks".

[0008] NAL units will now be discussed. Both HEVC and VVC define a network abstraction layer (NAL). All data (such as the video coding layer (VCL) or non-VCL data in HEVC and VVC) are encapsulated in NAL units. VCL NAL units may contain data representing picture sample values. Non-VCL NAL units may contain other associated data, such as parameter sets and supplementary enhancement information (SEI) messages. NAL units in HEVC begin with a header that may specify the NAL unit type of the NAL unit, which identifies which type of data is carried in the NAL unit, the layer ID to which the NAL unit belongs, and the time ID. The NAL unit type is sent in the nal_unit_type codeword in the NAL unit header, and the type indicates and can define how the NAL unit should be parsed and decoded. The remaining bytes of the NAL unit are payloads of the type indicated by the NAL unit type. The bitstream includes a series of concatenated NAL units. The bitstream includes a series of concatenated NAL units.

[0009] Figure 1 The syntax of the NAL unit header in HEVC is shown in FIG.

[0010] The first byte of each NAL unit in VVC and HEVC contains the nal_unit_type syntax element. A decoder or bitstream parser can infer how the NAL unit should be processed (e.g., parsed and decoded) after looking at the first byte. VCL NAL units provide information about the picture type of the current picture. Figure 2 The NAL unit types for the current version of the VVC draft, JVET-M1001-v5, at the time of writing are shown in FIG.

[0011] The decoding order is the order in which NAL units should be decoded, which is the same as the order of NAL units within the bitstream. The decoding order may be different from the output order, which is the order in which decoded pictures are to be output by a decoder (e.g., for display).

[0012] We will now discuss intra random access point (IRAP) pictures and coded video sequences (CVS). For single-layer coding in HEVC, an access unit (AU) is a coded representation of a single picture. An AU can include several video coding layer (VCL) NAL units as well as non-VCL NAL units.

[0013] An intra random access point (IRAP) picture in HEVC is a picture that is not predicted with reference to any picture other than itself during its decoding process. In HEVC, the first picture in the bitstream in decoding order must be an IRAP picture, but IRAP pictures may also appear later in the bitstream. HEVC can specify three types of IRAP pictures, broken link access (BLA) pictures, instantaneous decoder refresh (IDR) pictures, and clean random access (CRA) pictures.

[0014] A coded video sequence (CVS) in HEVC is a sequence of access units starting from an IRAP access unit and up to (but not including) the next IRAP access unit in decoding order.

[0015] An IDR picture always starts a new CVS. An IDR picture may have an associated Random Access Decodable Leading (RADL) picture. An IDR picture does not have an associated RASL picture.

[0016] BLA pictures also start a new CVS and have the same impact on the decoding process as IDR pictures. However, BLA pictures in HEVC can contain syntax elements that specify a non-empty set of reference pictures. BLA pictures can have associated RASL pictures, which are not output by the decoder and may not be decodable because they may contain references to pictures that may not be present in the bitstream. BLA pictures can also have associated RADL pictures, which are decoded.

[0017] A CRA picture may have an associated RADL or RASL picture. Like a BLA picture, a CRA picture may contain syntax elements that specify a non-empty set of reference pictures. For a CRA picture, a flag may be set to specify that the associated RASL pictures are not to be output by the decoder, since these pictures may not be decodable because they may contain references to pictures that are not present in the bitstream. A CRA may or may not start a CVS.

[0018] Now we will discuss parameter sets. HEVC can specify three types of parameter sets, picture parameter sets (PPS), sequence parameter sets (SPS), and video parameter sets (VPS). The PPS contains data common to the entire picture, the SPS contains data common to a coded video sequence (CVS), and the VPS contains data common to multiple CVSs.

[0019] Now we will discuss tiles. The HEVC and draft VVC video coding standards include a tool called tiles that divide a picture into spatially independent rectangular regions. The tiles in the draft VVC coding standard are very similar to the tiles used in HEVC. Using tiles, a picture in VVC can be divided into rows of samples and columns of samples, where a tile is the intersection of a row and a column. Figure 3 An example of a tile partitioning using 4 tile rows and 5 tile columns is shown, resulting in a picture with a total of 20 tiles. Figure 3 is an exemplary tile partitioning.

[0020] The block structure will now be discussed. In the HEVC and drafted VVC specifications, each picture is divided into square blocks called coding tree units (CTUs). The size of all CTUs is the same, and the division is done without any syntax controlling the division. Each CTU is further divided into coding units (CUs), which can have a square or rectangular shape. The coded picture may include a series of coded CTUs according to a determined scanning order, which may be a raster scan order, for example. Other CTU scanning orders may occur, such as when tiles are used. The coded picture may then include a series of coded tiles in a tile raster scan order, wherein each coded tile may include a series of CTUs in a CTU raster scan order.

[0021] Reference picture management will now be discussed. Pictures in HEVC are identified by their picture order count (POC) value (also called the full POC value). Each slice contains a codeword pic_order_cnt_lsb, which should be the same for all slices in the picture. Pic_order_cnt_lsb is also called the least significant bit (lsb) of the full POC because it is a fixed-length codeword and only the least significant bit of the full POC is signaled. Both the encoder and the decoder keep track of the POC and assign the POC value to each picture being encoded / decoded. Pic_order_cnt_lsb can be signaled by 4-16 bits. The variable MaxPicOrderCntLsb is used in HEVC, which is set to the maximum pic_order_cnt_lsb value plus 1. This means that if 8 bits are used to signal pic_order_cnt_lsb, the maximum value is 255, and MaxPicOrderCntLsb is set to 2^8 = 256. The picture order count value of a picture is called PicOrderCntVal in HEVC. Typically, the PicOrderCntVal of the current picture is simply referred to as PicOrderCntVal.

[0022] Reference picture management in HEVC is done using Reference Picture Sets (RPS). A reference picture set is a collection of reference pictures signaled in a slice header. When the decoder has decoded a picture, the decoded picture is placed in a decoded picture buffer (DPB) along with its POC value. When decoding subsequent pictures, the decoder parses the RPS syntax from the slice header and builds a list of reference picture POC values. These lists are compared with the POC values ​​of the pictures stored in the DPB, and the RPS can specify which pictures in the DPB to keep in the DPB and which pictures to remove. All pictures that are not included in the RPS are removed from the DPB. Based on the decoded RPS information, the pictures that remain in the DPB are either marked as short-term reference pictures or as long-term reference pictures.

[0023] One property of the HEVC reference picture management system is that, for each slice, it signals the state that the DPB should have before decoding the current picture. This enables the decoder to compare the signaled state with the actual state of the DPB and determine if any reference pictures are missing.

[0024] Reference picture management in the draft VVC specification is slightly different from that in HEVC. In HEVC, the RPS is signaled and the reference picture list for inter prediction is derived from the RPS. In the draft VVC specification, the reference picture list (RPL) is signaled and the RPS is derived. However, in both specifications, the signaling of which pictures to keep in the DPB and which pictures should be short-term and long-term is done. POC is used in the same way in both specifications for picture identification and determination of missing reference pictures.

[0025] Recovery points will now be discussed. Recovery points are used to perform random access operations in the bitstream using only temporal prediction pictures. Recovery points can also be used to refresh the video in the event of video data loss.

[0026] A decoder that performs random access operations in the bitstream decodes all pictures in the recovery point period without outputting them. When the last picture of the recovery point period (the recovery point picture) is reached, the video has been completely refreshed and the recovery point picture and subsequent pictures can be output. The recovery point mechanism is sometimes called gradual decoding refresh (GDR) because the mechanism refreshes the video step by step, picture by picture.

[0027] In practice, GDR is created by refreshing the video progressively using intra-coded blocks (e.g., CTUs). For each picture in the recovery point cycle, a larger portion of the video is refreshed until the video has been completely refreshed.

[0028] Figure 4 Two different example patterns for progressive decode refreshing of video are shown, vertical lines and a pseudo-random pattern. Figure 4 A progressive decoding refresh over five pictures is shown. White blocks are non-refreshed blocks (or "dirty" blocks), dark grey blocks are intra-coded blocks, and dark grey and medium grey blocks are refreshed blocks (or "clean" blocks). Figure 4 The top row shows the progressive refreshing of vertical lines using intra-coded blocks. Figure 4 The bottom row shows a progressive refresh using a pseudo-random pattern. Other common patterns can include horizontal lines and block by block in a raster scan order. Figure 4 The block in the example may be a CTU.

[0029] A refresh block can be configured to be predicted only from other refresh blocks in the current (spatial intra prediction) and previous pictures (temporal prediction). This prevents artifacts from spreading to refresh areas between pictures.

[0030] Slices or tiles may be used to restrict prediction between unrefreshed blocks and refreshed blocks in an efficient manner, since slice and tile boundaries may shut down prediction across the boundary but allow prediction elsewhere. Figure 5An example of using restrictions in tiles for GDR is shown. Figure 5 In , the tile borders are shown with thick lines. One tile is used for clean areas and one tile is used for dirty areas. Figure 5 In a first example, the picture can be divided into two tiles, one of which includes refreshed blocks and the other includes non-refreshed blocks. Figure 5 In the example of , the tile distribution and tile size are not constant over time.

[0031] A certain degree of artifacts can also be tolerated by not restricting or only partially restricting the temporal and spatial predictions for the refresh area.

[0032] We will now discuss the recovery point SEI message in HEVC. The mechanism used in AVC and HEVC for sending messages in the bitstream that are not strictly required for the decoding process but can help the decoder in various ways is the Supplemental Enhancement Information (SEI) message. SEI messages are signaled in SEI NAL units and are not normative for decoder parsing.

[0033] One type of SEI message defined by HEVC and AVC is the recovery point SEI message. The recovery point SEI message is sent at the location (at the picture) where the recovery period starts in the bitstream. When the decoder tunes to the bitstream, it can decode all pictures in decoding order from that location without outputting them until it reaches the recovery point picture, from which all pictures should be fully refreshed and can be output.

[0034] The syntax for recovery point SEI in HEVC is Figure 6 In Figure 6 In the recovery_point_cnt, the recovery point picture can be specified, and the decoder can start outputting pictures from this recovery point picture.

[0035] Still reference Figure 6 , exact_match_flag equal to 1 may specify that the recovery point picture generated from tuning to the recovery point exactly matches the recovery point picture as if the bitstream was decoded starting from the previous IRAP picture. exact_match_flag equal to 0 may specify that the recovery point picture should be substantially the same as if the bitstream was decoded starting from the previous IRAP picture, but may not be an exact match.

[0036] The broken link flag is used to indicate whether there is a broken link at the location of the SEI message in the bitstream. If the broken link flag is set equal to 1, the picture generated by starting decoding at the location of the previous IRAP picture may contain undesirable visual artifacts that should not be displayed before the picture is restored.

[0037] The work on recovery points in JVET will now be discussed. At the 11th JVET meeting in Ljubljana, an ad hoc group (AHG14) was established to study recovery points for VVCs.

[0038] At the 12th meeting in Macau, China in October 2018, the following two proposals were discussed:

[0039] In JVET-L0079, it is first discussed what non-normative changes are needed to the coding tools on the encoder side to be able to achieve exact matching using the recovery point SEI message in HEVC. The coding tools discussed are Advanced Temporal MV Prediction (ATMVP), Intra Prediction, Intra Block Copy, Inter Prediction, and Loop Filters (including Sample Adaptive Offset (SAO) filter, Deblocking Filter, and Adaptive Loop Filter (ALF)). The document also discusses some normative changes that can be applied to the coding tools to improve compression efficiency.

[0040] JVET-L0161 proposes to signal information about intra refresh in SPS, PPS and at slice level. The intra refresh information signaled in SPS / PPS includes a flag for enabling the intra refresh tool, the intra refresh mode (column, row, pseudo), the size of the intra refresh pattern (e.g., the width of a column or the length of a row), and the intra refresh increment QP. The intra refresh information signaled at the slice level includes the intra refresh direction (right to left / left to right / top to bottom / bottom to top) and the intra refresh position, the intra refresh direction is to be used to determine the motion vector constraint, and the intra refresh position specifies the position of the intra refresh block given by the size of the intra refresh pattern. If the CU belongs to the intra refresh area, the intra refresh pattern is derived at the picture level based on the intra refresh position value.

[0041] At the 13th meeting in Marrakech in January 2019, the input document JVET-M0529 proposed the use of NAL unit types to indicate recovery points instead of SEI messages as in HEVC and AVC. The proposed syntax consists of only one codeword, recovery_poc_cnt, similar to that used in HEVC. The proposed syntax is Figure 7 , which shows the recovery point NAL unit syntax proposed in JVET-M0529.

[0042] A decoding process is also proposed to start decoding at the recovery point. The proposed process includes defining the RPB access unit as the access unit associated with the recovery point NAL unit and the following:

[0043] If the RPB access unit containing the RPI NAL unit is not the first access unit in the CVS and the random access operation is not initialized at the RPB access unit, the RPI NAL unit in the RPB access unit shall be ignored.

[0044] Otherwise, if the RPB access unit containing the RPI NAL unit is the first access unit in the CVS, or a random access operation is initialized at the RPB access unit, the following applies:

[0045] o The decoder shall generate all reference pictures included in the RPS.

[0046] oWhen exporting the PicOrderCntVal of the RPB picture, the poc_msb_cycle_val of the RPB picture should be set to 0

[0047] oRPB pictures and all pictures following the RPB picture in decoding order shall be decoded.

[0048] o The RPB picture and all pictures following the RPB picture in decoding order up to (but not including) the recovery point picture shall not be output.

[0049] o Any SPS or PPS RBSP referenced by a picture in an RPB access unit or by any picture following that picture in decoding order shall be available to the decoding process before its activation.

[0050] This process implies that JVET-M0529 proposes that CVS can be started at the recovery point.

[0051]

[0013] Notwithstanding the above discussion of decoding of pictures, there still exists a need for improved recovery point handling in encoding and decoding. Summary of the invention

[0052] According to various embodiments of the inventive concept, a method for decoding a set of pictures from a bitstream is provided. The method includes identifying a recovery point in a bitstream according to a recovery point indication. The recovery point specifies a starting position in the bitstream for decoding the set of pictures. The set of pictures includes a first picture, which is the first picture in the set of pictures after the recovery point indication in a decoding order, and wherein the set of pictures includes coded picture data. The method also includes decoding the recovery point indication to obtain a set of decoded syntax elements. The recovery point indication includes a set of syntax elements. The method also includes deriving information for generating a set of unavailable reference pictures from the set of decoded syntax elements before the decoder parses any coded picture data. The method also includes generating a set of unavailable reference pictures based on the derived information. The method also includes decoding the set of pictures after generating the set of unavailable reference pictures.

[0053] In some embodiments, decoding of the set of pictures is initialized at a recovery point, and the method further includes determining a position of the first picture in the set of pictures. The method further includes determining a position of the second picture in the set of pictures. The method further includes decoding the first picture and all other pictures in the set of pictures that precede the second picture in decoding order during the recovery period without outputting the decoded pictures. The method further includes decoding and outputting the second picture.

[0054] In some embodiments, the method further includes performing a random access operation at the recovery point.

[0055] In some embodiments, the method further comprises rendering each picture in the set of pictures for display on a screen based on decoding the pictures from the bitstream after generating the set of unavailable reference pictures.

[0056] In some embodiments, the method further comprises receiving the bitstream from a remote device via a radio and / or network interface.

[0057] Corresponding embodiments of the inventive concept for a decoder and a computer program are also provided.

[0058] According to other embodiments of the present inventive concept, a method for encoding a recovery point indication into a bitstream is provided, the recovery point indication having information about how to generate an unavailable reference picture. The method includes encoding a first set of pictures into the bitstream. The method also includes determining a set of reference pictures that will be unavailable to the decoder if decoding in the bitstream is started after the first set of pictures. The method also includes encoding the recovery point indication into the bitstream. The recovery point indication includes a set of syntax elements for the set of reference pictures. The method also includes encoding a second set of pictures into the bitstream. At least one picture in the second set of pictures references a picture from the first set of pictures.

[0059] Corresponding embodiments of the inventive concept for a decoder and a computer program are also provided.

[0060] According to other embodiments of the present inventive concept, a method for encoding a recovery point indication into a bitstream is provided, the recovery point indication having information about how to generate an unavailable reference picture. The method includes encoding a first set of pictures into the bitstream. The method also includes determining a set of reference pictures that will be unavailable to the decoder if decoding in the bitstream is started after the first set of pictures. The method also includes encoding the recovery point indication into the bitstream. The recovery point indication includes a set of syntax elements for the set of reference pictures. The method also includes encoding a second set of pictures into the bitstream. At least one picture in the second set of pictures references a picture from the first set of pictures.

[0061] Corresponding embodiments of the inventive concept for an encoder and a computer program are also provided.

[0062] In some approaches, the generation of reference pictures can be done when it is known which reference pictures should be present in the RPS. This can be derived when the slice header (for HEVC) or tile group header (for draft VVC) is decoded. This means that reference pictures cannot be generated before the slice header or tile group header is received.

[0063] Various embodiments of the present disclosure may provide solutions to these and other potential problems. In various embodiments of the present disclosure, information may be added to a recovery point so that the information in the recovery point is sufficient to generate a reference picture for random access of the recovery point. As a result, the generation of the picture may be completed before a slice header or a tile group header is received. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The accompanying drawings illustrate certain non-limiting embodiments of the inventive concept, and are included to provide a further understanding of the present disclosure, and are incorporated in and constitute a part of this application. In the drawings:

[0065] Figure 1 shows the syntax of the NAL unit header in HEVC;

[0066] Figure 2 shows the NAL unit types in VCC;

[0067] Figure 3 An exemplary tile partitioning is shown;

[0068] Figure 4 shows a progressive decoding refresh over five pictures;

[0069] Figure 5 An example of using restrictions in tiles for GDR is shown. Tile borders are shown in bold. One tile is used for clean areas and one tile is used for dirty areas;

[0070] Figure 6 HEVC recovery point SEI NAL unit syntax is shown;

[0071] Figure 7 The recovery point NAL unit syntax proposed in JVET-M0529 is shown;

[0072] Figure 8 An example of a reference structure for low-latency video according to some embodiments of the inventive concept is shown;

[0073] Fig. 9An example of generating an unavailable reference picture according to information in recovery point indication data according to some embodiments of the inventive concept is shown;

[0074] Fig.10 An example bitstream with a recovery point indication NAL unit is shown. According to some embodiments of the inventive concept, the NAL unit header is marked in gray;

[0075] Fig.11 An example of a recovery point indication RBSP syntax according to some embodiments of the present inventive concept is shown;

[0076] Fig.12 Examples of NAL unit type codes and NAL unit type categories according to some embodiments of the inventive concept are shown;

[0077] Fig.13 An example of the syntax of a recovery point indication in a picture header according to some embodiments of the inventive concept is shown;

[0078] Fig.14 An example of a recovery point indication as a NAL unit type in a VCL NAL unit according to some embodiments of the inventive concept is shown;

[0079] Fig.15A An example of syntax for the content of a set of recovery point indication syntax elements for recovery point indication, which is signaled in an SEI message, according to some embodiments of the inventive concept is shown;

[0080] Fig. 15B An example syntax for signaling a set of recovery point indication syntax elements in a PPS according to some embodiments of the inventive concept is shown;

[0081] Figures 16 to 20 is a flowchart illustrating the operation of a decoder according to some embodiments of the inventive concept;

[0082] Fig.21 is a block diagram of a decoder according to some embodiments of the inventive concept;

[0083] Fig. 22 is a block diagram of an encoder according to some embodiments of the inventive concept; and

[0084] Fig.23 is a flowchart illustrating the operation of an encoder according to some embodiments of the inventive concept. DETAILED DESCRIPTION

[0085] Hereinafter, the inventive concept will be described more fully with reference to the accompanying drawings, in which examples of embodiments of the inventive concept are shown. However, the inventive concept can be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided so that the disclosure will be comprehensive and complete, and the scope of the inventive concept will be fully conveyed to those skilled in the art. It should also be noted that these embodiments are not mutually exclusive. A component from one embodiment can be assumed by default to be present in / used in another embodiment.

[0086] The following description presents various embodiments of the disclosed subject matter. These embodiments are presented as teaching examples and are not to be construed as limiting the scope of the disclosed subject matter. For example, some details of the embodiments may be modified, omitted or expanded without departing from the scope of the subject matter.

[0087] As discussed herein, various embodiments are applicable to controllers in encoders and decoders, such as Figures 8 to 23 shown. Fig.21 is a schematic block diagram of a decoder according to some embodiments. The decoder 2100 comprises an input unit 2102 configured to receive an encoded video signal. Fig.21 It is shown that according to various embodiments described herein, the decoder is configured to decode a set of pictures in a bitstream. In addition, the decoder 2100 includes a processor 2104 (also referred to herein as a controller or processor circuit or processing circuit) for implementing the various embodiments described herein. The processor 2104 is coupled to an input (IN) and a memory 2106 (also referred to herein as a memory circuit) coupled to the processor 2104. The decoded and reconstructed video signal obtained from the processor 2104 is output from the output (OUT) 2110. The memory 2106 may include a computer-readable program code 2108, which causes the processor to perform operations according to the embodiments disclosed herein when executed by the processor 2104. According to other embodiments, the processor 2104 may be defined as including a memory so that a separate memory is not required.

[0088] The processor 2104 is configured to decode a set of pictures from a bitstream. The processor 2104 may identify a recovery point in the bitstream according to a recovery point indication. The recovery point may specify a starting position in the bitstream for decoding the set of pictures. The set of pictures may include the first picture in the set of pictures after the recovery point indication in the decoding order, and the set of pictures may include coded picture data. The processor 2104 may decode the recovery point indication to obtain a set of decoded syntax elements. The recovery point indication may include a set of syntax elements. The processor 2104 may derive information for generating a set of unavailable reference pictures from the set of decoded syntax elements before the decoder parses any coded picture data. The processor 2104 may generate a set of unavailable reference pictures based on the derived information, and may decode the set of pictures after generating the set of unavailable reference pictures. In addition, modules may be stored in the memory 2106, and these modules may provide instructions so that when the processor 2104 executes the instructions of the modules, the processor 2104 performs corresponding operations (e.g., the operations discussed below with respect to the example embodiments involving the decoder).

[0089] The decoder with its processor 2104 can be implemented in hardware. A variety of variants of circuit elements can be used and combined to implement the functions of the units of the decoder. Such variants are covered by various embodiments. A specific example of a hardware implementation of the decoder is an implementation in digital signal processor (DSP) hardware and integrated circuit technology, including both general electronic circuits and dedicated circuits.

[0090] Fig. 22 is a schematic block diagram of an encoder according to some embodiments. The encoder 2200 comprises an input unit 2202 configured to send an encoded video signal. Fig. 22 It is shown that according to various embodiments described herein, the encoder is configured to encode a set of pictures into a bitstream. In addition, the encoder 2200 includes a processor 2204 (also referred to herein as a controller or processor circuit or processing circuit) for implementing the various embodiments described herein. The processor 2204 is coupled to an input (IN) and a memory 2206 (also referred to herein as a memory circuit) coupled to the processor 2204. The encoded video signal from the processor 2204 is output from the output (OUT) 2210. The memory 2206 may include a computer-readable program code 2208, which causes the processor to perform operations according to the embodiments disclosed herein when executed by the processor 2204. According to other embodiments, the processor 2204 may be defined as including a memory so that a separate memory is not required.

[0091] The processor 2204 is configured to encode a recovery point indication into a bitstream, the recovery point indication having information about how to generate an unavailable reference picture. The processor 2204 may encode a first set of pictures into the bitstream. The processor 2204 may determine a set of reference pictures that will be unavailable to the decoder if decoding in the bitstream is started after the first set of pictures. The processor 2204 may encode the recovery point indication into the bitstream. The recovery point indication may include a set of syntax elements for a set of reference pictures. The processor 2204 may encode a second set of pictures into the bitstream. At least one picture in the second set of pictures may reference a picture from the first set of pictures. In addition, modules may be stored in the memory 2206, and these modules may provide instructions so that when the processor 2204 executes the instructions of the modules, the processor 2204 performs corresponding operations (e.g., the operations discussed below with respect to example embodiments involving a decoder).

[0092] The encoder with its processor 2204 can be implemented in hardware. A variety of variations of circuit elements can be used and combined to implement the functions of the units of the decoder. Such variations are covered by various embodiments. A specific example of a hardware implementation of the encoder is an implementation in digital signal processor (DSP) hardware and integrated circuit technology, including both general electronic circuits and dedicated circuits.

[0093] We will now discuss Intra Random Access Point (IRAP) pictures. A coded picture that does not reference other pictures other than itself for prediction. This means that the coded picture contains only intra coded blocks. IRAP pictures can be used for random access.

[0094] Recovery points will now be discussed. A recovery point may be a location in the bitstream where a random access operation can be performed in the absence of any IRAP pictures.

[0095] The recovery point period will now be discussed. The recovery point period may be the period of the recovery point, for example, the period from the first picture refreshed to the last picture of the video fully decoded when performing a recovery point random access operation.

[0096] A Gradual Decoding Refresh (GDR) picture may be the first picture in a recovery point cycle. Refresh starts from this picture. Recovery point random access operations start by decoding this first picture. In this description, the term Recovery Point Start (RPB) picture may also be used interchangeably with a GDR picture.

[0097] The recovery point picture may be the last picture in the recovery point cycle. When the picture, the GDR picture, and the previous pictures in the recovery point cycle are decoded, the video is completely refreshed.

[0098] In some embodiments of the inventive concept, the recovery point indication described in the present disclosure may include: 1) the location of the start of the recovery point period, such as the location of the GDR picture that initiates the refresh, and 2) the location of the end of the recovery point period, such as the identification or location of the recovery point picture, where the video has been completely refreshed. In some embodiments, the location of the GDR picture can be explicitly signaled, for example, by a syntax element of the picture or a syntax element included in an access unit of the picture. In some embodiments, the recovery point indication is preferably signaled at the location of the GDR picture in the bitstream, and the location of the recovery point picture is explicitly signaled together with the recovery point indication.

[0099] The location of the recovery point picture can be signaled by information sent with the GDR picture or in the access unit of the GDR picture, for example, by information signaled so that the decoder can derive the ID of the recovery point picture when the GDR picture or GDR access unit is decoded. The decoder can then check for a match with the ID while decoding the picture that follows the GDR picture in decoding order, and the matching picture is identified as the recovery point picture. The derived ID can be a frame number, a picture sequence count number, a decoding order number, or any other number that the decoder derives for the decoded picture and can serve as a picture identifier.

[0100] When tuning to the bitstream at a recovery point, i.e., performing random access to the bitstream, or if the bitstream starts from a recovery point, the decoder first locates the canonical recovery point indication in the bitstream. Since the encoder knows that the decoder should support recovery points, the bitstream can be encoded using recovery points as the only type of random access points, which will enable random access operations while meeting low latency requirements. After the recovery point indication has been located, the start and end of the recovery point period are identified before decoding pictures in the recovery point period starting from the GDR picture. Pictures in the recovery point period should not be output except for the recovery point pictures. Starting from the recovery point picture, the pictures are decoded and output normally.

[0101] In another embodiment, the recovery point picture is the last picture not output in decoding order. In this case, the decoder does not output any pictures, including the recovery point picture, during the recovery point period, but begins outputting pictures after the recovery point picture in decoding order. Here, the recovery point picture may not be completely refreshed, and pictures after the recovery point picture may be completely refreshed.

[0102] When the decoder tunes to the bitstream at the recovery point, pictures referenced by pictures in the recovery point period may be unavailable if they are located before the recovery point indication in decoding order, see the figure below. Generation of unavailable pictures may include allocating memory for the picture, setting the block size in the picture to a specific value, setting the POC value of the picture to a specific value, etc. When generating an unavailable reference picture, each value in the sample array of the picture may be set to a specific value, such as medium gray, and each prediction mode of the reference picture may be set to an intra mode.

[0103] Figure 8 An example of a reference structure for low-delay video according to the inventive concept is shown. Figure 8 , pictures 2-4 are included in a recovery point period associated with a recovery point, where picture 2 is a GDR picture and picture 4 is a recovery point picture. If decoding is started at a recovery point, pictures 0 and 1 referenced by pictures in the recovery point period are not available and need to be generated.

[0104] In the method proposed in JVET-M0529, the generation of reference pictures can only be done when it is known which reference pictures should be present in the RPS. This is derived when decoding the slice header (for HEVC) or the tile group header (for draft VVC). This means that the generation of reference pictures cannot be done before receiving the slice header or tile group header.

[0105] In some embodiments of the present disclosure, the above method proposed in JVET-M529 may be improved by adding information to the recovery point NAL unit so that the information in the recovery point NAL unit is sufficient for generating necessary reference pictures for random access of the recovery point.

[0106] In some embodiments, the generation of reference pictures may be performed in advance. Instead of waiting for slice or tile group NAL units to be received and parsed, the decoder may generate reference pictures when the recovery point NAL units have been received and parsed.

[0107] In some embodiments, the generation of reference pictures in the decoder can be simplified. The decoder can first decode the recovery point NAL unit, second prepare the decoder by allocating the necessary reference pictures, and third decode the RPB picture without knowing whether the RPB picture should be regarded as a random access picture. The method discussed above with respect to JVET-M0529 is more complicated because the decoder will first decode the recovery point NAL unit, second decode the slice or tile group header of the RPB picture, third allocate reference pictures, and fourth decode the remaining coded slice data of the RPB picture. Since the allocation of reference pictures is done when the RPB NAL unit is decoded, the decoder needs to know whether the RPB picture should be regarded as a random access picture.

[0108] In some embodiments, the number of lines required to describe the decoding process in the specification may be significantly less than the number of lines required for the method described in JVET-M0529.

[0109] An exemplary embodiment of the inventive concept, Embodiment 1, will now be discussed with respect to generating and initializing a set of unavailable reference pictures before decoding any picture data. In this embodiment, the generation and initialization of unavailable reference pictures is completed before the decoding of any picture data begins. This is in contrast to JVET-M0529, where the generation and initialization are completed after the parsing of the VCL NAL unit of the RPB picture has begun. This is done in JVET-M0529 because the generation and initialization of unavailable reference pictures depends on information from the header in the VCL NAL unit. The header in the VCL NAL unit is here a fragment header, such as a slice header, a tile group header, etc. In JVET-M0529, the generation of unavailable reference pictures is based on a set of derived picture identifiers that are referenced by the RPB pictures and signaled in the slice or tile group header of the RPB pictures. In this embodiment, the generation of unavailable reference pictures is based on explicit signaling of all necessary properties of the reference pictures to be generated, wherein the explicit signaling is separate from the signaling of a set of picture identifiers referenced by the RPB picture, and wherein the explicit signaling is located earlier in decoding order than the signaling of reference picture identifiers for the RPB picture. In a preferred version of this embodiment, the set of unavailable reference pictures to be generated includes all reference pictures signaled in the RPL of the RPB picture.

[0110] According to some embodiments, Fig. 9 The generation of unavailable reference pictures according to the information in the recovery point indication data is shown in FIG, where the set of blocks in the top of the figure shows the encoded video bitstream. Fig. 9The invention shows that unavailable reference pictures are generated according to information in recovery point indication data according to some embodiments of the present inventive concept. Each large block is coded picture data, and the small block is recovery point indication data, and the recovery point indication data includes syntax elements with information about how to generate and initialize unavailable reference pictures referenced by pictures in the associated recovery point period. The recovery point period is marked with a dotted rectangle. An example reference structure of a picture is shown at the bottom of the figure. The recovery point period contains a recovery point indication and pictures 3, 4, 5, and 6, where picture 3 is an RPB picture. When random access is performed at the recovery point indication, reference pictures 0, 1, and 2 are unavailable to the decoder. These unavailable reference pictures are generated and initialized using information in the recovery point indication data. In a preferred version of this embodiment, all unavailable reference pictures are generated and initialized before decoding of the RPB picture (picture 3 in the example) is started. The generation and initialization are completed before the decoder parses any data of the RPB picture. Note that in a preferred version of this embodiment, picture 0 is also generated and initialized before decoding of the RPB picture is started, and picture 0 is not referenced by the RPB picture in the recovery point period but by picture 4 in the recovery point period. For the current version of the VVC draft, all pictures 0, 1 and 2 will be signaled in the reference picture list (RPL) in the tile group header of picture 3.

[0111] In some embodiments of the inventive concept, a decoder may perform the following operations to perform a random access operation on a bitstream:

[0112] 1. Identify and decode a recovery point indication in a bitstream, where the recovery point indication includes a set S of recovery point indication syntax elements.

[0113] 2. Then, information for generating and initializing a set of unavailable reference pictures is derived from the set S, and the set of unavailable reference pictures is generated and initialized based on the information.

[0114] 3. After generating and initializing the set of unavailable reference pictures, start decoding the first coded picture after the recovery point indication in decoding order.

[0115] 4. Then decode the pictures that follow the first encoded picture in decoding order.

[0116] In some embodiments of the present inventive concept, an encoder may perform the following operations to encode a recovery point indication into a bitstream, the recovery point indication having information on how to generate an unavailable reference picture:

[0117] 1. Encode a first set of pictures into a bitstream.

[0118] 2. Determine the set of reference pictures that will not be available to the decoder if decoding will start after the first set of pictures.

[0119] 3. Encode the recovery point indication including the syntax element of the reference picture into the bitstream.

[0120] 4. Encoding a second set of pictures into a bitstream, wherein at least one picture in the second set of pictures references a picture from the first set of pictures.

[0121] Now, another exemplary embodiment according to the inventive concept is discussed, which is Embodiment 2, and relates to the content of the recovery point indication syntax element. The content of the set S of the recovery point indication syntax element and the related operations that can be performed are discussed.

[0122] Now let's discuss parameter set indication. In some embodiments, set S may include one or more syntax elements that specify at least one parameter set identifier. The decoder will decode the parameter set identifier and use the identifier to identify parameter set P by comparing the decoded parameter set identifier with the parameter set ID associated with the decoded and stored parameter set. Parameter set P may have been decoded before set S is decoded or parsed. The decoder can then use information from parameter set P to generate and initialize unavailable reference pictures. This information may include, for example, bit depth, chroma subsampling type (e.g., 4:4:4 and 4:2:0), and picture width and height.

[0123] In some embodiments, there may also be a link to multiple parameter sets. Using HEVC as a non-limiting example, the decoder may have stored multiple picture parameter sets and sequence parameter sets (PPS and SPS), each with a separate ID value. Set S may contain a PPS value for identifying a stored PPS with a matching ID. The PPS may contain an SPS identifier value, and the decoder may use it to identify a stored SPS with a matching ID. The decoder may then use information from the identified SPS to generate and initialize unavailable reference pictures.

[0124] Now let's discuss the number of unavailable reference pictures. In some embodiments, the set S may contain one or more codewords that specify the number N of unavailable reference pictures to be generated and initialized. The decoder will decode this number and generate and initialize this number of reference pictures. The set S may contain other codewords that appear N times in the set S, where each value of these other codewords can specify the properties of the associated unavailable reference picture. For example, there may be a codeword in S that specifies that there are 2 unavailable reference pictures. In this example, the set S then also contains two occurrences of the picture type codeword, where the first occurrence can specify the picture type of the first unavailable reference picture and the second occurrence can specify the picture type of the second unavailable reference picture.

[0125] Now discuss the explicit picture order count value for each unavailable reference picture. In some embodiments, the set S may contain one or more syntax elements that specify the explicit picture order count value for each unavailable reference picture. This is preferably combined with the number of unavailable reference pictures, so that the set S may first specify the number of unavailable pictures N, and then contain an explicit picture order count value for each of the N pictures. The decoder may assign the explicit picture order count value to the corresponding unavailable reference picture, for example by setting the variable PicOrderCntVal of the corresponding unavailable picture to the value of the explicit picture order count value decoded from the set S.

[0126] In some embodiments, the explicit picture order count value may be signaled in the set S as a signed UVLC value. Alternatively, the explicit picture order count value may be signaled as a combination of two codewords, where a first codeword may specify the X least significant bits of the explicit picture order count value and a second codeword may specify the Y most significant bits. The derived explicit picture order count value may then be equal to X+Y*2^z, where X is a fixed length codeword with length z. Additionally, there may be a one-bit third codeword to specify the sign of the derived explicit picture order count value.

[0127] Now discuss deriving the picture order count value of each unavailable reference picture from the picture order count value of the GDR picture. In some embodiments, the set S may include one or more syntax elements specifying the explicit picture order count value of the GDR picture, for example, according to any method discussed above regarding the explicit picture order count value of each unavailable reference picture. The set S then includes one or more codewords for each unavailable reference picture, which represent the incremental picture order count value. The decoder then derives the picture order count value of a specific unavailable reference picture by adding the corresponding incremental picture order count value to the picture order count value of the GDR picture. The incremental picture order count may be signaled by a method similar to any method for signaling an explicit picture order count value, such as the method discussed above regarding the explicit picture order count value of each unavailable reference picture.

[0128] Now discuss the picture tag of each unavailable reference picture. In some embodiments, the set S may include one or more syntax elements that specify a picture tag value for each unavailable reference picture. The picture tag value may indicate whether the corresponding unavailable reference picture is a short-term reference picture or a long-term reference picture. Optionally, the picture tag may alternatively indicate that the corresponding unavailable reference picture is a picture that is not used for prediction.

[0129] In some embodiments, the decoder may mark the corresponding unavailable reference picture with a marking value derived from S. The decoder may store the corresponding unavailable reference picture as marked with a marking value derived from S in a decoded picture buffer.

[0130] Now discuss the common width and height of unavailable reference pictures. Set S may contain one or more syntax elements that specify a picture width value and a picture height value. These values ​​may be width and height values ​​in units of luma samples. In some embodiments, the decoder may generate and initialize all unavailable reference pictures to have picture width and height equal to the width and height values ​​derived from S.

[0131] Now discuss the individual width and height of each unavailable reference picture. In some embodiments, the set S may include one or more syntax elements that specify an individual width value and an individual height value for each unavailable reference picture. These values ​​may be width and height values ​​in units of luma samples. The decoder may generate and initialize a specific unavailable reference picture to have a picture width and height equal to the corresponding derived width and height values.

[0132] Now let's discuss the number of components and their characteristics. In some embodiments, the set S may include one or more codewords that specify the number M of components that the unavailable reference picture to be generated and initialized includes. The set S may include one or more codewords that specify the relative size of the components of the unavailable reference picture, such as the chroma subsampling type or the chroma array type. The set S may include one or more codewords that specify the bit depth of the component.

[0133] In some embodiments, the decoder may generate and initialize a specific unavailable reference picture to have a picture of a specified number of components, M. The decoder may derive the size of one or more components from the codewords in S, which specify the relative sizes of the components, for example by combining this information with the width and height of one particular component (e.g., a luma component) signaled elsewhere.

[0134] Now let's discuss picture types. In some embodiments, the set S may include one or more syntax elements that specify a picture type value for each unavailable reference picture. The decoder may assign a picture type value to a corresponding unavailable reference picture, for example, by setting a picture type variable of the corresponding unavailable picture to the value of the picture type value decoded from the set S for the unavailable picture.

[0135] In some embodiments, the picture type value may be one of the following non-limiting examples: trailing picture, non-STSA trailing picture, STSA picture, leading picture, RADL picture, RASL picture, IDR picture, CRA picture.

[0136] Now let's discuss the temporal ID. In some embodiments, the set S may include one or more syntax elements that specify a temporal ID value for each unavailable reference picture. The decoder may assign the temporal ID value to the corresponding unavailable reference picture, for example by setting the temporal ID variable of the corresponding unavailable picture to the value of the temporal ID value decoded from the set S for the unavailable picture.

[0137] Now let's discuss layer ID. In some embodiments, set S may include one or more syntax elements that specify a layer ID value for each unavailable reference picture. The decoder may assign a layer ID value to a corresponding unavailable reference picture, for example by setting a layer ID variable of the corresponding unavailable picture to the value of the layer ID value decoded from set S for the unavailable picture.

[0138] The picture parameter set (PPS) ID of each unavailable reference picture is now discussed. In some embodiments, the set S may include one or more syntax elements that specify a picture parameter set identifier for each unavailable reference picture. The decoder may assign a picture parameter set identifier to the corresponding unavailable reference picture, for example, by setting a picture parameter set identifier variable of the corresponding unavailable picture to the value of the picture parameter set identifier value decoded from the set S for the unavailable picture.

[0139] In some embodiments, there may be at least two unavailable reference pictures P1 and P2, so that the set S contains two corresponding picture parameter set identifiers I1 and I2, where the values ​​of I1 and I2 are different. This means that picture P1 is associated with one PPS and picture P2 is associated with another different PPS.

[0140] Now let's discuss block size.In some embodiments, the set S may contain one or more syntax elements that specify a block size (eg, a luma size of a coding tree unit and / or a chroma size of a coding tree unit).

[0141] In some embodiments, the decoder may assign block size values ​​to corresponding unavailable reference pictures, for example by setting a block size variable for each unavailable picture to the value of the decoded block size value. The decoder may generate and initialize at least one unavailable reference picture to have a block size equal to the corresponding block size value derived from S. The decoder may derive the number of blocks in the unavailable reference picture based on the size of the picture and the block size value of the picture, and assign at least one value, such as an intra mode, to each block in the unavailable reference picture.

[0142] Now discuss another exemplary embodiment of the present invention, embodiment 3, which involves, for example, a recovery point indication NAL unit. In some embodiments, the existence of a recovery point is indicated by a recovery point indication NAL unit, and a set S of recovery point indication syntax elements is located in the payload of the recovery point indication NAL unit.

[0143] In some embodiments, the indication is based on the presence of a recovery point indicating NAL unit, such that if a recovery point indicating NAL unit is present, then a recovery point is indicated. If a recovery point indicating NAL unit is not present, then no recovery point is indicated. Preferably, a non-VCL NAL unit type is used to indicate a recovery point, meaning that the NAL unit does not contain any video coding layer data.

[0144] Fig.10 An example of some embodiments according to the inventive concept is shown, which uses a recovery point indication (RPI) NAL unit to indicate a recovery point in a bitstream. Fig.10 , shows an example bitstream with a recovery point indication NAL unit. The NAL unit header is marked in grey.

[0145] Fig.10 An example of some embodiments according to the present inventive concept is shown, which uses a recovery point indication (RPI) NAL unit to indicate a recovery point in a bitstream. In this example, the bitstream contains one VCL NAL unit for each picture, such as one slice or one tile group for each picture. The recovery point indication NAL unit is placed before the VCL NAL unit containing the GDR picture that starts refreshing. In a preferred version of this embodiment, in the case of any SPS or PPS in the access unit, the recovery point indication NAL unit is placed before any SPS and PPS in the access unit. In other versions of the embodiment, the SPS and / or PPS are placed before the recovery point indication in the access unit or signaled out of band. The recovery point indication NAL unit may preferably be located before any VCL NAL unit in the access unit associated with the GDR picture. The access unit may be referred to as a recovery point access unit or a recovery point start (RPB) access unit. As in the example in 10, it may also be referred to as a random access point (RAP) access unit. A set S of recovery point indication syntax elements is located in the payload of the recovery point indication NAL unit (indicated by "RPI NAL unit" in 10).

[0146] Discussed below are example descriptions, syntax, and semantics on how to specify a recovery point indication as a NAL unit type on top of the latest VVC draft according to some embodiments of the inventive concept.

[0147] In some embodiments of the inventive concept, a decoder may perform the following operations to perform a random access operation on a bitstream:

[0148] 1. Identify and decode the recovery point indication NAL unit in the bitstream, which includes a set S of recovery point indication syntax elements.

[0149] 2. Derive information from the set S for generating and / or initializing a set of unavailable reference pictures.

[0150] 3. Generate a set of unavailable reference pictures based on this information.

[0151] 4. Initialize the set of unavailable reference pictures based on this information.

[0152] 5. After generating and / or initializing the set of unavailable reference pictures, start decoding the first coded picture after the recovery point indication in decoding order.

[0153] 6. Then decode the pictures that follow the first coded picture in decoding order.

[0154] The following is an example description, syntax and semantics of how some embodiments of the inventive concept may be specified on top of Versatile Video Coding (Draft 4) JVET-M1001-v1. Changes to some embodiments of the inventive concept are underlined.

[0155] ·3.18 Coded Video Sequence (CVS): Access unit sequence, composed of IRAP access units or RPB access unit Consists of, in decoding order, zero or more access units that are not IRAP access units, including all subsequent access units up to, but not including, any subsequent access unit that is an IRAP access unit.

[0156] · 3.74 recovery point: The point in the bitstream at which the next bit in the bitstream is an RPB access unit. The first bit .

[0157] · 3.75 Recovery Point Start (RPB) access unit: An access unit containing a recovery point indicator NAL unit. .

[0158] · 3.76 Recovery Point Start (RPB) picture: A coded picture in an RPB access unit. .

[0159] · 3.77 Recovery point period: A collection of pictures, including the RPB picture and all pictures after the RPB picture, until Up to and including the recovery point picture indicated by the recovery point indication NAL unit in the access unit containing the RPB picture .

[0160] · 3.78 Recovery point picture: The last coded picture in the decoding order in the recovery point period .

[0161] According to some embodiments of the present invention, the "7.3.2.5 Recovery Point Indication RBSP Syntax" section may be added to JVET-M1001-v1, for example, in the present disclosure. Fig.11 as shown in .

[0162] According to some embodiments of the inventive concept, the "7.4.2.2 NAL unit header semantics" section may be added on top of JVET-M1001-v1, for example, in the present disclosure. Fig.12 Changes to some embodiments according to the inventive concept are underlined.

[0163] In some embodiments of the present inventive concept, when nal_unit_type is equal to RPI_NUT, temporalId should be equal to 0.

[0164] The recovery point indication RBSP semantics according to some embodiments of the inventive concept will now be discussed.

[0165] In some embodiments of the inventive concept, the RPI NAL unit shall precede any VCL NAL unit in the access unit containing the RPI NAL unit. The RPI NAL unit shall follow any SPS or PPS NAL unit in the access unit containing the RPI NAL unit. The temporal Id of all VCL NAL units in the access unit containing the RPI NAL unit shall be equal to 0.

[0166] In some embodiments, if the RPB access unit containing the RPI NAL unit is not the first access unit in the CVS and the random access operation is not initialized at the RPB access unit, the RPI NAL unit in the RPB access unit shall be ignored.

[0167] In some embodiments, otherwise, if the RPB access unit containing the RPI NAL unit is the first access unit in the CVS, or a random access operation is initialized at the RPB access unit, the following applies:

[0168] - The decoder shall generate unavailable reference pictures according to the procedure described in 8.2.2.

[0169] - When deriving the PicOrderCntVal of an RPB picture, the poc_msb_cycle_val of the RPB picture shall be set to 0.

[0170] - The RPB picture and all pictures following the RPB picture in decoding order shall be decoded.

[0171] - The RPB picture and all pictures following the RPB picture in decoding order up to (but not including) the recovery point picture shall not but may be output.

[0172] - Any SPS or PPS RBSP referenced by a picture in an RPB access unit or any picture following that picture in decoding order shall be available to the decoding process before its activation.

[0173] In some embodiments of the inventive concept, a requirement for bitstream consistency may be that the decoded picture after the recovery point picture in decoding order should exactly match the picture that would be produced by starting the decoding process at the location of an IRAP or RPB access unit that precedes the RPB picture (if any) belonging to the same recovery point period as the recovery point picture in decoding order in the bitstream. In some embodiments of the inventive concept:

[0174] recovery_poc_cnt may specify the picture order count of the recovery point picture. A picture that follows the current picture in decoding order and has a PicOrderCntVal that is equal to the PicOrderCntVal of the current picture plus the value of recovery_poc_cnt is called a recovery point picture. The recovery point picture shall not precede the current picture in decoding order. The value of recovery_poc_cnt shall be in the range of -MaxPicOrderCntLsb / 2 to MaxPicOrderCntLsb / 2 − 1, inclusive.

[0175] rpi_pic_parameter_set_id may specify the value of pps_pic_parameter_set_id of the PPS in use. The value of rpi_pic_parameter_set_id shall be the same as the value of tile_group_pic_parameter_set_id of the tile group header of the coded picture in the RPB access unit.

[0176] number_of_reference_pictures may specify the number of reference pictures that should be generated if the RPI NAL unit is the first access unit in the CVS or a random access operation is initialized at the RPB access unit.

[0177] rpi_long_term_picture_flag[ i ] equal to 1 can specify that the i-th reference picture is a long-term picture. rpi_long_term_picture_flag equal to 0 can specify that the i-th reference picture is a short-term picture.

[0178] rpi_pic_order_cnt_val[ i ] may specify the PicOrderCntValue of the i-th generated unavailable reference picture.

[0179] A decoding process for generating an unavailable reference picture will now be discussed according to some embodiments of the inventive concept.

[0180] In some embodiments, this process is called for any RPB NAL unit in the bitstream if the corresponding RPB access unit is the first access unit in the CVS or a random access operation is initialized at an RPB access unit.

[0181] The following may apply:

[0182] – Set the SPS in use to the SPS with the value of sps_seq_parameter_set_id equal to the value of pps_seq_parameter_set_id of the PPS, and the value of pps_pic_parameter_set_id equal to the value of rpi_pic_parameter_set_id.

[0183] – For each i in the range 0 to number_of_reference_pictures − 1 (inclusive), an unusable picture is generated and the following applies:

[0184] – Set the value of PicOrderCntVal of the generated picture equal to rpi_pic_order_cnt_val[i].

[0185] – For the generated picture, the POC lsb value (of the variable tile_group_pic_order_cnt_lsb of the generated picture) is derived as PicOrderCntVal % MaxPicOrderCntLsb, where % is a modulo operation. This may be equivalent to assigning the POC lsb value equal to the n least significant bits of PicOrderCntVal, where n is equal to log2_max_pic_order_cnt_lsb_minus4 + 4.

[0186] – If rpi_long_term_picture_flag[ i ] is equal to 1, the generated picture is marked as “used for long-term reference”.

[0187] – If rpi_long_term_picture_flag[ i ] is equal to 0, the generated picture is marked as “used for short-term reference”.

[0188] – The variables BitDepthY, BitDepthC and ChromaArrayType are derived as specified in Section 7.4.3.1 for the SPS in use.

[0189] – Set the variable PicWidthInLumaSamples equal to the pic_width_in_luma_samples of the SPS in use.

[0190] – Set the variable PicHeightInLumaSamples equal to the pic_height_in_luma_samples of the SPS in use.

[0191] – Set the value of each element in the sample array SL for the generated picture equal to 1 << (BitDepthY − 1).

[0192] – When ChromaArrayType is not equal to 0, the value of each element in the sample arrays SCb and SCr for the generated picture is set equal to 1 << (BitDepthC − 1).

[0193] – For x = 0.. PicWidthInLumaSamples − 1, y = 0.. PicHeightInLumaSamples − 1, set the prediction mode CuPredMode[ x ][ y ] equal to MODE_INTRA.

[0194] Assuming the presence of the sps_max_dec_pic_buffering syntax element in the VVC specification, a decoder may choose to generate sps_max_dec_pic_buffering number of pictures instead of number_of_reference_pictures number of pictures.

[0195] Now discuss another exemplary embodiment according to some inventive concepts, embodiment 4, which relates to, for example, a method for generating and initializing an unavailable reference picture. Some embodiments of the inventive concepts may include assigning and / or allocating memory to store values ​​for unavailable reference pictures, wherein the stored values ​​include sample values ​​for each component of the picture.

[0196] In some embodiments, the contents of the elements described in Example Embodiment 2 may be used to determine the memory size that needs to be assigned / allocated for unavailable reference pictures, generate pictures for each reference picture in a set of unavailable reference pictures, and initialize each reference picture in a set of unavailable reference pictures.

[0197] In a preferred version of some embodiments, all unavailable reference pictures are generated and initialized before decoding of the first coded picture (eg, RPB picture) following the recovery point indication in decoding order.

[0198] In some embodiments of the present inventive concept, a decoder may perform the following operations to generate and initialize unavailable reference pictures when performing a random access operation on a bitstream:

[0199] 1. Identify and decode a recovery point indication in a bitstream, where the recovery point indication includes a set S of recovery point indication syntax elements.

[0200] 2. Derive information from the set S for generating and / or initializing a set of unavailable reference pictures.

[0201] 3. Determine the memory size required for the unavailable reference picture, wherein determining the memory size includes at least one of the following:

[0202] Number of unavailable reference images

[0203] The number of components per reference picture

[0204] The width and height of each component

[0205] The bit depth of samples in each component

[0206] 4. Assign and / or allocate memory for unavailable reference pictures based on the determined required memory size

[0207] 5. Generate a picture for each reference picture in the set of unavailable reference pictures, wherein the generating comprises at least one of the following:

[0208] Set the number of components of the image

[0209] For each component of the image, set the width and height

[0210] For each component of the picture, set the sample bit depth

[0211] For each sample in the image, set the sample value

[0212] Assigning PPS identifiers to reference pictures

[0213] Assigning SPS identifiers to reference pictures

[0214] Assign identifiers to reference pictures, such as picture sequence count values

[0215] Label reference images as short-term, long-term, or not used for prediction

[0216] Assigning image types to reference images

[0217] Assigning temporal IDs to reference pictures

[0218] Assign layer IDs to reference images

[0219] Assign a block size to each component

[0220] Mark each of the generated reference pictures as initialized

[0221] Another exemplary embodiment according to some inventive concepts, embodiment 5, is now discussed, which relates to recovery point indication in, for example, a picture header. The final version of VVC may include a picture header to efficiently encode header data that is the same between tile groups.

[0222] In some embodiments of the inventive concept, the content of a set S of recovery point indication syntax elements of the recovery point indication is signaled in such a picture header. Fig.13 Example syntax and semantics for this are shown according to some embodiments of the inventive concept. Fig.13 , recovery_point_start_flag equal to 1 can specify that the current picture is the first picture of the recovery point. The last picture of the recovery point is specified by recovery_poc_cnt. recovery_point_start_flag equal to 0 can specify that the current picture is not the first picture of the recovery point.

[0223] In this example according to some inventive concepts, the semantics of recovery_poc_cnt, rpi_pic_parameter_set_id, number_of_reference_pictures, rpi_long_term_picture_flag[i], and rpi_pic_order_cnt_val[i] are the same as described in Exemplary Embodiment 3.

[0224] A potential disadvantage of specifying the recovery point indication in the picture header may be that it may not be well exposed to the system layer. One way to make it more accessible to the system layer may be to use fixed length encoding for the recovery point syntax and syntax elements preceding the recovery point syntax and / or to place the recovery point syntax element at the beginning of the picture header.

[0225] In some embodiments of the inventive concept, the indication that the current picture is the first picture of the recovery point (recovery_point_start_flag in the above example) is signaled by some other means (eg, as nal_unit_type in a VCL NAL unit as in exemplary embodiment 6).

[0226] Discussing now another exemplary embodiment of the inventive concept, embodiment 6, it relates to a recovery point indication, for example, as a NAL unit type in a VCL NAL unit. In some embodiments of the inventive concept, the indication of the recovery point indication is signaled as a NAL unit type in a VCL NAL unit. In some embodiments, two new NAL unit types may be defined; a picture type NON_IRAP_RPI_BEGIN, which indicates the start of a recovery point period, and a NAL unit type NON_IRAP_RPI_END, which indicates the end of a recovery point period.

[0227] According to some embodiments of the present inventive concept, the example specification text above the current VVC draft is Fig.14 Shown in.

[0228] In some embodiments, the POC of the recovery point picture does not need to be explicitly signaled.

[0229] To fully support temporal layers, NON_IRAP_RPI_BEGIN_NUT and NON_IRAP_RPI_END_NUT should be restricted to not be set for pictures in different temporal layers.

[0230] In some embodiments, a benefit may be provided for easy access to recovery point information at the system level. A potential problem with this approach is that the recovery point indication becomes dependent on the picture type. In order to allow recovery points in pictures with different picture types, NAL unit types for all combinations or subsets of combinations may be required. This may include whether it is desired to support recovery points starting from the same picture as the picture at the end of the previous recovery point period. In order to be able to support overlapping recovery points, a mechanism may be needed to map the end of the recovery point period to the correct start of the new recovery point.

[0231] In some embodiments of the present inventive concept, at least one of information about the end of the recovery point period (e.g., the POC of the recovery point picture), the content of the set S of the recovery point indication syntax element, and other information related to the recovery point is signaled by other means (e.g., in a picture header described in Embodiment 5, in an SPS or PPS as described with respect to exemplary embodiment 7, or in a tile group header). Therefore, if the end of the recovery point period is signaled by other means, only the start of the recovery point (NON_IRAP_RPI_BEGIN_NUT) is signaled as a NAL unit type.

[0232] Now discuss another exemplary embodiment of the inventive concept, embodiment 7, which involves signaling information about generating reference pictures in, for example, SEI, PPS or SPS. In some embodiments, the content of a set S of recovery point indication syntax elements of the recovery point indication is signaled in the SEI message. Fig.15A Example syntax for this is shown according to some embodiments of the inventive concept.

[0233] exist Fig.15A In the example of , the semantics of recovery_poc_cnt, rpi_pic_parameter_set_id, number_of_reference_pictures, rpi_long_term_picture_flag[i] and rpi_pic_order_cnt_val[i] are the same as described with respect to exemplary embodiment 3.

[0234] In another embodiment, the content of a set S of recovery point indication syntax elements of which the recovery point indication is signaled in the SPS or PPS. Fig. 15B An example syntax for signaling a set S of recovery point indication syntax elements in a PPS according to some embodiments of the inventive concept is shown. In this exemplary embodiment, the semantics of recovery_poc_cnt, rpi_pic_parameter_set_id, number_of_reference_pictures, rpi_long_term_picture_flag[i], and rpi_pic_order_cnt_val[i] are the same as described with respect to exemplary embodiment 3.

[0235] The indication of where the recovery point period starts should preferably be signaled by other means than parameter sets, since parameter sets may be valid for multiple pictures. The indication of where the recovery point period starts may be signaled, for example, as a NAL unit type in a NAL unit header for a picture that has an active SPS or PPS containing additional recovery point information.

[0236] Another exemplary embodiment of the inventive concept is now discussed, embodiment 8, which involves, for example, starting the CVS with a reference point and generating unavailable reference pictures before decoding the RPB pictures. In some embodiments of the inventive concept, the CVS starts with a recovery point, where the unavailable reference pictures are generated and / or initialized before starting to decode the RPB pictures of the recovery point period, as described in any of the previous or following embodiments.

[0237] Defining a recovery point indication in a normative way, e.g. in a non-VCL NAL unit or as nal_unit_type in a VCL NAL unit, enables a CVS to start with a recovery point. This may be useful after segmenting a low-latency coded bitstream that uses recovery point coding to support random access.

[0238] In the current VVC draft, CVS is defined as follows, where the inventive concept is shown with underlining and strikethrough:

[0239] Access unit: A collection of NAL units that are associated with each other according to a specified classification rule, are consecutive in decoding order, and contain exactly one coded picture.

[0240] Coded video sequence (CVS): a sequence of access units consisting of an IRAP access unit, in decoding order, followed by zero or more access units that are not IRAP access units, including all subsequent access units up to, but not including, any subsequent access unit that is an IRAP access unit.

[0241] The following is example text for defining a CVS that allows a specification-specified recovery point to start the CVS for some embodiments of the present inventive concept, where the recovery point indicates that the access unit is an access unit associated with a GDR picture of the recovery point:

[0242] Coded Video Sequence (CVS): Access unit sequence, composed of IRAP access units or recovery point indicating access unit Consists of, in decoding order, zero or more access units that are not IRAP access units, including all subsequent access units up to, but not including, any subsequent access unit that is an IRAP access unit.

[0243] In some embodiments of the present invention, the resume point indicates that the access unit may also define the end of the CVS. An example text of the definition of the CVS is as follows:

[0244] Coded Video Sequence (CVS): Access unit sequence, composed of IRAP access units or recovery point indicating access unit In decoding order, followed by zero or more access units that are not IRAPs and is not a recovery point indicating access unit Access units, including up to IRAP access units Recovery Point Access Unit All subsequent access units of any subsequent access unit, but not including access units that are IRAP access units Recovery Point Access Unit Any subsequent access unit of .

[0245] A recovery point indication access unit may also be referred to by other names, such as a GDR access unit or a recovery point start (RPB) access unit.

[0246] In some embodiments of the present inventive concept, a random access point (RAP) access unit may be defined, which may include an IRAP picture or a GDR picture of a recovery point:

[0247] Coded video sequence (CVS): a sequence of access units, consisting of RAP Access units consist of, in decoding order, zero or more RAP Access unit of access unit, including up to as RAP All access units that follow any access unit that follows the access unit, but not including the access unit that is RAP Any subsequent access unit of an access unit.

[0248] Random access point (RAP) access unit: An access unit where a coded picture is an IRAP picture or where an access unit Contains restore point indication.

[0249] Now discuss another exemplary embodiment according to the inventive concept, embodiment 9, which relates to, for example, recovery points for a spatial subset of a picture. In some embodiments of the inventive concept, compared with some embodiments discussed above, the scope of the recovery point indication is not the entire picture, but a set of time-aligned fragments of the picture, where the fragment can be a tile, a tile group, a slice, etc. Therefore, the recovery point indication in this embodiment can specify when one or more fragments of the picture are completely refreshed.

[0250] In some embodiments, a recovery point indication is signaled just before each fragment, for example in a NAL unit or fragment header.

[0251] In another embodiment, the recovery point indication is signaled in the same container (eg, NAL unit, PPS, SPS, or picture header) for the entire picture, but the recovery period for each segment may have different start and / or end pictures.

[0252] In another embodiment, the signaled recovery point indication may include a start picture and an end picture of a recovery point period for the entire picture and a separate start picture and / or end picture of a recovery point period for each fragment.

[0253] In another embodiment, the recovery point indication includes a flag for determining whether the spatial extent is the entire picture or just a fragment in the bitstream.

[0254] In another embodiment, when a random access operation is initiated at a recovery point for a segment, only the spatial regions in the unavailable reference pictures that are co-located with the segment are generated.

[0255] In another embodiment, when a random access operation is initiated at a recovery point for a segment, a complete unavailable reference picture is generated.

[0256] The following is a further discussion of exemplary embodiments 1 to 9 according to the inventive concept.

[0257] Some embodiments of example embodiment 1 of generating and initializing a set of unavailable reference pictures before decoding any picture data may include:

[0258] 1. A method for decoding a video bitstream, the video bitstream comprising a coded video sequence (CVS) of pictures, the pictures comprising recovery points (e.g., at least one recovery point), wherein:

[0259] A recovery point is a position in the bitstream where decoding can start from a picture A containing at least one block that is not an intra-coded block.

[0260] Identify picture B that follows picture A in decoding order,

[0261] If decoding starts from picture A, and pictures after picture A and before picture B in decoding order and picture B are decoded, the video is completely refreshed at picture B, and the method includes:

[0262] obtaining (e.g., receiving) a video bitstream;

[0263] decoding an indication of a recovery point from a video bitstream;

[0264] deriving information for generating and initializing a set of (unavailable) reference pictures by decoding a set S of syntax elements from a bitstream;

[0265] generating and initializing a set of reference pictures according to information for generating and initializing a set of reference pictures; and

[0266] After generating and initializing the set of reference pictures, start decoding picture A

[0267] Some embodiments of exemplary embodiment 2 may include embodiment 1, wherein deriving information for generating and initializing a set of reference pictures by decoding a set S of syntax elements from a bitstream comprises: deriving (from S) and using one or more of the following information:

[0268] 1. Derive at least one parameter set identifier that identifies the parameter set active for picture A

[0269] 2. Export the number of reference images to be generated and initialized

[0270] 3. Derive the picture order count value for each reference picture and assign the derived picture order count value to the associated reference picture

[0271] 4. derive a picture order count value for picture A, derive an incremental picture order count for each reference picture relative to the picture order count of picture A, and use these derived values ​​to calculate a picture order count value for each reference picture, and assign the calculated picture order count values ​​to the associated reference pictures

[0272] 5. Derive the picture tag state of each reference picture, where the picture tag state is one of the long-term picture and the short-term picture (and an optional unpredicted tag state), and tag each reference picture with the derived tag state

[0273] 6. Export the brightness width value and brightness height value, and generate a reference image with the width and height

[0274] 7. Derive the luma width value and luma height value of each reference picture, and generate each reference picture to have a width and height with the associated derived luma width value and height value

[0275] 8. Derive the number of components of picture A, and the reference may include the relative size of the components (ChromaArrayType in HEVC) and the bit depth of each component or all components. Generate a reference picture with the number of components, relative size and bit depth according to the derived values.

[0276] 9. Derive the picture type value for each reference picture and assign the derived picture type value to the associated reference picture

[0277] 10. Derive the temporal ID value of each reference picture and assign the derived temporal ID value to the associated reference picture

[0278] 11. Derive the layer ID value of each reference picture and assign the derived layer ID value to the associated reference picture

[0279] 12. Derive at least one picture parameter set identifier for each reference picture and assign the derived at least one picture parameter set identifier value to the associated reference picture

[0280] 13. Derive the block size (e.g., the size of the coding tree unit), generate the reference picture to have the block size, and assign the block size to the reference picture

[0281] Some embodiments of exemplary embodiment 3 may include embodiments 1 and 2, wherein the set S of syntax elements is decoded from a non-VCL NAL unit having a non-VCL NAL unit type indicating that the non-VCL NAL unit is a recovery point indicating non-VCL NAL unit.

[0282] Some embodiments of example embodiment 4 may include embodiments 1 to 3, wherein generating and initializing a reference picture in a set of reference pictures comprises allocating or assigning memory to store values ​​for the picture, wherein the stored values ​​comprise sample values ​​for each component of the picture.

[0283] Some embodiments of Exemplary Embodiment 4 may further include a case where generating and initializing a reference picture in a set of reference pictures includes at least one of the following:

[0284] a. Set the number of components of the image

[0285] b. Set the width and height for each component of the image

[0286] c. For each component of the image, set the sample bit depth

[0287] d. For each sample in the image, set the sample value

[0288] e. Assigning PPS identifiers to reference pictures

[0289] f. Assign SPS identifiers to reference pictures

[0290] g. Assign identifiers to reference pictures, such as picture sequence count values

[0291] h. Mark reference images as short-term images, long-term images, or not used for prediction

[0292] i. Assign image types to reference images

[0293] j. Assigning a temporal ID to a reference picture

[0294] k. Assign layer IDs to reference images

[0295] l. Assign block size to each component

[0296] m. Mark each of the generated reference pictures as initialized

[0297] Some embodiments of exemplary embodiment 5 may include embodiments 1 to 4, wherein the set S of syntax elements is decoded from a picture header.

[0298] Some embodiments of example embodiment 6 may include embodiments 1 to 5, wherein at least one of the indication of the start of the recovery point and the indication of the end of the recovery point period is decoded from a NAL unit type syntax element in the VCL NAL unit.

[0299] Some embodiments of exemplary embodiment 7 may include embodiments 1 to 6, wherein the set S of syntax elements is decoded from a SEI message.

[0300] Some embodiments of example embodiment 7 may also include where the set S of syntax elements is decoded from a picture parameter set such as a PPS or SPS.

[0301] Some embodiments of exemplary embodiment 8 may include embodiments 1 to 7, wherein the CVS starts with a recovery point.

[0302] Some embodiments of exemplary embodiment 8 may also include where the CVS is a compliant portion of the bitstream that complies with a standard specification, such that a decoder that is required to comply with the standard specification is able to decode the CVS.

[0303] Some embodiments of exemplary embodiment 9 may include embodiments 1 to 8, wherein:

[0304] The recovery point indication is valid only for a spatial subset of the picture.

[0305] According to some embodiments of the present invention, reference will now be made to Fig.16 Flowchart to discuss (using Fig.21 The operation of the decoder 2100 is implemented by the structure of the block diagram. For example, the module can be stored in Fig.21 The modules may be stored in the memory 2106 of the wireless device, and these modules may provide instructions such that when the instructions of the modules are executed by the corresponding wireless device processing circuit 2104, the processing circuit 2104 performs the corresponding operations of the flowchart.

[0306] Fig.16 The operation of a decoder to decode a set of pictures from a bitstream is shown. Fig.21 The structure shown provides a decoder.

[0307] exist Fig.16At block 1600 of the decoder, a processor 2104 of the decoder identifies a recovery point in the bitstream according to the recovery point indication. The recovery point may specify a starting position in the bitstream for decoding a set of pictures. The set of pictures may include a first picture, which is the first picture in the set of pictures after the recovery point indication in decoding order, and the set of pictures includes coded picture data. The coded picture data includes data carrying coded samples, including headers accompanying the coded samples. In general, coded picture data refers to coded data that is packaged into data units (e.g., NAL units known from HEVC and VVC draft specifications). The coded picture data may include all data in a data unit or NAL unit carrying coded samples, including headers such as slice headers and / or tile group headers. For example, the coded picture data may include all VCL NAL units in the bitstream, and no non-VCL NAL units may be considered as coded picture data. Decoding of the coded picture data results in determining a set of sample values ​​for the picture. Decoding of data that is non-coded picture data may not result in determining any sample values ​​because the data does not contain any coded samples. A picture header may not be considered as coded picture data, in particular if the unit into which it is packed does not include any coded sample data.

[0308] Still reference Fig.16 In block 1600, the first picture may include a block that is not an intra-coded block. The set of unavailable reference pictures may include at least one unavailable reference picture. The set of pictures may also include at least one picture. The recovery point indication may be preceded by a set of unavailable reference pictures and followed by a set of pictures. The set of pictures may also include a reference to the set of unavailable reference pictures. The set of pictures may include a recovery point period starting from the first picture and ending at the recovery point picture. The recovery point indication may also include a picture at which the recovery point period in the specified bitstream ends. The bitstream may start from a starting position specified by the recovery point in the bitstream. The bitstream may include a compliant portion of the bitstream that complies with a standard specification, and the decoding decodes the compliant portion of the bitstream. The compliant portion of the bitstream may be a CVS.

[0309] Still reference Fig.16In block 16, the recovery point indication may be valid for a spatial subset of each picture in the set of pictures. A first picture in the set of pictures may be followed by a second picture in the set of pictures, wherein the first picture and the second picture are different pictures, and wherein the second picture follows the first picture in decoding order. The recovery point indication may include a canonical indication of the recovery point, and the canonical indication of the recovery point may include a temporal position of at least one of the first picture and the second picture in the set of pictures. The canonical indication of the recovery point may be ignored if at least one of the following is true: the recovery point does not start the bitstream, and no random access operation is performed at the recovery point. The canonical recovery point indication may not be included in a supplemental enhancement information (SEI) message decoded from the set of syntax elements. The recovery point indication and the first picture in the set of pictures may belong to the same access unit. The set of unavailable reference pictures may include all unavailable reference pictures in the bitstream that precede the first picture in the set of pictures in decoding order, and decoding the set of pictures may use the set of unavailable reference pictures to decode all pictures in the set of pictures in decoding order, decoding starting in the bitstream from the first picture in the set of pictures and ending in the bitstream with the second picture in the set of pictures.

[0310] exist Fig.16 At block 1602 of the decoder, a processor 2104 of the decoder decodes a recovery point indication to obtain a set of decoded syntax elements. The recovery point indication may include a set of syntax elements. The set of syntax elements may include a set of recovery point indication syntax elements. The set of syntax elements may include at least one syntax element. The set of decoded syntax elements is decoded from a recovery point indication in a non-video coding layer (non-VCL) network abstraction layer (NAL), the non-video coding layer (non-VCL) network abstraction layer (NAL) having a non-VCL NAL unit type indicating that the non-VCL NAL unit is a recovery point indication non-VCL NAL unit. The set of syntax elements may be decoded from a picture header. The recovery point indication may be decoded from a video coding layer (VCL) NAL unit including a network abstraction layer (NAL) unit type syntax element. The decoded syntax element may include at least one of: a start position of a recovery point and an end position of a recovery point period. The set of decoded syntax elements may be decoded from a supplemental enhancement information (SEI) message. The set of decoded syntax elements may be decoded from a picture parameter set comprising at least one of: a picture parameter set (PPS) and a sequence parameter set (SPS).

[0311] exist Fig.16At block 1604 of the decoder, the processor 2104 of the decoder derives information for generating a set of unavailable reference pictures from a set of decoded syntax elements before the decoder parses any coded picture data. Deriving information for generating a set of unavailable reference pictures from a set of decoded syntax elements includes at least one of the following:

[0312] deriving at least one parameter set identifier that identifies a parameter set active for a first picture in the set of pictures;

[0313] deriving the number of unusable reference pictures in the set of unusable reference pictures to be generated;

[0314] deriving a picture order count value for each picture in the set of unavailable reference pictures, and assigning the derived picture order count value to each associated picture in the set of unavailable reference pictures;

[0315] deriving a picture order count value for a first picture in a set of pictures, deriving an incremental value for an incremental picture order count of each picture in a set of unavailable reference pictures relative to the picture order count value of the first picture in the set of pictures, and calculating a picture order count value for each picture in the set of unavailable reference pictures using the derived incremental value, and assigning the calculated picture order count value to each picture in the associated unavailable reference pictures;

[0316] deriving a picture marking state of each picture in a set of unavailable reference pictures, wherein the picture marking state is at least one of the following: a long-term picture, a short-term picture, and marking each picture in the set of unavailable reference pictures with the derived marking state;

[0317] deriving a luma width value and a luma height value, and generating each picture in a set of unavailable reference pictures, each picture having the luma width value and the luma height value;

[0318] deriving luma width values ​​and luma height values ​​for each picture in the set of unavailable pictures, and generating each picture in the set of unavailable reference pictures to have a width and height with associated derived luma width and height values;

[0319] deriving a number of components of the unavailable reference picture, including a relative size value for each component and a bit depth value for each component; and generating each picture in the set of unavailable reference pictures having the number of components, the relative size, and the bit depth according to the derived values;

[0320] deriving a picture type value for each picture in the set of unavailable reference pictures and assigning the derived picture type value to each associated unavailable reference picture in the set of unavailable reference pictures;

[0321] deriving a temporal identification value for each picture in the set of unavailable reference pictures and assigning the derived temporal identification value to each associated unavailable reference picture in the set of unavailable reference pictures;

[0322] deriving a layer identification value for each picture in the set of unavailable reference pictures and assigning the derived layer identification value to each associated unavailable reference picture in the set of unavailable reference pictures;

[0323] deriving at least one picture parameter set identifier for each picture in the set of unavailable reference pictures and assigning the derived at least one picture parameter set identifier value to each associated unavailable reference picture in the set of unavailable reference pictures; and

[0324] • deriving a block size, the block size comprising a size of a coding tree unit, generating each picture in a set of unavailable reference pictures to have the block size, and assigning the block size to each unavailable reference picture in the set of unavailable reference pictures.

[0325] exist Fig.16 At block 1606 of , the processor 2104 of the decoder generates a set of unavailable reference pictures based on the derived information. For example, the generation may be performed before the decoder parses any encoded picture data. Generating the set of unavailable reference pictures may include generating each picture in the set of unavailable reference pictures. Generating the set of unavailable reference pictures based on the derived information may include generating the set of unavailable reference pictures before starting to decode any picture in the set of pictures. Generating the set of unavailable reference pictures may include allocating or assigning memory to store values ​​for each picture in the set of unavailable reference pictures. The stored values ​​may include sample values ​​for each component of each picture in the set of unavailable reference pictures. Generating each picture in the set of unavailable reference pictures may include at least one of the following:

[0326] · Setting the number of components of a picture in the set of unavailable reference pictures;

[0327] For each component of the pictures in the set of unavailable reference pictures, set the width and height;

[0328] For each component of a picture in the set of unavailable reference pictures, set the sample bit depth;

[0329] For each sample in a picture in the set of unavailable reference pictures, set the sample value;

[0330] · assigning a PPS identifier to a picture in the set of unavailable reference pictures;

[0331] Assigning SPS identifiers to pictures in the set of unavailable reference pictures;

[0332] Assigning an identifier to a picture in the set of unavailable reference pictures, wherein the identifier comprises a picture order count value;

[0333] Marking a picture in the set of unavailable reference pictures as at least one of the following: a short-term picture, a long-term picture, and not used for prediction;

[0334] Assigning picture types to pictures in the set of unavailable reference pictures;

[0335] Assigning temporal IDs to pictures in the set of unavailable reference pictures;

[0336] Assigning a layer ID to a picture in the set of unavailable reference pictures;

[0337] • For each component of a picture in the set of unavailable reference pictures, assigning a block size; and

[0338] • Mark pictures in the set of unavailable reference pictures as initialized.

[0339] exist Fig.16 At block 1608 of , the processor 2104 of the decoder decodes the set of pictures after generating the set of unavailable reference pictures. After generating the set of unavailable reference pictures, decoding the set of pictures may include: if decoding starts from the first picture, and all other pictures in the recovery point period after the first picture and before the recovery point picture and including the recovery point picture in the decoding order are decoded, then the video is completely refreshed at the recovery point picture.

[0340] Fig.17 Additional operations that a decoder may perform to decode a set of pictures from a bitstream are shown. Fig.21 The structure shown provides a decoder.

[0341] exist Fig.17 At block 1700 of , the processor 2104 of the decoder determines a position of a first picture in the set of pictures when decoding of the set of pictures is initialized at a recovery point.

[0342] exist Fig.17 At block 1702 of , the processor 2104 of the decoder determines a position of a second picture in the set of pictures.

[0343] exist Fig.17At block 1704 of , the processor 2104 of the decoder decodes the first picture and all other pictures in the set of pictures that precede the second picture in decoding order in the recovery period without outputting the decoded picture.

[0344] exist Fig.17 In block 1706 , the processor 2104 of the decoder decodes and outputs the second picture.

[0345] Regarding some embodiments of decoders and related methods, from Fig.17 Various operations of the flowchart may be optional, such as operation 1706 .

[0346] Fig.18 Additional operations that a decoder may perform to decode a set of pictures from a bitstream are shown. Fig.21 The structure shown provides a decoder. Fig.18 At block 1800 of , the processor 2104 of the decoder performs a random access operation at the recovery point.

[0347] Fig.19 shows additional operations that a decoder may perform after decoding a set of pictures from a bitstream. Fig.21 The structure shown provides a decoder. Fig.19 At block 1900 of the decoder, a processor 2104 of the decoder renders each picture in the set of pictures for display on a screen based on decoding the pictures from the bitstream after generating the set of unavailable reference pictures.

[0348] Fig. 20 shows additional operations that a decoder may perform after decoding a set of pictures from a bitstream. Fig.21 The structure shown provides a decoder. Fig. 20 At block 2000 of the decoder, a processor 2104 receives a bitstream from a remote device via a radio and / or network interface.

[0349] Regarding some embodiments of decoders and related methods, from Figures 18 to 20 Various operations of the flowchart of FIG. 1800 may be optional, such as operations 1800 , 1900 , and 2000 .

[0350] According to some embodiments of the present invention, reference will now be made to Fig.23 Flowchart to discuss (using Fig. 22 The operation of the encoder 2200 is implemented by the structure of the block diagram. For example, the module can be stored in Fig. 22 The modules may be stored in the memory 2206 of the wireless device, and these modules may provide instructions such that when the instructions of the modules are executed by the corresponding wireless device processing circuit 2204, the processing circuit 2204 performs the corresponding operations of the flowchart.

[0351] Fig.23 The operation of an encoder in encoding a recovery point indication with information about how to generate unavailable reference pictures into a bitstream is shown. Fig. 22 The structure shown provides an encoder.

[0352] exist Fig.23 At block 2300 of the encoder, the processor 2204 of the encoder encodes a first set of pictures into a bitstream. The first set of pictures may include at least one picture.

[0353] exist Fig.23 At block 2302 of , the processor 2204 of the encoder determines a set of reference pictures that will be unavailable to the decoder if decoding in the bitstream is started after the first set of pictures. The set of reference pictures may include at least one reference picture.

[0354] exist Fig.23 At block 2304 of the encoder, the processor 2204 of the encoder encodes a recovery point indication into the bitstream. The recovery point indication may include a set of syntax elements for a set of reference pictures. The set of syntax elements may include at least one syntax element for at least one picture in the set of reference pictures.

[0355] exist Fig.23 At block 2306 of the encoder, the processor 2204 of the encoder encodes the second set of pictures into the bitstream. At least one picture in the second set of pictures may reference a picture from the first set of pictures.

[0356] Example embodiments are discussed below. Reference numerals / letters are provided in parentheses by way of example / illustration and not to limit example embodiments to the specific elements indicated by the reference numerals / letters.

[0357] Embodiment 1, a method for decoding a set of pictures from a bitstream. The method includes identifying (1600) a recovery point in the bitstream according to a recovery point indication. The recovery point specifies a starting position in the bitstream for decoding the set of pictures. The set of pictures includes a first picture, which is the first picture in the set of pictures after the recovery point indication in decoding order, and wherein the set of pictures includes coded picture data. The method also includes decoding the recovery point indication (1602) to obtain a set of decoded syntax elements. The recovery point indication includes the set of syntax elements. The method also includes deriving (1604) information for generating a set of unavailable reference pictures from the decoded set of syntax elements before the decoder parses any coded picture data. The method also includes generating (1606) a set of unavailable reference pictures based on the derived information. The method also includes decoding the set of pictures (1608) after generating the set of unavailable reference pictures.

[0358] Embodiment 2: A method according to embodiment 1, wherein the generation is completed before the decoder parses any encoded picture data.

[0359] Embodiment 3: A method according to any one of embodiments 1 to 2, wherein the first picture includes blocks that are not intra-coded blocks.

[0360] Embodiment 4, a method according to any one of embodiments 1 to 3, wherein the set of unavailable reference pictures includes at least one unavailable reference picture, and wherein generating the set of unavailable reference pictures includes generating each picture in the set of unavailable reference pictures.

[0361] Embodiment 5: A method according to any one of embodiments 1 to 4, wherein the set of pictures includes at least one picture.

[0362] Embodiment 6: A method according to any one of embodiments 1 to 5, wherein the set of unavailable reference pictures is before the recovery point indication, and the set of pictures is after the recovery point indication.

[0363] Embodiment 7: A method according to any one of embodiments 1 to 6, wherein the set of pictures includes a reference to a set of unavailable reference pictures.

[0364] Embodiment 8, a method according to any one of embodiments 1 to 7, wherein generating a set of unavailable reference pictures according to the derived information comprises: generating a set of unavailable reference pictures before starting to decode any picture in the set of pictures.

[0365] Embodiment 9: A method according to any one of embodiments 1 to 8, wherein the set of syntax elements includes a set of recovery point indication syntax elements.

[0366] Embodiment 10: A method according to any one of embodiments 1 to 9, wherein the set of syntax elements includes at least one syntax element.

[0367] Embodiment 11: A method according to any one of embodiments 1 to 10, wherein the set of pictures includes a recovery point period starting from a first picture and ending at a recovery point picture.

[0368] Embodiment 12, a method according to any one of embodiments 1 to 11, wherein, after generating a set of unavailable reference pictures, decoding the set of pictures includes: if decoding starts from the first picture, and all other pictures in the recovery point period that are after the first picture and before the recovery point picture and include the recovery point picture in the decoding order are decoded, then the video is completely refreshed at the recovery point picture.

[0369] Embodiment 13: A method according to any one of embodiments 1 to 12, wherein the recovery point indication further includes specifying an end picture of a recovery point period in the bitstream.

[0370] Embodiment 14. The method according to any one of embodiments 1 to 13, wherein deriving information for generating a set of unavailable reference pictures from a set of decoded syntax elements comprises at least one of the following:

[0371] deriving at least one parameter set identifier, the at least one parameter set identifier identifying a parameter set active for the first picture in the set of pictures;

[0372] deriving the number of unusable reference pictures in the set of unusable reference pictures to be generated;

[0373] deriving a picture order count value for each picture in a set of unavailable reference pictures, and assigning the derived picture order count value to each associated picture in the set of unavailable reference pictures;

[0374] deriving a picture order count value for the first picture in the set of pictures, deriving, for each picture in the set of unavailable reference pictures, an incremental value of an incremental picture order count relative to the picture order count value for the first picture in the set of unavailable reference pictures, and calculating a picture order count value for each picture in the set of unavailable reference pictures using the derived incremental value, and assigning the calculated picture order count value to each unavailable reference picture in the associated unavailable reference pictures;

[0375] For each picture in the set of unavailable reference pictures, derive a picture marking state, wherein the picture marking state is at least one of a long-term picture and a short-term picture, and mark each picture in the set of unavailable reference pictures with the derived marking state;

[0376] deriving a luma width value and a luma height value, and generating each picture in the set of unavailable reference pictures, each picture having the luma width value and the luma height value;

[0377] deriving, for each picture in the set of unavailable pictures, a luma width value and a luma height value, and generating each picture in the set of unavailable reference pictures to have a width and height with associated derived luma width and height values;

[0378] deriving a number of components of an unavailable reference picture, including a relative size value for each component and a bit depth value for each component; and generating each picture in the set of unavailable reference pictures according to the derived values, each picture having the number of components, the relative size and the bit depth;

[0379] deriving, for each picture in a set of unavailable reference pictures, a picture type value, and assigning the derived picture type value to each associated unavailable reference picture in the set of unavailable reference pictures;

[0380] deriving, for each picture in a set of unavailable reference pictures, a temporal identification value, and assigning the derived temporal identification value to each associated unavailable reference picture in the set of unavailable reference pictures;

[0381] deriving, for each picture in a set of unavailable reference pictures, a layer identification value, and assigning the derived layer identification value to each associated unavailable reference picture in the set of unavailable reference pictures;

[0382] deriving, for each picture in a set of unavailable reference pictures, at least one picture parameter set identifier, and assigning the derived at least one picture parameter set identifier value to each associated unavailable reference picture in the set of unavailable reference pictures; and

[0383] deriving a block size, the block size comprising a size of a coding tree unit, generating each picture in the set of unavailable reference pictures to have the block size, and assigning the block size to each unavailable reference picture in the set of unavailable reference pictures.

[0384] Embodiment 15. A method according to any one of embodiments 1 to 14, wherein the set of decoded syntax elements is decoded from a recovery point indication in a non-video coding layer (non-VCL) network abstraction layer (NAL), and the non-VCL NAL has a non-VCL NAL unit type indicating that the non-VCL NAL unit is a recovery point indication non-VCL NAL unit.

[0385] Embodiment 16, a method according to any one of embodiments 1 to 15, wherein generating a set of unavailable reference pictures includes: allocating or assigning a memory to store values ​​for each picture in the set of unavailable reference pictures, wherein the stored values ​​include sample values ​​for each component of each picture in the set of unavailable reference pictures.

[0386] Embodiment 17, the method according to any one of embodiments 4 to 16, wherein generating each picture in the set of unavailable reference pictures comprises at least one of the following:

[0387] For pictures in the set of unavailable reference pictures, set the number of components;

[0388] For each component of the pictures in the set of unavailable reference pictures, set the width and height;

[0389] For each component of a picture in the set of unavailable reference pictures, set the sample bit depth;

[0390] For each sample in a picture in the set of unavailable reference pictures, set the sample value;

[0391] · assigning a PPS identifier to a picture in the set of unavailable reference pictures;

[0392] Assigning SPS identifiers to pictures in the set of unavailable reference pictures;

[0393] Assigning an identifier to a picture in the set of unavailable reference pictures, wherein the identifier comprises a picture order count value;

[0394] Marking a picture in the set of unavailable reference pictures as at least one of the following: a short-term picture, a long-term picture, and not used for prediction;

[0395] Assigning picture types to pictures in the set of unavailable reference pictures;

[0396] Assigning temporal IDs to pictures in the set of unavailable reference pictures;

[0397] Assigning a layer ID to a picture in the set of unavailable reference pictures;

[0398] • For each component of a picture in the set of unavailable reference pictures, assigning a block size; and

[0399] • Mark pictures in the set of unavailable reference pictures as initialized.

[0400] Embodiment 18: A method according to any one of embodiments 1 to 17, wherein a set of syntax elements is decoded from a picture header.

[0401] Embodiment 19. A method according to any one of embodiments 1 to 18, wherein a recovery point indication is decoded from a video coding layer (VCL) NAL unit including a network abstraction layer (NAL) unit type syntax element, and wherein the decoded syntax element includes at least one of: a starting position of the recovery point and an ending position of the recovery point period.

[0402] Embodiment 20. The method of any one of embodiments 1 to 19, wherein the set of decoded syntax elements is decoded from a supplemental enhancement information (SEI) message.

[0403] Embodiment 21. A method according to any one of embodiments 1 to 20, wherein the set of decoded syntax elements is decoded from a picture parameter set comprising at least one of: a picture parameter set (PPS) and a sequence parameter set (SPS).

[0404] Embodiment 22: A method according to any one of embodiments 1 to 21, wherein the bitstream starts from a starting position specified by a recovery point in the bitstream.

[0405] Embodiment 23. A method according to embodiment 22, wherein the bitstream includes a compliant portion of the bitstream that complies with a standard specification, and wherein the decoding decodes the compliant portion of the bitstream.

[0406] Embodiment 24: A method according to embodiment 23, wherein the conforming portion of the bitstream is CVS.

[0407] Embodiment 25. A method according to any one of embodiments 1 to 24, wherein the recovery point indication is valid for a spatial subset of each picture in the set of pictures.

[0408] Embodiment 26, a method according to any one of embodiments 1 to 25, wherein the second picture in the set of pictures is after the first picture in the set of pictures, wherein the first picture and the second picture are different pictures, and wherein the second picture is after the first picture in decoding order.

[0409] Embodiment 27, a method according to any one of embodiments 1 to 26, wherein the recovery point indication includes a standard indication of the recovery point, and wherein the standard indication of the recovery point includes a time position of at least one of a first picture and a second picture in a set of pictures.

[0410] Embodiment 28. A method according to any one of embodiments 1 to 27, wherein decoding of the set of pictures is initialized at a recovery point. The method further includes determining (1700) a position of a first picture in the set of pictures. The method further includes determining (1702) a position of a second picture in the set of pictures. The method further includes decoding (1704) the first picture and all other pictures in the set of pictures that precede the second picture in decoding order during a recovery period without outputting the decoded picture. The method further includes decoding and outputting (1706) the second picture.

[0411] Embodiment 29: The method according to any one of embodiments 1 to 28, further comprising performing (1800) a random access operation at a recovery point.

[0412] Embodiment 30: A method according to any one of embodiments 1 to 29, wherein the recovery point indication and the first picture in the set of pictures belong to the same access unit.

[0413] Embodiment 31. A method according to any one of embodiments 1 to 30, wherein the specification indication of the recovery point is ignored if at least one of the following is true: the recovery point does not start a bit stream, and no random access operation is performed at the recovery point.

[0414] Embodiment 32. A method according to any one of embodiments 1 to 31, wherein the canonical recovery point indication is not included in a supplemental enhancement information (SEI) message decoded from the set of syntax elements.

[0415] Embodiment 33, a method according to any one of embodiments 1 to 32, wherein the set of unavailable reference pictures includes all unavailable reference pictures that precede the first picture in the set of pictures in the decoding order in the bitstream, and wherein decoding the set of pictures uses the set of unavailable reference pictures to decode all pictures in the set of pictures in the decoding order, and the decoding starts from the first picture in the set of pictures in the bitstream and ends with the second picture in the set of pictures in the bitstream.

[0416] Embodiment 34: The method according to any one of embodiments 1 to 33 further includes: rendering (1900) each picture in the set of pictures for display on a screen based on decoding the pictures from the bitstream after generating the set of unavailable reference pictures.

[0417] Embodiment 35: The method according to any one of embodiments 1 to 34, further comprising: receiving (2000) a bit stream from a remote device via a radio and / or a network interface.

[0418] Embodiment 36, a decoder (2100), configured to operate to decode a set of pictures from a bitstream, the decoder comprising: a processor (2104); and a memory (2106), coupled to the processor (2104). The memory (2106) comprises instructions, which, when executed by the processor (2104), cause the decoder (2100) to perform the operations described in any one of embodiments 1 to 35.

[0419] Embodiment 37. A computer program comprising a program code (2108) to be executed by a processor (2104) of a decoder (2100), the decoder (2100) being configured to operate to decode a set of pictures from a bitstream, whereby execution of the program code (2108) causes the decoder (2100) to perform operations according to any one of embodiments 1 to 35.

[0420] Embodiment 38. A method for encoding a recovery point indication into a bitstream, the recovery point indication having information about how to generate unavailable reference pictures. The method includes encoding (2300) a first set of pictures into the bitstream. The method also includes determining (2302) a set of reference pictures that will be unavailable to the decoder if decoding in the bitstream is started after the first set of pictures. The method also includes encoding (2304) the recovery point indication into the bitstream. The recovery point indication includes a set of syntax elements for the set of reference pictures. The method also includes encoding (2306) a second set of pictures into the bitstream. At least one picture in the second set of pictures references a picture from the first set of pictures.

[0421] Embodiment 39: A method according to embodiment 38, wherein the first set of pictures includes at least one picture.

[0422] Embodiment 40: A method according to any one of embodiments 38 to 39, wherein the set of reference pictures includes at least one reference picture.

[0423] Embodiment 41. A method according to any one of embodiments 38 to 40, wherein the set of syntax elements includes at least one syntax element for at least one picture in the set of reference pictures.

[0424] Embodiment 42, an encoder (2200), configured to operate to encode a recovery point indication into a bitstream, the recovery point indication having information about how to generate an unavailable reference picture, the encoder comprising: a processor (2204); and a memory (2206), coupled to the processor (2204). The memory (2206) comprises instructions, which, when executed by the processor (2204), cause the encoder (2200) to perform the operations described in any one of embodiments 38 to 41.

[0425] Embodiment 43. A computer program comprising a program code (2208) to be executed by a processor (2204) of an encoder (2200), the encoder being configured to operate to encode in a bitstream a recovery point indication having information on how to generate an unavailable reference picture, whereby execution of the program code (2208) causes the encoder (2200) to perform operations according to any one of embodiments 38 to 41.

[0426] Further definitions and embodiments are discussed below:

[0427] In the above description of various embodiments of the inventive concept, it is to be understood that the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the inventive concept. Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meanings as commonly understood by ordinary technicians in the field to which the inventive concept belongs. It should also be understood that terms such as those defined in general dictionaries should be interpreted as having meanings consistent with their meanings in the context of this specification and the relevant technology, and not interpreted as ideal or overly superficial meanings, unless so clearly defined herein.

[0428] When an element is referred to as being "connected to", "coupled to", "responsive to" another element or a variation thereof, it may be directly connected to, coupled to, or responsive to another element, or there may be an intermediate element. Conversely, when an element is referred to as being "directly connected to", "directly coupled to", "directly responsive to" another element or a variation thereof, there is no intermediate element. Throughout the text, similar reference numerals represent similar elements. In addition, "coupling", "connection", "responsiveness" or variations thereof used herein may include wireless coupling, connection or response. As used herein, the singular forms "one", "an" and "said" are intended to also include plural forms, unless the context clearly indicates otherwise. For the sake of brevity and / or clarity, well-known functions or structures may not be described in detail. The term "and / or" includes any and all combinations of one or more items listed in association.

[0429] It will be understood that although the terms first, second, third, etc. can be used herein to describe each element / operation, these elements / operations should not be limited by these terms. These terms are only used to distinguish one element / operation from another element / operation. Therefore, the first element / operation in some embodiments can be referred to as the second element / operation in other embodiments without departing from the teaching of the inventive concept. Throughout the specification, the same reference numerals or the same reference signs represent the same or similar elements.

[0430] As used herein, the terms "comprise, comprising, comprises, include, including, includes," "have, has, having," or variations thereof, are open ended and include one or more stated features, integers, elements, steps, components, or functions, but do not preclude the presence or addition of one or more other features, integers, elements, steps, components, functions, or combinations thereof. In addition, as used herein, the commonly used abbreviation "eg," derived from the Latin phrase "exempli gratia," may be used to introduce or specify a general example of a previously mentioned item without being intended as a limitation of that item. The commonly used abbreviation "ie," derived from the Latin phrase "id est," may be used to specify a specific item of a more general reference.

[0431] This article describes example embodiments with reference to block diagrams and / or flowchart illustrations of computer-implemented methods, devices (systems and / or equipment), and / or computer program products. It should be understood that the blocks of the block diagrams and / or flowchart illustrations and the combination of blocks in the block diagrams and / or flowchart illustrations can be implemented by computer program instructions executed by one or more computer circuits. These computer program instructions can be provided to processor circuits of general-purpose computer circuits, special-purpose computer circuits, and / or other programmable data processing circuits to produce a machine, so that instructions executed by processors of computers and / or other programmable data processing devices convert and control transistors, values ​​stored in memory locations, and other hardware components within such circuits to implement the functions / actions specified in the block diagrams and / or flowcharts, and thereby create a device (functional body) and / or structure for implementing the functions / actions specified in the block diagrams and / or flowcharts.

[0432] These computer program instructions may also be stored in a tangible computer-readable medium that can direct a computer or other programmable data processing device to act in a specific manner so that the instructions stored in the computer-readable medium produce an article of manufacture, the article of manufacture including instructions for implementing the functions / actions specified in the blocks of the block diagrams and / or flow charts. Thus, embodiments of the inventive concept may be implemented in hardware and / or in software (including firmware, stored software, microcode, etc.) running on a processor such as a digital signal processor, which may be collectively referred to as a "circuit," "module," or variations thereof.

[0433] It should also be noted that in some alternative implementations, the function / action marked in the frame may not occur in the order marked in the flow chart. For example, depending on the function / action involved, the two frames shown in succession can actually be performed substantially simultaneously, or the frame can sometimes be performed in the opposite order. In addition, the function of a given frame of a flow chart and / or block diagram can be divided into multiple frames and / or the function of two or more frames of a flow chart and / or block diagram can be integrated at least in part. Finally, without departing from the scope of the inventive concept, other frames can be added / inserted between the frames shown, and / or the frame / operation can be omitted. In addition, although some figures include arrows on the communication path to illustrate the main direction of communication, it should be understood that communication can occur in the direction opposite to the arrows depicted.

[0434] Under the premise of not departing from the principle of the inventive concept basically, many changes and modifications can be made to the embodiment. All these changes and modifications are intended to be included in the scope of the inventive concept herein. Therefore, the above-mentioned subject matter should be understood as exemplary and not restrictive, and the examples of the embodiments are intended to cover all these modifications, improvements and other embodiments that fall within the spirit and scope of the inventive concept. Therefore, to the maximum extent permitted by law, the scope of the inventive concept should be determined by the widest permissible interpretation of the present disclosure including the examples of the embodiments and their equivalents, and should not be limited to or restricted to the previous specific embodiments.

Claims

1. A method for decoding a set of pictures from a video bitstream, the method comprising: identifying (1600) a recovery point in a video bitstream based on a recovery point indication, wherein the recovery point specifies a starting position in the video bitstream for decoding the set of pictures, wherein the set of pictures includes a first picture, the first picture being a first picture in the set of pictures that follows the recovery point indication in decoding order, and wherein the set of pictures includes coded picture data; Decoding the recovery point indication (1602) to obtain a set of decoded syntax elements, wherein the recovery point indication includes the set of syntax elements; Prior to parsing any coded picture data included in the set of pictures, deriving (1604) information for generating a set of unavailable reference pictures from the set of decoded syntax elements; generating (1606) the set of unavailable reference pictures based on the derived information; and After generating the set of unavailable reference pictures, the set of pictures is decoded (1608).

2. The method according to claim 1, wherein: The set of syntax elements is decoded from a picture header.

3. The method according to any one of claims 1 to 2, wherein: The coded picture data includes all VCL NAL units included in the set of pictures in the video bitstream.

4. The method according to any one of claims 1 to 3, wherein: The generating is done prior to parsing any encoded picture data included in the set of pictures.

5. The method according to any one of claims 1 to 4, wherein: The first picture includes blocks that are not intra-coded blocks.

6. The method according to any one of claims 1 to 5, wherein: The set of pictures includes a reference to the set of unavailable reference pictures.

7. The method according to any one of claims 1 to 6, wherein: The set of pictures includes a recovery point period starting with the first picture and ending at a recovery point picture.

8. The method according to claim 7, further comprising: After generating the set of unavailable reference pictures, a first picture in the set of pictures and all other pictures in the recovery point period after the first picture up to and including the recovery point picture are decoded, thereby completely refreshing the video carried in the video bitstream.

9. The method according to any one of claims 1 to 8, wherein: Deriving information for generating a set of unavailable reference pictures from a set of decoded syntax elements includes at least one of the following: deriving at least one parameter set identifier, the at least one parameter set identifier identifying a parameter set active for the first picture in the set of pictures; deriving the number of unusable reference pictures in the set of unusable reference pictures to be generated; deriving a picture order count value for each picture in a set of unavailable reference pictures, and assigning the derived picture order count value to each associated picture in the set of unavailable reference pictures; deriving a picture order count value for the first picture in the set of pictures, deriving, for each picture in the set of unavailable reference pictures, an incremental value of an incremental picture order count relative to the picture order count value for the first picture in the set of unavailable reference pictures, and calculating a picture order count value for each picture in the set of unavailable reference pictures using the derived incremental value, and assigning the calculated picture order count value to each unavailable reference picture in the associated unavailable reference pictures; For each picture in the set of unavailable reference pictures, derive a picture marking state, wherein the picture marking state is at least one of a long-term picture and a short-term picture, and mark each picture in the set of unavailable reference pictures with the derived marking state; deriving a luma width value and a luma height value, and generating each picture in the set of unavailable reference pictures, each picture having the luma width value and the luma height value; deriving, for each picture in the set of unavailable pictures, a luma width value and a luma height value, and generating each picture in the set of unavailable reference pictures to have a width and height with associated derived luma width and height values; deriving a number of components of an unavailable reference picture, including a relative size value for each component and a bit depth value for each component; and generating each picture in the set of unavailable reference pictures according to the derived values, each picture having the number of components, the relative size and the bit depth; deriving, for each picture in a set of unavailable reference pictures, a picture type value, and assigning the derived picture type value to each associated unavailable reference picture in the set of unavailable reference pictures; deriving, for each picture in a set of unavailable reference pictures, a temporal identification value, and assigning the derived temporal identification value to each associated unavailable reference picture in the set of unavailable reference pictures; deriving, for each picture in a set of unavailable reference pictures, a layer identification value, and assigning the derived layer identification value to each associated unavailable reference picture in the set of unavailable reference pictures; deriving, for each picture in a set of unavailable reference pictures, at least one picture parameter set identifier, and assigning the derived at least one picture parameter set identifier value to each associated unavailable reference picture in the set of unavailable reference pictures; and deriving a block size, the block size comprising a size of a coding tree unit, generating each picture in the set of unavailable reference pictures to have the block size, and assigning the block size to each unavailable reference picture in the set of unavailable reference pictures.

10. The method according to any one of claims 1 to 9, wherein: The set of decoded syntax elements is decoded from a recovery point indication in a non-video coding layer network abstraction layer, ie, non-VCL NAL.

11. The method according to any one of claims 1 to 10, wherein: Generating the set of unavailable reference pictures includes allocating or assigning memory to store values ​​for each picture in the set of unavailable reference pictures, wherein the stored values ​​include sample values ​​for each component of each picture in the set of unavailable reference pictures.

12. The method according to any one of claims 1 to 11, wherein: The set of unavailable reference pictures includes at least one unavailable reference picture, and wherein generating the set of unavailable reference pictures includes generating each picture in the set of unavailable reference pictures, and wherein generating each picture in the set of unavailable reference pictures includes at least one of the following: For a picture in the set of unavailable reference pictures, setting the number of components; For each component of a picture in the set of unavailable reference pictures, set the width and height; For each component of a picture in the set of unavailable reference pictures, setting a sample bit depth; For each sample in a picture in the set of unavailable reference pictures, setting a sample value; · assigning a PPS identifier to a picture in the set of unavailable reference pictures; · assigning an SPS identifier to a picture in the set of unavailable reference pictures; assigning an identifier to a picture in the set of unavailable reference pictures, wherein the identifier comprises a picture order count value; Marking a picture in the set of unavailable reference pictures as at least one of the following: a short-term picture, a long-term picture, and not used for prediction; · assigning picture types to pictures in the set of unavailable reference pictures; · assigning a temporal ID to a picture in the set of unavailable reference pictures; Assigning a layer ID to a picture in the set of unavailable reference pictures; • for each component of a picture in the set of unavailable reference pictures, assigning a block size; and • Marking pictures in the set of unavailable reference pictures as initialized.

13. The method according to any one of claims 1 to 12, wherein: The recovery point indication is decoded from a video coding layer (VCL) NAL unit including a network abstraction layer (NAL) unit type syntax element, and wherein the decoded syntax element includes at least one of: a start position of a recovery point and an end position of the recovery point period.

14. The method according to any one of claims 1 to 13, further comprising performing (1800) a random access operation at the recovery point.

15. The method according to any one of claims 1 to 14, wherein: The recovery point indication and a first picture in the set of pictures belong to the same access unit.

16. A decoder (2100) configured to operate to decode a set of pictures from a video bitstream, comprising: Processor(2104); as well as A memory (2106) coupled to the processor (2104), wherein the memory (2106) comprises instructions, which, when executed by the processor (2104), cause the decoder (2100) to perform the operations according to any one of claims 1 to 15.

17. A computer program comprising program code (2108) to be executed by a processor (2104) of a decoder (2100), the decoder (2100) being configured to operate to decode a set of pictures from a video bitstream, whereby execution of the program code (2108) causes the decoder (2100) to perform operations according to any one of claims 1 to 15.

18. A method for encoding a recovery point indication into a video bitstream, the method comprising: encoding (2300) a first set of pictures into the video bitstream; determining (2302) a set of reference pictures that would be unavailable to a decoder if decoding in the video bitstream was started after the first set of pictures; encoding (2304) a recovery point indication into the video bitstream, wherein the recovery point indication comprises a set of syntax elements for a set of reference pictures that are to be unavailable, thereby providing information to enable a decoder to generate a picture in the set of unavailable reference pictures before parsing a second set of pictures in the video bitstream, the second set of pictures comprising at least one picture that references a picture in the first set of pictures; and A second set of pictures is encoded (2306) into the video bitstream.

19. The method according to claim 18, wherein: Encoding a recovery point indication into the video bitstream includes encoding the set of syntax elements into a picture header.

20. An encoder (2200) configured to operate to encode a recovery point indication into a video bitstream, the recovery point indication comprising information to enable a decoder to generate a picture from a set of unavailable reference pictures, comprising: Processor (2204); as well as A memory (2206) coupled to the processor (2204), wherein the memory (2206) includes instructions, which, when executed by the processor (2204), cause the encoder (2200) to perform the operations according to claim 18 or claim 19.

21. A computer program comprising program code (2208) to be executed by a processor (2204) of an encoder (2200), the encoder (2200) being configured to operate to encode a recovery point indication into a video bitstream, the recovery point indication comprising information to enable a decoder to generate a picture from a set of unavailable reference pictures, whereby execution of the program code (2208) causes the encoder (2200) to perform operations as claimed in claim 18 or claim 19.

Images