Video encoding and decoding method, device and medium

Abstract

The present disclosure provides a video decoding method, apparatus and medium. The apparatus includes computer code. The computer code is used to enable one or more processors to parse at least one video parameter set. The at least one video parameter set includes at least one syntax element, and the at least one syntax element indicates whether at least one layer in a scalable bitstream is one of the following two: (i) a dependent layer in the scalable bitstream; (ii) an independent layer of the scalable bitstream; by parsing an inter-layer reference picture list, decoding a picture in the dependent layer; and, without parsing the inter-layer reference picture list, decoding a picture in the independent layer.

Description

[0001] Incorporation by Reference

[0002] This disclosure claims priority to U.S. Provisional Application No. 62 / 903,647, filed on September 20, 2019, and priority to U.S. Application No. 17 / 019,567, filed on September 14, 2020, the entire contents of which are incorporated herein. Background Art

[0003] 1. Field

[0004] The disclosed subject matter relates to video encoding and decoding, and more particularly, to signaling of inter-layer prediction in video bitstreams.

[0005] 2. Description of Related Technology

[0006] It has been known for decades that video encoding and decoding can be performed by inter-frame picture prediction techniques with motion compensation. An uncompressed digital video may include a series of pictures, each picture having a spatial dimension of, for example, 1920×1080 luma samples and associated chroma samples. The series of pictures has a fixed or variable picture rate (also informally called frame rate), such as 60 pictures per second or 60Hz. Uncompressed video has very large bitrate requirements. For example, 1080p60 4:2:0 video (1920x1080 luma sample resolution, 60Hz frame rate) with 8 bits per sample requires nearly 1.5Gbit / s bandwidth. One hour of such video requires more than 600GB of storage space.

[0007] One goal of video encoding and decoding is to reduce redundant information in the input video signal through compression. Video compression can help reduce the bandwidth or storage space requirements mentioned above, in some cases by two or more orders of magnitude. Both lossless and lossy compression, as well as combinations of the two, can be used. Lossless compression refers to techniques that reconstruct an exact replica of the original signal from the compressed original signal. When lossy compression is used, the reconstructed signal may not be exactly the same as the original signal, but the distortion between the original and the reconstructed signal is small enough that the reconstructed signal can be used for the intended application. Lossy compression is widely used in video. The amount of distortion allowed depends on the application. For example, users of some consumer streaming applications may tolerate higher distortion than users of television applications. The achievable compression ratio reflects that higher allowed / tolerable distortion results in higher compression ratios.

[0008] Video encoders and decoders can utilize several broad categories of techniques including, for example, motion compensation, transforms, quantization, and entropy coding, some of which are described below.

[0009] Historically, video encoders and decoders have tended to operate on a given picture size, which in most cases is defined and remains constant for a Coded Video Sequence (CVS), a Group of Pictures (GOP), or a similar multi-picture temporal frame. For example, in MPEG-2, system designs are known to change the horizontal resolution (and, thereby, the picture size) according to factors such as scene activity, but only at I pictures, and therefore typically for GOPs. For example, according to ITU-T Rec. H.263 Annex P, resampling of reference pictures using different resolutions within a CVS is known. However, here the picture size is not changed, only the reference pictures are resampled, which may result in only part of the picture canvas being used (in the case of downsampling), or only part of the scene being captured (in the case of upsampling). In addition, H.263 Annex Q allows a single macroblock to be resampled up or down by a factor of 2 (in each dimension). Again, the picture size remains the same. In H.263, the size of the macroblock is fixed, and therefore there is no need to signal the size of the macroblock.

[0010] In modern video codecs, changes in picture size in predicted pictures have become more mainstream. For example, VP9 allows reference picture resampling (RPR) and changing the resolution of the entire picture. Similarly, for Versatile Video Coding (VVC), certain proposals have been made (including, for example, Hendry et al., "On adaptive resolution change (ARC) for VVC", Joint Video Team document JVET-M0135-v1, January 9-19, 2019, incorporated herein in its entirety) to allow the entire reference picture to be resampled to different higher or lower resolutions. In this document, it is proposed that different candidate resolutions be encoded in a sequence parameter set (SPS) and referenced by each picture syntax element in a picture parameter set (PPS). Summary of the invention

[0011] To solve one or more different technical problems, the present disclosure describes a new syntax designed for signaling in a scaling video bitstream and its use. Thus, improved (de)coding efficiency may be achieved.

[0012] According to an embodiment of the present invention, using Reference Picture Resampling (RPR) or Adaptive Resolution Change (ARC), the additional burden of scalability support can be achieved by modifying the High-Level Syntax (HLS). In terms of technology, inter-layer prediction is used in a scalable system to improve the encoding and decoding efficiency of the enhancement layer. In addition to the spatial and temporal motion compensation predictions available in a single-layer codec, inter-layer prediction uses resampled video data from a reconstructed reference picture from a reference layer to predict the current enhancement layer. Then, by modifying the existing interpolation process for motion compensation, the resampling process for inter-layer prediction is performed at the block level. This means that no additional resampling process is required to support scalability. In the present disclosure, high-level syntax elements that support spatial / quality scalability using RPR are disclosed.

[0013] A method and apparatus are provided, the apparatus comprising a memory and one or more processors, wherein the memory is used to store computer program code, and the one or more processors are used to access the computer program code and operate according to the instructions of the computer program code. The computer program code comprises: a parsing code, which is used to enable the at least one processor to parse at least one video parameter set VPS, the at least one VPS comprising at least one syntax element, wherein the at least one syntax element indicates whether at least one layer in the scalable bitstream is one of the following: (i) a dependent layer of the scalable bitstream; (ii) an independent layer of the scalable bitstream; a first decoding code, which is used to enable the at least one processor to decode the picture in the dependent layer by parsing and interpreting the inter-layer reference picture ILRP list; and a second decoding code, which is used to enable the at least one processor to decode the picture in the independent layer without parsing and interpreting the ILRP list.

[0014] According to an embodiment, the second decoding code is further used to cause the at least one processor to decode the picture in the independent layer by parsing and interpreting a reference picture list, wherein the reference picture list does not include any decoded picture of another layer.

[0015] According to an embodiment, the inter-layer reference picture list comprises decoded pictures of another layer.

[0016] According to an embodiment, the parsing code further causes the at least one processor to: parse the at least one VPS by determining whether another syntax element indicates a maximum number of layers.

[0017] According to an embodiment, the parsing code further causes the at least one processor to parse the at least one VPS by determining whether the VPS includes a flag, wherein the flag indicates whether another layer in the scalable bitstream is a reference layer of the at least one layer.

[0018] According to an embodiment, the parsing code further causes the at least one processor to parse the at least one VPS by specifying an index of another layer and an index of the at least one layer, determining whether the flag indicates that the other layer is a reference layer of the at least one layer.

[0019] According to an embodiment, the parsing code further causes the at least one processor to parse the at least one VPS by specifying an index of another layer and an index of the at least one layer, determining whether the flag indicates that the other layer is not a reference layer of the at least one layer.

[0020] According to an embodiment, the parsing code further causes the at least one processor to parse the at least one VPS by determining whether the VPS includes a flag indicating whether to decode a plurality of layers including the at least one layer by interpreting the ILRP list.

[0021] According to an embodiment, the parsing code further causes the at least one processor to parse the at least one VPS by determining whether the VPS includes a flag indicating whether to decode a plurality of layers including the at least one layer without interpreting the ILRP list.

[0022] According to an embodiment, the parsing code further causes the at least one processor to parse the at least one VPS and further include: determining whether the VPS includes a flag indicating whether to decode multiple layers including the at least one layer by interpreting the ILRP list. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Other features, properties and various advantages of the disclosed subject matter will become further apparent from the following detailed description and accompanying drawings, in which:

[0024] Figure 1 is a schematic diagram of a simplified block diagram of a communication system according to an embodiment.

[0025] Figure 2 is a schematic diagram of a simplified block diagram of a communication system according to an embodiment.

[0026] Figure 3 is a schematic diagram of a simplified block diagram of a decoder according to an embodiment.

[0027] Figure 4is a schematic diagram of a simplified block diagram of an encoder according to an embodiment.

[0028] FIG. 5A is a diagram illustrating options for signaling ARC / RPR parameters according to the related art.

[0029] FIG. 5B is a diagram illustrating options for signaling ARC / RPR parameters according to the related art.

[0030] Figure 5C is a schematic diagram of options for signaling ARC / RPR parameters according to an embodiment.

[0031] Figure 5D is a schematic diagram of options for signaling ARC / RPR parameters according to an embodiment.

[0032] Figure 5E is a schematic diagram of options for signaling ARC / RPR parameters according to an embodiment.

[0033] Figure 6 is a schematic diagram of signaling picture resolution according to an embodiment.

[0034] Figure 7 is a schematic diagram of signaling picture size and consistency window in SPS according to an embodiment.

[0035] Figure 8 It is a schematic diagram of signaling the presence of inter-layer prediction in SPS according to an embodiment.

[0036] Fig. 9 is a schematic diagram of signaling inter-layer prediction index in a slice header according to an embodiment.

[0037] Fig.10 is a schematic diagram of a computer system according to an embodiment. DETAILED DESCRIPTION

[0038] The features of the suggestions discussed below may be used alone or in any combination in any order. In addition, embodiments may be implemented by processing circuits (e.g., one or more processors, or, one or more integrated circuits). In one example, one or more processors execute a program stored in a non-volatile computer-readable medium.

[0039] Figure 1A simplified block diagram of a communication system (100) according to an embodiment of the present disclosure is shown. The system (100) may include at least two terminal devices (110 and 120) interconnected by a network (150). For unidirectional data transmission, a first terminal device (110) may encode video data at a local location for transmission to another terminal device (120) via the network (150). A second terminal device (120) may receive encoded video data from another terminal device from the network (150), decode the encoded video data, and display the recovered video data. Unidirectional data transmission is more common in applications such as media services.

[0040] Figure 1 A second pair of terminal devices (130, 140) is shown that supports bidirectional transmission of encoded video, which can occur, for example, during a video conference. For bidirectional data transmission, each terminal device (130, 140) can encode video data collected at a local location for transmission to the other terminal device via a network (150). Each terminal device (130, 140) can also receive encoded video data transmitted by the other terminal device, and can decode the encoded video data and display the recovered video data on a local display device.

[0041] exist Figure 1 In the embodiment of the present disclosure, the terminal devices (110, 120, 130, 140) may be servers, personal computers, and smart phones, but the principles of the present disclosure may not be limited thereto. Embodiments of the present disclosure are applicable to laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. The network (150) represents any number of networks that transmit encoded video data between the terminal devices (110, 120, 130, 140), including, for example, wired and / or wireless communication networks. The communication network (150) may exchange data in circuit switching and / or packet switching channels. Representative networks include telecommunication networks, local area networks, wide area networks, and / or the Internet. For the purposes of this discussion, unless explained below, the architecture and topology of the network (150) may be irrelevant to the operation of the present disclosure.

[0042] As an application example of the disclosed subject matter, Figure 2 The placement of the video decoder and encoder in a streaming environment is shown. The disclosed subject matter is equally applicable to other video-enabled applications including, for example, video conferencing, digital TV, storing compressed video on digital media including CDs, DVDs, memory sticks, etc., and the like.

[0043] The streaming system may include an acquisition subsystem (213), which may include a video source (201), such as a digital camera, which creates, for example, an uncompressed video sample stream (202). The sample stream (202) is depicted as a thick line compared to an encoded video bitstream to emphasize that it is a high data volume video sample stream, and the sample stream (202) may be processed by an encoder (203) coupled to the camera (201). The encoder (203) may include hardware, software, or a combination of hardware and software to implement or implement various aspects of the disclosed subject matter as described in more detail below. The encoded video bitstream (204) is depicted as a thin line compared to the sample stream (202) to emphasize the lower data volume of the encoded video bitstream, which may be stored on a streaming server (205) for future use. One or more streaming clients (206, 208) may access the streaming server (205) to retrieve a copy (207, 209) of the encoded video bitstream (204). The client (206) may include a video decoder (210). The video decoder (210) decodes an incoming copy of an encoded video bitstream (207) and produces an output video sample stream (211) that can be presented on a display (212) or another presentation device (not shown). In some streaming systems, the video bitstream (204, 207, 209) may be encoded according to certain video codec / compression standards. Examples of these standards include ITU-T Recommendation H.265. The video codec standard under development is informally referred to as Versatile Video Codec (VVC). The disclosed subject matter may be used in the context of VVC.

[0044] Figure 3 is a functional block diagram of a video decoder (210) according to an embodiment of the present disclosure.

[0045] The receiver (310) may receive one or more encoded video sequences to be decoded by the video decoder (210); in the same or another embodiment, one encoded video sequence is received at a time, wherein the decoding of each encoded video sequence is independent of the other encoded video sequences. The encoded video sequence may be received from a channel (312), which may be a hardware / software link to a storage device storing the encoded video data. The receiver (310) may receive the encoded video data as well as other data, such as encoded audio data and / or auxiliary data streams that may be forwarded to their respective consuming entities (not shown). The receiver (310) may separate the encoded video sequence from the other data. To prevent network jitter, a buffer memory (315) may be coupled between the receiver (310) and an entropy decoder / parser (320) (hereinafter referred to as "parser"). When the receiver (310) receives data from a storage / forward device with sufficient bandwidth and controllability or from an isochronous synchronous network, it may not be necessary to configure the buffer memory (315), or the buffer memory may be made smaller. Of course, for use over a packet network such as the Internet, a buffer memory (315) may also be required, which may be relatively large and may advantageously be of an adaptive size.

[0046] The video decoder (210) may include a parser (320) to reconstruct symbols (321) from an entropy coded video sequence. The categories of these symbols include information for managing the operation of the video decoder (210) and potentially information for controlling a display device such as display 212 that is not part of the decoder but may be coupled to the decoder, such as Figure 3As shown in . The control information for the display device may be in the form of a parameter set fragment (not indicated) of Supplementary Enhancement Information (SEI message) or Video Usability Information (VUI). The parser (320) may parse / entropy decode the received coded video sequence. The coding and decoding of the coded video sequence may be performed according to a video coding technique or standard, and may follow principles known to those skilled in the art, including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, and the like. The parser (320) may extract a subgroup parameter set of at least one subgroup of the subgroups of pixels in the video decoder from the coded video sequence based on at least one parameter corresponding to the group. The subgroup may include a Group of Pictures (GOP), a picture, a tile, a slice, a macroblock, a Coding Unit (CU), a block, a Transform Unit (TU), a Prediction Unit (PU), and the like. The entropy decoder / parser can also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, and so on.

[0047] The parser (320) may perform entropy decoding / parsing operations on the video sequence received from the buffer memory (315) to create symbols (321).

[0048] Depending on the type of coded video picture or portion of coded video picture (e.g., inter- and intra-pictures, inter- and intra-blocks) and other factors, the reconstruction of the symbol (321) may involve multiple different units. Which units are involved and how they are involved may be controlled by subgroup control information parsed by the parser (320) from the coded video sequence. For the sake of brevity, such subgroup control information flow between the parser (320) and the multiple units below is not described.

[0049] In addition to the functional blocks already mentioned, the video decoder 210 can be conceptually subdivided into several functional units as described below. In actual implementations operating under commercial constraints, many of these units interact closely with each other and can be at least partially integrated with each other. However, for the purpose of describing the disclosed subject matter, the conceptual subdivision into the following functional units is appropriate.

[0050] The first unit is a sealer / inverse transform unit (351). The sealer / inverse transform unit (351) receives quantized transform coefficients as symbols (321) from the parser (320) and control information, including which transform method to use, block size, quantization factor, quantization scaling matrix, etc. The sealer / inverse transform unit (351) can output a block including sample values, which can be input into an aggregator (355).

[0051] In some cases, the output samples of the scaler / inverse transform unit (351) may belong to an intra-coded block; that is, a block that does not use predictive information from a previously reconstructed picture, but may use predictive information from a previously reconstructed portion of the current picture. Such predictive information may be provided by an intra-picture prediction unit (352). In some cases, the intra-picture prediction unit (352) generates a block of the same size and shape as the block being reconstructed using surrounding reconstructed information extracted from the (partially reconstructed) current picture (358). In some cases, the aggregator (355) adds the prediction information generated by the intra-prediction unit (352) to the output sample information provided by the scaler / inverse transform unit (351) on a per-sample basis.

[0052] [1] In other cases, the output samples of the scaler / inverse transform unit (351) may belong to an inter-frame coded and potentially motion compensated block. In this case, the motion compensated prediction unit (353) may access the reference picture memory (357) to extract samples for prediction. After the extracted samples are motion compensated according to the symbols (321), these samples may be added to the output of the scaler / inverse transform unit (in this case referred to as residual samples or residual signals) by the aggregator (355) to generate output sample information. The motion compensated prediction unit's retrieval of predicted samples from addresses within the reference picture memory may be controlled by a motion vector, and the motion vector is provided to the motion compensated prediction unit (353) in the form of the symbols (321), for example, including X, Y and reference picture components. Motion compensation may also include interpolation of sample values ​​extracted from the reference picture memory (357) when using sub-sample accurate motion vectors, motion vector prediction mechanisms, and the like.

[0053] The output samples of the aggregator (355) may be employed by various loop filtering techniques in a loop filter unit (356). The video compression techniques may include in-loop filter techniques controlled by parameters included in the encoded video bitstream and available to the loop filter unit (356) as symbols (321) from the parser (320). However, the video compression techniques may also be responsive to meta-information obtained during decoding of a previous (in decoding order) portion of an encoded picture or encoded video sequence, and to previously reconstructed and loop filtered sample values.

[0054] The output of the loop filter unit (356) may be a sample stream that may be output to a display device (212) and stored in a reference picture memory (357) for subsequent inter-picture prediction.

[0055] Once fully reconstructed, certain coded pictures may be used as reference pictures for future prediction. Once a coded picture is fully reconstructed, and the coded picture is identified as a reference picture (e.g., by the parser (320)), the current picture (358) may become part of the reference picture memory (357), and a new current picture memory may be reallocated before starting reconstruction of a subsequent coded picture.

[0056] The video decoder (210) may perform decoding operations according to a predetermined video compression technique as documented, for example, in the ITU-T H.265 standard. The encoded video sequence may conform to the syntax specified by the video compression technique or standard used, in the sense that the encoded video sequence follows the syntax of the video compression technique or standard as specified in the video compression technique document or standard, in particular, in a profile. For compliance, the complexity of the encoded video sequence is also required to be within the limits defined by the hierarchy of the video compression technique or standard. In some cases, the hierarchy limits the maximum picture size, maximum frame rate, maximum reconstruction sampling rate (measured in, for example, mega samples per second), maximum reference picture size, etc. In some cases, the limits set by the hierarchy may be further limited by the Hypothetical Reference Decoder (HRD) specification and metadata of the HRD buffer management signaled in the encoded video sequence.

[0057] In an embodiment, the receiver (310) may receive additional (redundant) data along with the encoded video. The additional data may be part of the encoded video sequence. The additional data may be used by the video decoder (210) to properly decode the data and / or more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0058] Figure 4 is a functional block diagram of a video encoder (203) according to an embodiment of the present disclosure.

[0059] The video encoder (203) may receive video samples from a video source (201) (not a part of the encoder) that may capture video images to be encoded by the video encoder (203).

[0060] The video source (201) may provide a source video sequence in the form of a digital video sample stream to be encoded by the video encoder (203), wherein the digital video sample stream may have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, ...), any color space (e.g., BT.601 Y CrCB, RGB, ...), and any suitable sampling structure (e.g., Y CrCb 4:2:0, Y CrCb 4:4:4). In a media service system, the video source (201) may be a storage device storing previously prepared videos. In a video conferencing system, the video source (201) may be a camera that captures local image information as a video sequence. The video data may be provided as a plurality of separate pictures that are given motion when viewed sequentially. The pictures themselves may be constructed as a spatial pixel array, wherein each pixel may include one or more samples depending on the sampling structure, color space, etc. used. The relationship between pixels and samples may be easily understood by those skilled in the art. The following description focuses on the samples.

[0061] According to an embodiment, the video encoder (203) may encode and compress pictures of a source video sequence into an encoded video sequence (443) in real time or under any other time constraints required by the application. Implementing an appropriate encoding speed is a function of a controller (450). The controller controls other functional units as described below and is functionally coupled to these units. For the sake of brevity, couplings are not shown in the figure. Parameters set by the controller may include rate control related parameters (e.g., picture skipping, quantizer, lambda value of rate-distortion optimization techniques, etc.), picture size, group of pictures (GOP) layout, maximum motion vector search range, etc. Those skilled in the art can easily identify other functions of the controller (450) because they may belong to a video encoder (203) optimized for a specific system design.

[0062] Some video encoders operate in a manner that is readily recognizable to those skilled in the art as a "coding loop". As a simple description, the coding loop may include an encoding portion of an encoder (430) (hereinafter referred to as a "source encoder", which is responsible for creating symbols based on an input picture to be encoded and a reference picture), and a (local) decoder (433) embedded in the video encoder (203). The "local" decoder (433) reconstructs the symbols to create sample data in a manner similar to how the (remote) decoder creates sample data (because in the video compression techniques contemplated by the disclosed subject matter, any compression between the symbols and the encoded video code stream is lossless). The reconstructed sample stream is input to a reference picture memory (434). Since the decoding of the symbol stream produces bit-accurate results that are independent of the decoder location (local or remote), the contents of the reference picture memory also correspond bit-by-bit between the local encoder and the remote encoder. In other words, the reference picture samples "seen" by the prediction portion of the encoder are exactly the same sample values ​​that the decoder will "see" when using the prediction during decoding. Such basic principles of reference picture synchronization (and the drift that occurs when synchronization cannot be maintained, eg due to channel errors) are well known to those skilled in the art.

[0063] The operation of the "local" decoder (433) can be combined with the above Figure 4 The "remote" decoder (210) is identical to the one described in detail. However, additional brief reference is made to Figure 4 , when symbols are available and the entropy encoder (445) and the parser (320) are capable of losslessly encoding / decoding the symbols into an encoded video sequence, the entropy decoding portion of the video decoder (210) (including the channel (312), the receiver (310), the buffer memory (315), and the parser (320)) may not be fully implemented in the local decoder (433).

[0064] At this point it can be observed that any decoder techniques other than parsing / entropy decoding present in the decoder must also be present in the corresponding encoder in substantially the same functional form. For this reason, the disclosed subject matter focuses on decoder operation. The description of encoder techniques can be abbreviated because they are inverse to the fully described decoder techniques. A more detailed description is only required in certain areas and is provided below.

[0065] As part of the operation, the source encoder (430) may perform motion compensated predictive coding. The motion compensated predictive coding predictively encodes the input frame with reference to one or more previously encoded frames from the video sequence designated as "reference frames." In this manner, the encoding engine (432) encodes the differences between the pixel blocks of the input frame and the pixel blocks of the reference frame, which may be selected as a prediction reference for the input frame.

[0066] The local video decoder (433) may decode the encoded video data of the frame that may be designated as the reference frame based on the symbol created by the source encoder (430). The operation of the encoding engine (432) may advantageously be a lossy process. When the encoded video data is available at the video decoder ( Figure 4 When the video sequence is decoded at a remote location (not shown), the reconstructed video sequence may typically be a copy of the source video sequence with some errors. The local video decoder (433) replicates the decoding process that may be performed by the video decoder on the reference frame and may cause the reconstructed reference frame to be stored in the reference picture memory (434). In this way, the encoder (203) may store a copy of the reconstructed reference frame locally that has common content (absent transmission errors) with the reconstructed reference frame to be obtained by the remote video decoder.

[0067] The predictor (435) may perform a prediction search for the encoding engine (432). That is, for a new frame to be encoded, the predictor (435) may search the reference picture memory (434) for sample data (as candidate reference pixel blocks) or certain metadata, such as reference picture motion vectors, block shapes, etc., that may serve as appropriate prediction references for the new picture. The predictor (435) may operate pixel-by-pixel based on sample blocks to find a suitable prediction reference. In some cases, based on the search results obtained by the predictor (435), it may be determined that the input picture may have prediction references taken from a plurality of reference pictures stored in the reference picture memory (434).

[0068] The controller (450) may manage encoding operations of the source encoder (430), including, for example, setting parameters and subgroup parameters for encoding video data.

[0069] The outputs of all the above functional units may be entropy encoded in an entropy encoder (445). The entropy encoder may perform lossless compression on the symbols generated by the various functional units according to techniques known to those skilled in the art, such as Huffman coding, variable length coding, arithmetic coding, etc., thereby converting the symbols into a coded video sequence.

[0070] The transmitter (440) may buffer the encoded video sequence created by the entropy encoder (445) in preparation for transmission over a communication channel (460), which may be a hardware / software link to a storage device where the encoded video data will be stored. The transmitter (440) may combine the encoded video data from the source encoder (430) with other data to be transmitted, such as encoded audio data and / or ancillary data streams (source not shown).

[0071] The controller (450) can manage the operation of the video encoder (203). During encoding, the controller (450) can assign a certain coded picture type to each coded picture, but this may affect the coding techniques that can be applied to the corresponding picture. For example, a picture can generally be assigned to any of the following picture types.

[0072] An intra picture (I picture) may be a picture that can be encoded and decoded without using any other frame in the sequence as a prediction source. Some video codecs allow different types of intra pictures, including, for example, Independent Decoder Refresh (IDR) pictures. Those skilled in the art are aware of those variants of I pictures and their respective applications and features.

[0073] A predictive picture (P picture) may be a picture that can be encoded and decoded using intra prediction or inter prediction, which uses at most one motion vector and a reference index to predict sample values ​​for each block.

[0074] Bidirectional predictive pictures (B pictures), which can be pictures that can be encoded and decoded using intra prediction or inter prediction, which uses up to two motion vectors and reference indices to predict the sample values ​​of each block. Similarly, multiple predictive pictures can use more than two reference pictures and associated metadata for reconstruction of a single block.

[0075] The source picture may typically be spatially subdivided into blocks of samples (e.g., blocks of 4×4, 8×8, 4×8, or 16×16 samples), and coded block by block. These blocks may be predictively coded with reference to other (already coded) blocks, which are determined according to the coding allocation applied to the block's corresponding picture. For example, blocks of an I picture may be non-predictively coded, or the blocks may be predictively coded (spatial prediction or intra-frame prediction) with reference to already coded blocks of the same picture. Blocks of pixels of a P picture may be non-predictively coded by spatial prediction with reference to one previously coded reference picture or by temporal prediction. Blocks of a B picture may be non-predictively coded by spatial prediction with reference to one or two previously coded reference pictures or by temporal prediction.

[0076] The video encoder (203) may perform encoding operations according to a predetermined video encoding technique or standard, such as ITU-T H.265 Recommendation. In operation, the video encoder (203) may perform various compression operations, including predictive encoding operations that exploit temporal and spatial redundancy in an input video sequence. Thus, the encoded video data may conform to the syntax specified by the video encoding technique or standard used.

[0077] In an embodiment, the transmitter (440) may transmit additional data along with the encoded video. The source encoder (430) may include such data as a portion of the encoded video sequence. The additional data may include other forms of redundant data such as temporal / spatial / SNR enhancement layers, redundant pictures and slices, SEI messages, VUI parameter set fragments, etc.

[0078] Before describing certain aspects of the disclosed subject matter in greater detail, some terminology needs to be introduced that will be referred to in the remainder of this specification.

[0079] Hereinafter, in some cases, a sub-picture refers to a rectangular arrangement of samples, blocks, macroblocks, coding units, or similar entities, which are semantically grouped and can be independently encoded using a changed resolution. One or more sub-pictures can form a picture. One or more encoded sub-pictures can form an encoded picture. One or more sub-pictures can be assembled into a picture, and one or more sub-pictures can be extracted from a picture. In some environments, one or more encoded sub-pictures can be assembled in a compressed domain without transcoding to a sample level and called an encoded picture. And in the same or certain other cases, one or more encoded sub-pictures can be extracted from the encoded picture in the compressed domain.

[0080] Hereinafter, Reference Picture Resampling (RPR) or Adaptive Resolution Change (ARC) refers to a mechanism that allows the resolution of a picture or sub-picture within a coded video sequence to be changed, for example, by reference picture resampling. Hereinafter, RPR / ARC parameters refer to the control information required to perform adaptive resolution change, which may include, for example, filter parameters, scaling factors, resolution of output and / or reference pictures, various control flags, etc.

[0081] The above description focuses on encoding and decoding of a single, semantically independent coded video picture.Before describing the implications of encoding / decoding of multiple sub-pictures with independent RPR / ARC parameters and the additional complexity it implies, the options for signaling RPR / ARC parameters will be described.

[0082] With reference to Figure 5, several novel options for signaling RPR / ARC parameters are shown. As pointed out by each in the options, from the perspective of coding efficiency, complexity and architecture, these options have some advantages and some disadvantages. Video coding standards or technology can select one or more of these options, or options known in the prior art, for signaling RPR / ARC parameters. These options may not be mutually exclusive, and may be interchanged based on the selection of application requirements, involved standard technologies or encoders.

[0083] Categories of RPR / ARC parameters may include:

[0084] - up / down sampling factors, separate or combined in the X and Y dimensions,

[0085] - up / down sampling factor, plus the time dimension, indicating constant speed up / down scaling for a given number of pictures,

[0086] - Either of the above may involve encoding or decoding of one or more possibly short syntax elements which may point to a table containing one or more factors,

[0087] - Resolution of the input picture, output picture, reference picture, coded picture in the X or Y dimension, in units of samples, blocks, macroblocks, coding units (CUs), or any other suitable granularity, and combined or separate (if there is more than one resolution (e.g., one resolution for input pictures and one resolution for reference pictures)), in some cases one set of values ​​may be inferred from another set of values. This can be gated, for example, by using a flag. For a more detailed example, see:

[0088] - "warped" coordinates, similar to those used in H.263 Annex P, and at a suitable granularity as described above (H.263 Annex P defines an efficient way to encode such warped coordinates, but other, potentially more efficient ways are also contemplated. For example, according to an embodiment, the variable-length reversible "Huffman" type codec of the warped coordinates of Annex P is replaced by a binary codec of appropriate length, where, for example, the length of the binary codeword can be derived from the maximum picture size, possibly multiplied by a factor and offset by a value to allow "warping" outside the boundaries of the maximum picture size), and / or

[0089] - upsampling or downsampling filter parameters (in the easiest case, there may be only a single filter for upsampling and / or downsampling. However, in some cases it is advantageous to allow more flexibility in the filter design, and this may require signaling the filter parameters. Such parameters may be selected by an index into a list of possible filter designs, the filter may be fully specified (e.g., by a list of filter coefficients, using appropriate entropy coding techniques), the filter may be selected implicitly by the up / down sampling ratio, which in turn may be signaled according to any of the mechanisms described above, etc.).

[0090] Hereinafter, this specification assumes a codec with a finite set of up / down sampling factors indicated by a codeword (using the same factor in both the X and Y dimensions). The codeword can be conveniently variable length encoded, for example using Ext-Golomb codes common to certain syntax elements in video codec specifications such as H.264 and H.265. A suitable mapping from values ​​to up / down sampling factors can be shown in Table 1 below:

[0091] Table 1

[0092]

[0093] Many similar mappings can be designed, depending on the needs of the application and the capabilities of the upscaling and downscaling mechanisms available in the video compression technology or standard. The table can be extended to more values. The values ​​can also be represented by entropy coding mechanisms other than Ext-Golomb codes, for example, using a binary codec. This may have certain advantages when the resampling factors are of interest outside the video processing engine (first of all the encoder and decoder), such as in MANE. It should be noted that for the (probably) most common cases where no resolution change is required, a shorter Golomb code can be chosen. In the table above, there is only one bit. For the most common cases, this has a codec efficiency advantage over using a binary code.

[0094] The number of entries in the table and their semantics may be fully or partially configurable. For example, the essentials of the table may be communicated in a "high" parameter set, such as a sequence parameter set or a decoder parameter set. Alternatively or additionally, one or more such tables may be defined in a video codec or standard, and may be selected, for example, by a decoder parameter set or a sequence parameter set.

[0095] Hereinafter, we describe how to include the upsampling / downsampling factors (ARC information) encoded as described above in a video codec or standard syntax. Similar considerations can be applied to one or several codewords controlling the up / down sampling filters. When the filters or other data structures require relatively large amounts of data, see the discussion below.

[0096] As shown in the example of FIG. 5A , diagram (500A) shows that H.263 Annex P includes ARC information (502) in the form of four warp coordinates in the picture header (501), specifically in the H.263PLUSPTYPE (503) header extension. This can be a wise design choice when a) there is an available picture header, and b) the ARC information is expected to change frequently. However, when H.263-type signaling is used, the overhead may be quite high, and the scaling factor may not belong to the picture boundary because the picture header may have transient characteristics. In addition, as shown in the example of FIG. 5B , diagram (500B) shows that JVET-M0135 includes PPS information (504), ARC reference information (505), SPS information (507), and target resource table information (506).

[0097] According to an exemplary embodiment, Figure 5C An example (500C) is shown where tile group header information (508) and ARC information (509) are shown. Figure 5D An example (500D) is shown, in which tile group header information (514), ARC reference information (513), SPS information (516), and ARC information (515) are shown, Figure 5E An example (500E) is shown in which one or more adaptation parameter set (APS) information (511) and ARC information (512) are shown.

[0098] Figure 6 An example (600) of a table using adaptive resolution is shown, in which the output resolution in units of samples (613) is encoded. The number (613) refers to output_pic_width_in_luma_samples and output_pic_height_in_luma_samples, which together can define the resolution of the output picture. Elsewhere in the video codec technology or standard, certain restrictions on either value may be defined. For example, a level definition may limit the number of total output samples, which may be the product of the values ​​of those two syntax elements. Furthermore, certain video codec technologies or standards, or external technologies or standards (e.g., system standards) may limit the number range (e.g., one or both dimensions must be divisible by a power of 2) or the aspect ratio (e.g., the width and height must be in a relationship such as 4:3 or 16:9). Such restrictions may be introduced to facilitate hardware implementation or for other reasons that will be understood by those skilled in the art in view of the present disclosure.

[0099] In some applications, it may be advisable for the encoder to instruct the decoder to use a certain reference picture size, rather than implicitly assuming that size is the output picture size. In this example, the syntax element reference_pic_size_present_flag (614) gates the conditional presence of the reference picture dimensions (615) (again, the numbers refer to width and height).

[0100] Some video codecs or standards (e.g., VP9) implement some form of reference picture resampling (signaled in a manner quite different from the disclosed subject matter) to support spatial scalability in conjunction with temporal scalability. Specifically, certain reference pictures may be upsampled to a higher resolution using ARC-type techniques to form the basis of a spatial enhancement layer. Those upsampled pictures may be modified using conventional prediction mechanisms at a higher resolution to increase detail.

[0101] According to an embodiment, the disclosed subject matter can be used in such an environment. In some cases, in the same or another embodiment, the value in the NAL unit header (e.g., the time ID field) can be used to indicate the time layer and the spatial layer. Doing so has some advantages for certain system designs; for example, existing selected forwarding units (SFUs) created and optimized for the time layer select forwarding based on the NAL unit header time ID value and can be used without modification for scalable environments. To achieve this, it may be necessary to indicate the mapping between the coded picture size and the time layer through the time ID field in the NAL unit header.

[0102] In an embodiment, information about inter-layer dependencies may be signaled in a VPS (or DPS, SPS or SEI message). The inter-layer dependency information may be used to identify which layer is used as a reference layer to decode the current layer. When n is greater than m and two pictures picA and picB belong to the same access unit, the decoded picture picA in the direct dependent layer with nuh_layer_id equal to m may be used as a reference picture for the picture picB with nuh_layer_id equal to n.

[0103] In the same or other embodiments, the inter-layer reference picture (ILRP) list and the inter-prediction reference picture (IPRP) list can be explicitly signaled together in the slice header (or parameter set). Both the ILRP list and the IPRP list can be used to construct the forward and backward prediction reference picture lists.

[0104] In the same or other embodiments, syntax elements in the VPS (or other parameter set) may indicate whether each layer is dependent or independent. Figure 7 In the example (700) in, the syntax element vps_max_layers_minus1 (703) plus 1 can specify the maximum number of layers allowed in one or more (possibly all) CVSs, reference VPS (701). vps_all_independent_layers_flag (704) equal to 1 can specify that all layers in the CVS are independently coded, that is, inter-layer prediction is not used. vps_all_independent_layers_flag (704) equal to 0 can specify that one or more layers in the CVS can use inter-layer prediction. When not present, the value of vps_all_independent_layers_flag can be inferred to be equal to 1. When the value of vps_all_independent_layers_flag is equal to 1, the value of vps_independent_layer_flag[i] (706) is inferred to be equal to 1. When vps_all_independent_layers_flag is equal to 0, the value of vps_independent_layer_flag[0] is inferred to be equal to 1. Reference Figure 7 , vps_independent_layer_flag[i](706) equal to 1 can specify that the layer with index i does not use inter-layer prediction. vps_independent_layer_flag[i] equal to 0 can specify that the layer with index i can use inter-layer prediction, and vps_layer_dependency_flag[i] is present in the VPS. vps_direct_dependency_flag[i][j](707) equal to 0 can specify that the layer with index j is not the direct reference layer of the layer with index i. vps_direct_dependency_flag[i][j] equal to 1 can specify that the layer with index j is the direct reference layer of the layer with index i. When i and j are in the range of 0 to vps_max_layers_minus1 (including vps_max_layers_minus1) and vps_direct_dependency_flag[i][j] does not exist, it is inferred that the value of vps_direct_dependency_flag[i][j] is equal to 0. Specify the variable DirectDependentLayerIdx[i][j] of the jth directly dependent layer of the i-th layer, which is derived as follows:

[0105] for(i=1;i<=vps_max_layers_minus1;i++)

[0106] if(!vps_independent_layer_flag[i])

[0107] for(j=i-1,k=0;j>=0;j--)

[0108] if(vps_direct_dependency_flag[i][j])

[0109] DirectDependentLayerIdx[i][k++]=j

[0110] The variable GeneralLayerIdx[i] that specifies the layer index of the layer whose nuh_layer_id is equal to vps_layer_id[i] is derived as follows:

[0111] for(i=0;i<=vps_max_layers_minus1;i++)

[0112] GeneralLayerIdx[vps_layer_id[i]]=i

[0113] In the same or other embodiments, reference Figure 7 , when the value of vps_max_layers_minus1 is greater than 0 and the value of vps_all_independent_layers_flag is equal to 0, vps_output_layers_mode and vps_output_layer_flags[i] may be signaled. vps_output_layers_mode (708) equal to 0 may specify that only the highest layer is output. vps_output_layers_mode equal to 1 may specify that all layers may be output. vps_output_layers_mode equal to 2 may specify that the layers output are the layers for which vps_output_layer_flag[i] (709) is equal to 1. The value of vps_output_layers_mode shall be in the range of 0 to 2, inclusive. The value of vps_output_layers_mode of 3 may be reserved for future use. When not present, the value of vps_output_layers_mode may be inferred to be equal to 1. vps_output_layer_flag[i] equal to 1 may specify that the i-th layer is output. vps_output_layer_flag[i] equal to 0 may specify that the i-th layer is not output.

[0114] List OutputLayerFlag[i], where value 1 specifies outputting layer i, and value 0 specifies not outputting layer i, as follows:

[0115]

[0116] In the same or other embodiments, the output of the current picture may be specified as follows:

[0117] -If PictureOutputFlag is equal to 1 and DpbOutputTime[n] is equal to CpbRemovalTime[n], output the current picture.

[0118] -Otherwise, if PictureOutputFlag is equal to 0, the current picture is not output, but the current picture is stored in the DPB according to the provisions of the clause.

[0119] - Otherwise (PictureOutputFlag is equal to 1 and DpbOutputTime[n] is greater than CpbRemovalTime[n]), the current picture is output later and will be stored in the DPB (as specified in the clause) and output at time DpbOutputTime[n] unless it was not output at a time before DpbOutputTime[n], either by decoding or inference with no_output_of_prior_pics_flag equal to 1. When output, the picture is cropped using the consistent cropping window specified for the picture in the PPS.

[0120] In the same or other embodiments, PictureOutputFlag can be set as follows:

[0121] - Set PictureOutputFlag to 0 if one of the following conditions is true:

[0122] -The current image is a RASL image and the associated IRAP image

[0123] NoIncorrectPicOutputFlag is equal to 1.

[0124] -gdr_enabled_flag is equal to 1, and the current picture is a GDR picture with NoIncorrectPicOutputFlag equal to 1.

[0125] -gdr_enabled_flag is equal to 1, the current picture is associated with a GDR picture with NoIncorrectPicOutputFlag equal to 1, and the PicOrderCntVal of the current picture is less than the RpPicOrderCntVal of the associated GDR picture.

[0126] -vps_output_layers_mode is equal to 0 or 2, and

[0127] OutputLayerFlag[GeneralLayerIdx[nuh_layer_id]] is equal to 0.

[0128] - Otherwise, set PictureOutputFlag equal to pic_output_flag.

[0129] In the same or other embodiments, alternatively, PictureOutputFlag can be set as follows:

[0130] - Set PictureOutputFlag equal to 0 if one of the following conditions is true:

[0131] - The current image is a RASL image and the related IRAP image

[0132] NoIncorrectPicOutputFlag is equal to 1.

[0133] -gdr_enabled_flag is equal to 1, and the current picture is a GDR picture with NoIncorrectPicOutputFlag equal to 1.

[0134] -gdr_enabled_flag is equal to 1, the current picture is associated with a GDR picture with NoIncorrectPicOutputFlag equal to 1, and the PicOrderCntVal of the current picture is less than the RpPicOrderCntVal of the associated GDR picture.

[0135] -vps_output_layers_mode is equal to 0 and the current access unit contains PictureOutputFlag

[0136] A picture equal to 1, where the nuh_layer_id nuhLid of the picture is greater than the nuh_layer_id nuhLid of the current picture, and the picture belongs to the output layer (i.e.,

[0137] OutputLayerFlag[GeneralLayerIdx[nuhLid]] is equal to 1).

[0138] -vps_output_layers_mode is equal to 2, and

[0139] OutputLayerFlag[GeneralLayerIdx[nuh_layer_id]] is equal to 0.

[0140] - Otherwise, set PictureOutputFlag equal to pic_output_flag.

[0141] In the same or other embodiments, a flag in the VPS (or another parameter set) may indicate whether to signal an ILRP list for the current slice (or picture). Figure 8 In the example (800) in FIG. 5 , inter_layer_ref_pics_present_flag equal to 0 may specify that no ILRP is used for inter prediction of any coded picture in the CVS. inter_layer_ref_pics_flag equal to 1 may specify that ILRPs may be used for inter prediction of one or more coded pictures in the CVS.

[0142] In the same or other embodiments, when layer k is a dependent layer, the inter-layer reference picture ILRP list for pictures in layer k may or may not be signaled. However, when layer k is an independent layer, the ILRP list for pictures in layer k shall not be signaled, and no ILRP shall be included in the reference picture list. When sps_video_parameter_set_id is equal to 0, when nuh_layer_id is equal to 0, or when vps_independent_layer_flag[GeneralLayerIdx[nuh_layer_id]] is equal to 1, the value of inter_layer_ref_pics_present_flag may be set equal to 0.

[0143] In the same or other embodiments, reference Fig. 9 In the example (900) in FIG. 1 , reference picture lists RefPicList[0] and RefPicList[1] may be constructed as follows:

[0144]

[0145]

[0146] The above-described techniques for signaling adaptive resolution parameters may be implemented as computer software via computer-readable instructions and physically stored in one or more computer-readable media. Fig.10 A computer system (1000) is shown that is suitable for implementing certain embodiments of the disclosed subject matter.

[0147] The computer software may be encoded in any suitable machine code or computer language, and may be assembled, compiled, linked, or other mechanisms to create a code including instructions, which may be directly executed by a computer central processing unit (CPU), graphics processing unit (GPU), or the like, or executed by decoding, microcode, or the like.

[0148] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablets, servers, smartphones, gaming devices, IoT devices, etc.

[0149] Fig.10 The components shown for the computer system (1000) are exemplary in nature and are not intended to limit the scope of use or functionality of the computer software implementing the embodiments of the present application. The configuration of the components should not be interpreted as having any dependency or requirement on any component or combination of components shown in the exemplary embodiment of the computer system (1000).

[0150] The computer system (1000) may include certain human-computer interface input devices. Such human-computer interface input devices may respond to input from one or more human users through tactile input (e.g., keyboard input, sliding, data glove movement), audio input (e.g., sound, applause), visual input (e.g., gestures), and olfactory input (not shown). The human-computer interface devices may also be used to capture certain media that are not necessarily directly related to human conscious input, such as audio (e.g., speech, music, ambient sound), images (e.g., scanned images, photographic images obtained from a still image camera), and videos (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0151] The human-computer interface input device may include one or more of the following (only one of which is depicted): keyboard (1001), mouse (1002), touchpad (1003), touch screen (1010), joystick (1005), microphone (1006), scanner (1007), camera (1008).

[0152] The computer system (1000) may also include certain human-computer interface output devices. Such human-computer interface output devices may stimulate one or more human user senses through, for example, tactile output, sound, light, and smell / taste. Such human-computer interface output devices may include tactile output devices (e.g., tactile feedback through a touch screen (1010) or a joystick (1005), but there may also be tactile feedback devices that are not used as input devices), audio output devices (e.g., speakers (1009), headphones (not shown)), visual output devices (e.g., screens (1010) including cathode ray tube screens, liquid crystal screens, plasma screens, organic light emitting diode screens, each of which has or does not have a touch screen input function, each of which has or does not have a tactile feedback function - some of which can output two-dimensional visual output or output of more than three dimensions through means such as stereoscopic image output; virtual reality glasses (not shown), holographic displays, and smoke boxes (not shown)) and printers (not shown).

[0153] The computer system (1000) may also include human-accessible storage devices and their associated media, such as optical media including high-density read-only / rewritable optical disks (CD / DVD ROM / RW) (1020) with CD / DVD or similar media (1021), thumb drives (1022), removable hard disk drives or solid state drives (1023), traditional magnetic media such as tapes and floppy disks (not shown), special-purpose ROM / ASIC / PLD-based devices such as security software protectors (not shown), and the like.

[0154] Those skilled in the art should also understand that the term "computer-readable media" used in connection with the disclosed subject matter does not include transmission media, carrier waves, or other transient signals.

[0155] The computer system (1000) may also include an interface to one or more communication networks. For example, the network may be wireless, wired, or optical. The network may also be a local area network, a wide area network, a metropolitan area network, an in-vehicle network, an industrial network, a real-time network, a delay-tolerant network, and the like. The network also includes local area networks such as Ethernet, wireless local area networks, cellular networks (GSM, 3G, 4G, 5G, LTE, etc.), television wired or wireless wide area digital networks (including cable television, satellite television, and terrestrial broadcast television), in-vehicle and industrial networks (including CANBus), and the like. Some networks typically require an external network interface adapter for connecting to some universal data ports or peripheral buses (1049) (e.g., a USB port of the computer system (1000)); other systems are typically integrated into the core of the computer system (1000) by connecting to a system bus as described below (e.g., an Ethernet interface integrated into a PC computer system or a cellular network interface integrated into a smart phone computer system). By using any of these networks, the computer system (1000) can communicate with other entities. The communication can be one-way, for receiving only (e.g., wireless television), one-way for sending only (e.g., CAN bus to certain CAN bus devices), or two-way, such as to other computer systems via a local or wide area digital network. Each of the above networks and network interfaces can use certain protocols and protocol stacks.

[0156] The above-mentioned human-machine interface devices, human-accessible storage devices, and network interfaces may be connected to the core (1040) of the computer system (1000).

[0157] The core (1040) may include one or more central processing units (CPUs) (1041), graphics processing units (GPUs) (1042), dedicated programmable processing units in the form of field programmable gate arrays (FPGAs) (1043), hardware accelerators for specific tasks (1044), etc. These devices, as well as read-only memory (ROM) (1045), random access memory (1046), internal mass storage (e.g., internal non-user accessible hard disk drives, solid-state drives, etc.) (1047), etc., may be connected via a system bus (1048). In some computer systems, the system bus (1048) may be accessed in the form of one or more physical plugs so that it can be expanded by additional central processing units, graphics processing units, etc. Peripheral devices may be directly attached to the core's system bus (1048), or connected via a peripheral bus (1049). The architecture of the peripheral bus includes PCI, USB, etc.

[0158] The CPU (1041), GPU (1042), FPGA (1043) and accelerator (1044) can execute certain instructions, which can be combined to form the above-mentioned computer code. The computer code can be stored in ROM (1045) or RAM (1046). Transition data can also be stored in RAM (1046), while permanent data can be stored in, for example, internal mass storage (1047). Fast storage and retrieval of any memory device can be achieved by using a cache memory, which can be closely associated with one or more CPUs (1041), GPUs (1042), mass storage (1047), ROM (1045), RAM (1046), etc.

[0159] The computer readable medium may have computer code thereon for performing various computer-implemented operations. The medium and computer code may be specially designed and constructed for the purpose of this application, or may be medium and code well known and available to those skilled in the art of computer software.

[0160] As an example and not a limitation, a computer system having an architecture (1000), in particular a core (1040), can be provided as a processor (including a CPU, a GPU, an FPGA, an accelerator, etc.) to execute software contained in one or more tangible computer-readable media. Such a computer-readable medium can be a medium associated with the above-mentioned user-accessible mass storage, as well as a specific memory of the core (1040) having non-volatility, such as a core internal mass storage (1047) or a ROM (1045). Software implementing various embodiments of the present application can be stored in such a device and executed by the core (1040). Depending on specific needs, the computer-readable medium may include one or more storage devices or chips. The software can enable the core (1040), in particular the processor therein (including a CPU, a GPU, an FPGA, etc.) to perform a specific process or a specific part of a specific process described herein, including defining a data structure stored in the RAM (1046) and modifying such a data structure according to a software-defined process. Additionally or alternatively, the computer system may provide functionality hardwired in logic or otherwise contained in circuitry (e.g., accelerator (1044)) that may operate in place of or in conjunction with software to perform a particular process or a particular portion of a particular process described herein. Where appropriate, references to software may include logic and vice versa. Where appropriate, references to computer-readable media may include circuitry (e.g., an integrated circuit (IC)) storing the executing software, circuitry containing the executing logic, or both. The present application includes any suitable combination of hardware and software.

[0161] Although the present application has described a number of exemplary embodiments, various changes, arrangements and various equivalent substitutions of the embodiments are within the scope of the present application. Therefore, it should be understood that those skilled in the art can design a variety of systems and methods, which, although not explicitly shown or described herein, embody the principles of the present application and are therefore within the spirit and scope of the present application.

Claims

1. A video decoding method, characterized in that: The method comprises: Parsing at least one video parameter set VPS, the at least one VPS comprising at least one syntax element, wherein the at least one syntax element indicates whether at least one layer in the scalable bitstream is one of the following: (i) a dependent layer of the scalable bitstream; (ii) an independent layer of the scalable bitstream; Decoding the first picture in the dependent layer by parsing the inter-layer reference picture ILRP list; and without parsing the ILRP list, decoding the picture in the independent layer; without signaling the ILRP list, decoding the second picture in the dependent layer; The at least one VPS further comprises a flag indicating whether the ILRP list is signaled for a current picture; The method further comprises: when there is no video parameter set output layer mode vps_output_layers_mode, inferring that the value of vps_output_layers_mode is equal to 1, wherein vps_output_layers_mode is equal to 0, indicating that only the highest layer is output, vps_output_layers_mode is equal to 1, indicating that all layers are output, vps_output_layers_mode is equal to 2, indicating that the output layer is the layer where vps_output_layer_flag[i] is equal to 1, and the value of vps_output_layers_mode should be in the range of 0 to 2; When vps_output_layers_mode is equal to 0 and the output layer flag OutputLayerFlag[GeneralLayerIdx[nuh_layer_id]] is equal to 0, the value of the picture output flag PictureOutputFlag is determined to be 0, where when the value of OutputLayerFlag[i] is equal to 1, it means that the i-th layer is output; when the value of OutputLayerFlag[i] is equal to 0, it means that the i-th layer is not output; GeneralLayerIdx[nuh_layer_id] represents the layer index identified as nuh_layer_id; and when PictureOutputFlag is equal to 0, it means that the current picture is not output.

2. The method according to claim 1, characterized in that Decoding the picture in the independent layer includes parsing a reference picture list, the reference picture list not including any decoded picture of another layer.

3. The method according to claim 1, characterized in that The inter-layer reference picture list includes decoded pictures of another layer.

4. The method according to any one of claims 1 to 3, characterized in that: Parsing the at least one VPS further includes determining whether another syntax element indicates a maximum number of layers.

5. The method according to any one of claims 1 to 3, characterized in that: The parsing of at least one VPS further includes determining whether the VPS includes a flag indicating whether another layer in the scalable bitstream is a reference layer for the at least one layer.

6. The method according to claim 5, characterized in that The parsing of the at least one VPS further includes determining whether the flag indicates that the other layer is a reference layer for the at least one layer by specifying an index of the other layer and an index of the at least one layer.

7. The method according to claim 6, characterized in that in, The flag is a variable DirectDependentLayerIdx[i][j] that specifies the jth directly dependent layer of the i-th layer. The value of the variable is derived according to the following sentence: Among them, vps_max_layers_minus1 plus 1 indicates the maximum number of layers allowed in one or more coded video sequences CVS; vps_independent_layer_flag[i] equal to 1 indicates that the layer indexed by i does not use inter-layer prediction, vps_independent_layer_flag[i] equal to 0 indicates that the layer indexed by i can use inter-layer prediction, and vps_layer_dependence_flag[i] exists in the VPS; vps_direct_dependency_flag[i][j] equal to 0 indicates that the layer indexed by j is not the direct reference layer of the layer indexed by i, vps_direct_dependency_flag[i][j] equal to 1 indicates that the layer indexed by j is the direct dependent layer of the layer indexed by i, the values ​​of i and j are above 0 and below vps_max_layers_minus1, and the value of k is in the range of 0 to vps_max_layers_minus1 minus 1.

8. The method according to claim 5, characterized in that The parsing of the at least one VPS further includes determining whether the flag indicates that the other layer is not a reference layer of the at least one layer by specifying an index of the other layer and an index of the at least one layer.

9. A video encoding method for generating a bit stream, characterized in that The method comprises: Encoding at least one video parameter set (VPS), the at least one VPS comprising at least one syntax element, wherein the at least one syntax element indicates whether at least one layer in the scalable bitstream is one of: (i) a dependency layer of the scalable bitstream; (ii) an independent layer of the scalable bitstream; encoding the first picture in the dependent layer by signaling an inter-layer reference picture ILRP list; and encoding the picture in the independent layer and the second picture in the dependent layer without notifying the ILRP list; wherein the VPS further comprises a flag, the flag indicating whether to signal the ILRP list for the current picture; Wherein, the method further comprises: When there is no video parameter set output layer mode vps_output_layers_mode, the value of vps_output_layers_mode is inferred to be equal to 1, where vps_output_layers_mode equal to 0 indicates that only the highest layer is output, vps_output_layers_mode equal to 1 indicates that all layers are output, and vps_output_layers_mode equal to 2 indicates that the output layer is the layer with vps_output_layer_flag[i] equal to 1. The value of vps_output_layers_mode should be in the range of 0 to 2; When vps_output_layers_mode is equal to 0 and the output layer flag OutputLayerFlag[GeneralLayerIdx[nuh_layer_id]] is equal to 0, the picture output flag PictureOutputFlag is set to 0, where when the value of OutputLayerFlag[i] is equal to 1, it means that the i-th layer is output; when the value of OutputLayerFlag[i] is equal to 0, it means that the i-th layer is not output; GeneralLayerIdx[nuh_layer_id] represents the layer index identified as nuh_layer_id; and when PictureOutputFlag is equal to 0, it means that the current picture is not output.

10. A video encoding device, characterized in that: The device comprises: at least one memory for storing computer program code; At least one processor is configured to access the computer program code and execute the method according to claim 9 according to instructions of the computer program code.

11. A video decoding device, characterized in that: The device comprises: at least one memory for storing computer program code; At least one processor is configured to access the computer program code and execute the method according to any one of claims 1 to 8 according to instructions of the computer program code.

12. A video decoding device, characterized in that: The device comprises: A parsing module, configured to parse at least one video parameter set VPS, wherein the at least one VPS comprises at least one syntax element, wherein the at least one syntax element indicates whether at least one layer in a scalable bitstream is one of the following: (i) a dependent layer of the scalable bitstream; (ii) an independent layer of the scalable bitstream; A dependent layer picture decoding module, configured to decode the first picture in the dependent layer by parsing an inter-layer reference picture ILRP list; and an independent layer picture decoding module, configured to decode the picture in the independent layer without parsing the ILRP list; and decode the second picture in the dependent layer without signaling the ILRP list; The at least one VPS further comprises a flag indicating whether the ILRP list is signaled for a current picture; When there is no video parameter set output layer mode vps_output_layers_mode, it is inferred that the value of vps_output_layers_mode is equal to 1, wherein the vps_output_layers_mode is equal to 0 to indicate that only the highest layer is output, the vps_output_layers_mode is equal to 1 to indicate that all layers are output, and vps_output_layers_mode is equal to 2 to indicate that the output layer is the layer with vps_output_layer_flag[i] equal to 1, and the value of vps_output_layers_mode should be in the range of 0 to 2; When vps_output_layers_mode is equal to 0 and the output layer flag OutputLayerFlag[GeneralLayerIdx[nuh_layer_id]] is equal to 0, the value of the picture output flag PictureOutputFlag is determined to be 0, where when the value of OutputLayerFlag[i] is equal to 1, it means that the i-th layer is output; when the value of OutputLayerFlag[i] is equal to 0, it means that the i-th layer is not output; GeneralLayerIdx[nuh_layer_id] represents the layer index identified as nuh_layer_id; and when PictureOutputFlag is equal to 0, it means that the current picture is not output.

13. A video encoding device, characterized in that: The device comprises: A first encoding module is configured to encode at least one video parameter set VPS, wherein the at least one VPS includes at least one syntax element, wherein the at least one syntax element indicates whether at least one layer in a scalable bitstream is one of the following: (i) a dependency layer of the scalable bitstream; (ii) an independent layer of the scalable bitstream; A second encoding module is configured to encode the first picture in the dependent layer by signaling an inter-layer reference picture ILRP list; and encode the picture in the independent layer and the second picture in the dependent layer without notifying the ILRP list; wherein the VPS further comprises a flag, the flag indicating whether to signal the ILRP list for the current picture; An inference module, used for inferring that the value of vps_output_layers_mode is equal to 1 when there is no video parameter set output layer mode vps_output_layers_mode, wherein vps_output_layers_mode is equal to 0, indicating that only the highest layer is output, vps_output_layers_mode is equal to 1, indicating that all layers are output, and vps_output_layers_mode is equal to 2, indicating that the output layer is the layer with vps_output_layer_flag[i] equal to 1, and the value of vps_output_layers_mode should be in the range of 0 to 2; A setting module is used to set the picture output flag PictureOutputFlag to 0 when vps_output_layers_mode is equal to 0 and the output layer flag OutputLayerFlag[GeneralLayerIdx[nuh_layer_id]] is equal to 0, wherein when the value of OutputLayerFlag[i] is equal to 1, it means that the i-th layer is output, and when the value of OutputLayerFlag[i] is equal to 0, it means that the i-th layer is not output, GeneralLayerIdx[nuh_layer_id] represents the layer index identified as nuh_layer_id, and when PictureOutputFlag is equal to 0, it means that the current picture is not output.

14. A non-volatile computer readable medium, characterized in that: The non-volatile computer-readable medium stores a program, and the program enables a computer to execute the method according to any one of claims 1 to 9.

15. A method for processing a video bitstream, characterized in that: The video bit stream is generated according to the encoding method according to claim 9, or is decoded based on the decoding method according to any one of claims 1 to 8.

Images