Method and apparatus for reconstructing an encoded enhanced layer picture

Abstract

A method for reconstructing an encoded enhanced layer picture, comprising: decoding at least one independent picture header of an encoded picture of a reference layer, the encoded picture of the reference layer being referenced by the encoded enhanced layer picture; decoding a dependent picture header of the encoded enhanced layer picture; and performing picture header prediction, which includes one or both of the following: predicting at least one value of at least one syntax element of the dependent picture header according to at least one value of at least one corresponding syntax element of the at least one independent picture header; and inferring at least one value of a non-existent syntax element of the dependent picture header to be equal to at least one value of an existent syntax element of the at least one independent picture header.

Description

[0001] Priority Information

[0002] This application claims priority to U.S. Provisional Applications No. 62 / 947,236 and 62 / 947,226, filed on December 12, 2019, and U.S. Application No. 17 / 071,479, filed on October 15, 2020, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to video encoding and decoding technologies, and more particularly, to methods and devices, apparatuses, and storage media for reconstructing encoded enhancement layer pictures. Background Art

[0004] It has been well known for decades to use inter-picture prediction with motion compensation for video encoding and decoding. Uncompressed digital video may include a series of pictures, each picture having a spatial dimension such as 1920×1080 luminance samples and associated chrominance samples. The series of pictures has a fixed or variable picture rate (also informally referred to as the frame rate), such as 60 pictures per second or 60 Hz. Uncompressed video has very high bitrate requirements. For example, a 1080p60 4:2:0 video with 8 bits per sample (1920x1080 luminance sample resolution, 60 Hz frame rate) requires a bandwidth of nearly 1.5 Gbit / s. An hour of such video would require more than 600 GB of storage space.

[0005] One purpose of video encoding and decoding is to reduce redundant information in the input video signal through compression. Video compression can help reduce the requirements for the above-mentioned bandwidth or storage space, and in some cases, can reduce by two or more orders of magnitude. Both lossless and lossy compression, as well as combinations of the two, can be employed. Lossless compression refers to a technique for reconstructing an exact copy of the original signal from the compressed original signal. When lossy compression is used, the reconstructed signal may not be exactly the same as the original signal, but the distortion between the original signal and the reconstructed signal is small enough that the reconstructed signal can be used for the intended application. Lossy compression is widely used in video. The amount of allowable distortion depends on the application. For example, users of some consumer streaming applications can tolerate higher distortion compared to users of television applications. The achievable compression ratio reflects that higher allowed / tolerated distortion can result in a higher compression ratio.

[0006] Video encoders and decoders can utilize several major categories of techniques, such as including motion compensation, transformation, quantization, and entropy coding, some of which will be introduced below.

[0007] The concept of dividing an encoded video bitstream into multiple packets for transmission over a packet network has been used for decades. In the early days, most video codec standards and technologies were optimized for robot-oriented transmission and defined bitstreams. Packetization occurs, for example, in the system layer interface specified in the Real-Time Transport Protocol (RTP) payload format. With the emergence of Internet connections suitable for the widespread use of video on the Internet, video codec standards have reflected this prominent use case through the concept distinction of the video coding layer (VCL) and the network abstraction layer (NAL). NAL units were introduced in H.264 in 2003 and have since been retained in some video codec standards and technologies with only minor modifications.

[0008] In many cases, NAL units can be considered the smallest entities on which a decoder can operate without having to decode all previous NAL units of the encoded video sequence. In this context, NAL units enable certain error resilience techniques as well as certain bitstream processing techniques, including bitstream pruning, through media-aware network entities (MANEs) such as selective forwarding units (SFUs) or multi-point control units (MCUs).

[0009] Figure 1 The relevant parts of the syntax diagrams of the NAL unit headers according to H.264 (101) and H.265 (102) are shown, without their respective extensions in both cases. In both cases, the forbidden_zero_bit is a zero bit used to prevent start code emulation in certain system layer environments. The nal_unit_type syntax element refers to the type of data carried by the NAL unit, which can be, for example, one of certain slice types, parameter set types, supplementary enhancement information (SEI) messages. The H.265 NAL unit header also includes nuh_layer_id and nuh_temporal_id_plus1, which indicate the spatial / SNR and temporal layers of the encoded picture to which the NAL unit belongs.

[0010] It can be observed that the NAL unit headers only include fixed-length codewords that are easily parsed and have no parsing dependencies on other data in the bitstream, such as other NAL unit headers, parameter sets, etc. Since the NAL unit headers are the first octets in the NAL units, MANEs can easily extract and parse them and operate on them. In contrast, other high-level syntax elements, such as slice or tile headers, are less accessible to MANEs because they may require maintaining parameter set context and / or need to process variable-length or arithmetic-coded code points.

[0011] It can further be observed that Figure 1 the NAL unit header shown does not include information that can associate the NAL unit to an encoded picture composed of multiple NAL units (e.g., including multiple tiles or slices, where at least some of the tiles or slices are packed in separate NAL units).

[0012] Certain transport technologies, such as RTP (RFC 3550), MPEG system standards, ISO file formats, etc., may include certain information, typically in the form of timing information, such as presentation time (in the case of MPEG and ISO file formats) or capture time (in the case of RTP), which can be easily accessed by the MANE and help associate their respective transport units with the encoded picture. However, the semantics of this information may vary depending on the transport / storage technology and may not have a direct relationship with the picture structure used in video coding / decoding. Therefore, this information is at best tentative and may not be particularly suitable for identifying whether NAL units in a NAL unit stream belong to the same encoded picture.

[0013] Conventional video syntax redundantly signals parameters referenced by one or more slices or VCL NAL units. SUMMARY OF THE INVENTION

[0014] To solve the above problems, embodiments of the present application provide a method and device, apparatus, and storage medium for reconstructing an encoded enhancement layer picture.

[0015] According to an embodiment, a method for reconstructing an encoded enhancement layer picture is provided, including: decoding at least one independent picture header of an encoded picture of a reference layer, the encoded picture of the reference layer being referenced by the encoded enhancement layer picture; decoding a dependent picture header of the encoded enhancement layer picture; and performing picture header prediction, which includes one or both of the following: predicting at least one value of at least one syntax element of the dependent picture header according to at least one value of at least one corresponding syntax element of the at least one independent picture header; and inferring that at least one value of a non-existent syntax element of the dependent picture header is equal to at least one value of an existent syntax element of the at least one independent picture header.

[0016] According to an embodiment, a device for reconstructing an encoded enhanced layer picture is provided, including: at least one memory for storing program code; and at least one processor for accessing the at least one memory and operating according to the program code, the program code including: first decoding code for decoding at least one independent picture header of an encoded picture of a reference layer, the encoded picture of the reference layer being referred to by the encoded enhanced layer picture; second decoding code for decoding a dependent picture header of the encoded enhanced layer picture; and execution code for performing picture header prediction, which includes one or both of the following: predicting at least one value of at least one syntax element of the dependent picture header according to at least one value of at least one corresponding syntax element of the at least one independent picture header; and inferring that at least one value of a non-existent syntax element of the dependent picture header is equal to at least one value of an existent syntax element of the at least one independent picture header.

[0017] In an embodiment, the at least one processor of the device accesses the at least one memory and operates according to the program code stored in the at least one memory to execute the method for reconstructing an encoded enhanced layer picture described in the embodiment.

[0018] According to an embodiment, a device for reconstructing an encoded enhanced layer picture is provided, including: first decoding code for decoding at least one independent picture header of an encoded picture of a reference layer, the encoded picture of the reference layer being referred to by the encoded enhanced layer picture; second decoding code for decoding a dependent picture header of the encoded enhanced layer picture; and execution code for performing picture header prediction, which includes one or both of the following: predicting at least one value of at least one syntax element of the dependent picture header according to at least one value of at least one corresponding syntax element of the at least one independent picture header; and inferring that at least one value of a non-existent syntax element of the dependent picture header is equal to at least one value of an existent syntax element of the at least one independent picture header.

[0019] According to an embodiment, a non - volatile computer - readable storage medium is provided for storing instructions that, when executed by one or more processors, cause the one or more processors to perform: decoding at least one independent picture header of an encoded picture of a reference layer, the encoded picture of the reference layer being referenced by the encoded enhancement - layer picture; decoding a dependent picture header of the encoded enhancement - layer picture; and performing picture - header prediction, which includes one or both of the following: predicting at least one value of at least one syntax element of the dependent picture header according to at least one value of at least one corresponding syntax element of the at least one independent picture header; and inferring that at least one value of a non - existent syntax element of the dependent picture header is equal to at least one value of an existent syntax element of the at least one independent picture header.

[0020] In an embodiment, when the instructions stored in the non - volatile computer - readable storage medium are executed by one or more processors, the one or more processors are caused to perform the method for reconstructing an encoded enhancement - layer picture described in the embodiment.

[0021] In an embodiment of the present application, predicting at least one value of at least one syntax element of a dependent picture header according to at least one value of at least one corresponding syntax element of at least one independent picture header and inferring that at least one value of a non - existent syntax element of the dependent picture header is equal to at least one value of an existent syntax element of the at least one independent picture header can eliminate the need to signal parameters referenced by one or more slices or VCL NAL units, improving the encoding and decoding efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a schematic diagram of NAL unit headers according to H.264 and H.265.

[0023] Figure 2 is a simplified block diagram of a communication system according to an embodiment.

[0024] Figure 3 is a schematic diagram of the placement of a video encoder and a video decoder in a streaming environment according to an embodiment.

[0025] Figure 4 is a functional block diagram of a video decoder according to an embodiment.

[0026] Figure 5 is a functional block diagram of a video encoder according to an embodiment.

[0027] Figure 6 is a schematic diagram of a NAL unit header according to an embodiment.

[0028] Figure 7Schematic diagram of a picture header having an independent mode and a dependent mode according to an embodiment.

[0029] Figure 8A Schematic diagram of a NAL unit structure in which each PU has one PH VCL unit in an AU according to an embodiment.

[0030] Figure 8B Schematic diagram of a NAL unit structure having an independent picture header (PH) VCL unit and one or more dependent PH VCL units in a picture unit (PU) of an access unit (AU) according to an embodiment.

[0031] Figure 8C Schematic diagram of a NAL unit structure in which each PU has one or more PH VCL units in an AU according to an embodiment.

[0032] Figure 8D Flowchart of a method for signaling a dependent picture header and an independent picture header according to an embodiment.

[0033] Figure 8E Block diagram of a device for signaling a dependent picture header and an independent picture header according to an embodiment.

[0034] Figure 9 Schematic diagram of a computer system according to an embodiment. Detailed implementation

[0035] Figure 2 Simplified block diagram of a communication system (200) according to an embodiment. The communication system (200) includes at least two terminal devices (210, 220), and the terminal devices can communicate with each other through a network (250). For unidirectional data transmission, the first terminal device (210) can encode video data at a local location for transmission to the second terminal device (220) through the network (250). The second terminal device (220) can receive the encoded video data of another terminal from the network (250), decode the encoded video data to recover the video data, and display the recovered video data. Unidirectional data transmission is more common in applications such as media services.

[0036] Figure 2Shows a second pair of terminal devices (230, 240) that support two-way transmission of encoded video, which can occur, for example, during a video conference. For two-way data transmission, each of the third terminal device (230) and the fourth terminal device (240) can encode video data collected at a local location for transmission over a network (250) to the other of the third terminal device (230) and the fourth terminal device (240). Each of the third terminal device (230) and the fourth terminal device (240) can also receive the encoded video data transmitted by the other of the third terminal device (230) and the fourth terminal device (240), and can decode the encoded video data and display the restored video data on a local display device.

[0037] In Figure 2 , the first terminal device (210), the second terminal device (220), the third terminal device (230), and the fourth terminal device (240) can be servers, personal computers, and smart phones, but the principles of the embodiments are not limited thereto. The embodiments are applicable to laptop computers, tablet computers, media players, and / or dedicated video conferencing devices. The network (250) represents any number of networks that transmit encoded video data between the first terminal device (210), the second terminal device (220), the third terminal device (230), and the fourth terminal device (240), including, for example, wired and / or wireless communication networks. The communication network (250) can exchange data in circuit-switched and / or packet-switched channels. The network can include a telecommunications network, a local area network, a wide area network, and / or the Internet. For the purposes of this application, unless otherwise explained below, the architecture and topology of the network (250) may be immaterial to the operations disclosed in this application.

[0038] Figure 3 Shows the placement of a video encoder and a video decoder in a streaming environment according to an embodiment. The subject matter disclosed in this application is equally applicable to other video-supported applications, including, for example, video conferencing, digital TV, storing compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0039] A streaming system may include an acquisition subsystem (313), which may include a video source (301) such as a digital camera that creates a stream of, for example, uncompressed video samples (302). The video sample stream (302) is depicted as a thick line as compared to an encoded video bitstream to emphasize that it is a video sample stream with a high data volume. The video sample stream (302) may be processed by an encoder (303) coupled to the camera (301). The encoder (303) may include hardware, software, or a combination of both to implement or carry out aspects of the disclosed subject matter described in more detail below. The encoded video bitstream (304) is depicted as a thin line as compared to the video sample stream (302) to emphasize the lower data volume of the encoded video bitstream, which may be stored on a streaming server (305) for future use. One or more streaming clients (306, 308) may access the streaming server (305) to retrieve copies (307) and (309) of the encoded video bitstream (304). The client (306) may include a video decoder (310). The video decoder (310) decodes an incoming copy (307) of the encoded video bitstream and produces an output video sample stream (311) that may be presented on a display (312) or another presentation device (not shown). In some streaming systems, the video bitstreams (304, 307, 309) may be encoded according to certain video codec / compression standards. Examples of such standards include ITU-T Recommendation H.265. A video codec standard under development is informally referred to as Versatile Video Coding (VVC). The disclosed subject matter may be used in the context of VVC.

[0040] Figure 4 is a functional block diagram of a video decoder (310) according to an embodiment.

[0041] A receiver (410) may receive one or more encoded video sequences to be decoded by a video decoder (310); in the same or another embodiment, one encoded video sequence is received at a time, where the decoding of each encoded video sequence is independent of other encoded video sequences. The encoded video sequences may be received from a channel (412), which may be a hardware / software link to a storage device storing the encoded video data. The receiver (410) may receive the encoded video data as well as other data, e.g., encoded audio data and / or auxiliary data streams that may be forwarded to their respective using entities (not shown). The receiver (410) may separate the encoded video sequences from the other data. To prevent network jitter, a buffer memory (415) may be coupled between the receiver (410) and an entropy decoder / parser (420) (hereinafter referred to as "parser"). When the receiver (410) receives data from a store-and-forward device with sufficient bandwidth and controllability or from an isochronous network, it may also be unnecessary to configure the buffer memory (415), or the buffer memory may be made smaller. Of course, for use on a service packet network such as the Internet, a buffer memory (415) may be required, which may be relatively large and may have an adaptive size.

[0042] The video decoder (310) may include a parser (420) to reconstruct symbols (421) from the entropy-coded video sequences. The classes of these symbols include information for managing the operation of the video decoder (310), as well as potential information for controlling a display device such as a display (312), which is not part of the decoder but may be coupled to the decoder, as Figure 4As shown. The control information for the display device can be a parameter set segment (not labeled) of Supplementary Enhancement Information (SEI message) or Video Usability Information (VUI). The parser (420) can perform parsing / entropy decoding on the received encoded video sequence. The encoding of the encoded video sequence can be carried out according to video coding techniques or standards and can follow principles well-known to those skilled in the art, including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, and so on. The parser (420) can extract subgroup parameter sets for at least one subgroup of pixels in the video decoder from the encoded video sequence based on at least one parameter corresponding to the group. The subgroup can include Group of Pictures (GOP), picture, tile, slice, macroblock, Coding Unit (CU), block, Transform Unit (TU), Prediction Unit (PU), and so on. The entropy decoder / parser can also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter (QP) values, motion vectors, and so on.

[0043] The parser (420) can perform entropy decoding / parsing operations on the video sequence received from the buffer memory (415) to create symbols (421). The parser (420) can receive the encoded data and selectively decode specific symbols (421). In addition, the parser (420) can determine whether to provide the specific symbols (421) to the motion compensation prediction unit (453), the scaler / inverse transform unit (451), the intra prediction unit (452), or the loop filter unit (454).

[0044] Depending on the type of the encoded video picture or a part of the encoded video picture (e.g., inter-picture and intra-picture, inter-block and intra-block) and other factors, the reconstruction of the symbols (421) can involve multiple different units. Which units are involved and the way they are involved can be controlled by the subgroup control information parsed by the parser (420) from the encoded video sequence. For the sake of brevity, such subgroup control information flows between the parser (420) and multiple units below are not described.

[0045] In addition to the functional blocks already mentioned, the video decoder (310) can conceptually be subdivided into several functional units as described below. In practical embodiments operating under commercial constraints, many of these units interact closely with each other and can be integrated with each other. However, for the purpose of describing the disclosed subject matter, it is appropriate to conceptually subdivide into the functional units below.

[0046] The first unit can be a scaler / inverse transform unit (451). The scaler / inverse transform unit (451) can receive quantized transform coefficients as symbols (421) and control information from the parser (420), including which transform mode to use, block size, quantization factor, quantization scaling matrix, etc. The scaler / inverse transform unit (451) can output a block including sample values, and the sample values can be input into the aggregator (455).

[0047] In some cases, the output samples of the scaler / inverse transform unit (451) can belong to intra-coded blocks; that is: blocks that do not use predictive information from previously reconstructed pictures, but can use predictive information from previously reconstructed parts of the current picture. Such predictive information can be provided by the intra prediction unit (452). In some cases, the intra prediction unit (452) generates surrounding blocks of the same size and shape as the block being reconstructed using the reconstructed information extracted from the (partially reconstructed) current picture (456). In some cases, the aggregator (455) adds the prediction information generated by the intra prediction unit (452) to the output sample information provided by the scaler / inverse transform unit (451) based on each sample.

[0048] In other cases, the output samples of the scaler / inverse transform unit (451) can belong to inter-coded and potentially motion-compensated blocks. In this case, the motion compensation prediction unit (453) can access the reference picture memory (457) to extract samples for prediction. After motion-compensating the extracted samples according to the symbol (421), these samples can be added by the aggregator (455) to the output of the scaler / inverse transform unit (451) (referred to as residual samples or residual signals in this case), thereby generating output sample information. The motion compensation prediction unit (453) obtaining prediction samples from addresses within the reference picture memory (457) can be controlled by a motion vector, and the motion vector is in the form of the symbol (421) for use by the motion compensation prediction unit (453), and the symbol (421) includes, for example, X, Y, and reference picture components. Motion compensation can also include interpolation of sample values extracted from the reference picture memory (457), a motion vector prediction mechanism, etc. when using sub-sample accurate motion vectors.

[0049] The output samples of the aggregator (455) can be adopted by various loop filtering techniques in the loop filter unit (454). Video compression techniques can include in-loop filter techniques that are controlled by parameters included in the encoded video bitstream, and the parameters can be used in the loop filter unit (454) as symbols (421) from the parser (420). However, in other embodiments, video compression techniques can also respond to meta-information obtained during decoding of a previous (in decoding order) portion of an encoded picture or an encoded video sequence, and to previously reconstructed and loop-filtered sample values.

[0050] The output of the loop filter unit (454) can be a sample stream that can be output to the display device (312) and stored in the reference picture memory (457) for subsequent inter-picture prediction.

[0051] Once fully reconstructed, some encoded pictures can be used as reference pictures for future prediction. Once an encoded picture is fully reconstructed and the encoded picture is identified as a reference picture (e.g., by the parser (420)), the current picture (456) can become part of the reference picture memory (457), and a new current picture memory can be reallocated before starting to reconstruct subsequent encoded pictures.

[0052] The video decoder (310) can perform decoding operations according to a predetermined video compression technique recorded, for example, in the ITU-T H.265 standard. An encoded video sequence can conform to the syntax specified by the video compression technique or standard used in the sense that the encoded video sequence follows the syntax of the video compression technique or standard, especially the profile, specified in the video compression technique literature or standard. For compliance, it is also required that the complexity of the encoded video sequence is within the range defined by the level of the video compression technique or standard. In some cases, the level limits the maximum picture size, maximum frame rate, maximum reconstruction sampling rate (measured, for example, in megasamples per second), maximum reference picture size, etc. In some cases, the limits set by the level can be further defined by the Hypothetical Reference Decoder (HRD) specification and the metadata of the HRD buffer management signaled in the encoded video sequence.

[0053] In an embodiment, the receiver (410) may receive additional (redundant) data along with the encoded video. The additional data may be part of the encoded video sequence. The additional data may be used by the video decoder (310) to decode the data appropriately and / or reconstruct the original video data more accurately. The additional data may be in the form of, for example, temporal, spatial, or signal noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, and the like.

[0054] Figure 5 is a functional block diagram of a video encoder (303) according to an embodiment.

[0055] The video encoder (303) may receive video samples from a video source (301) (not part of the decoder), which may capture video images to be encoded by the video encoder (303).

[0056] The video source (301) may provide a source video sequence in the form of a digital video sample stream to be encoded by the video encoder (303), and the digital video sample stream may have any suitable bit depth (e.g., 8 bits, 10 bits, 12 bits...), any color space (e.g., BT.601 Y CrCb, RGB...), and any suitable sampling structure (e.g., Y CrCb 4:2:0, Y CrCb 4:4:4). In a media service system, the video source (301) may be a storage device storing previously prepared videos. In a video conferencing system, the video source (301) may be a camera that captures local image information as a video sequence. The video data may be provided as a plurality of individual pictures that are given motion when viewed in sequence. The pictures themselves may be constructed as spatial pixel arrays, where each pixel may include one or more samples depending on the sampling structure, color space, etc. used. Those skilled in the art can easily understand the relationship between pixels and samples. The following focuses on describing samples.

[0057] According to an embodiment, a video encoder (303) may encode and compress pictures of a source video sequence into an encoded video sequence (543) in real time or under any other time constraints required by an application. Enforcing an appropriate encoding speed is a function of a controller (550). The controller (550) controls other functional units as described below and is functionally coupled to these units. For the sake of simplicity, the couplings are not labeled in the figures. Parameters set by the controller (550) may include rate control related parameters (e.g., picture skipping, quantizer, λ value of rate-distortion optimization techniques, etc.), picture size, group of pictures (GOP) layout, maximum motion vector search range, etc. Those skilled in the art can easily identify other functions of the controller (550) as these functions relate to the video encoder (303) optimized for a certain system design.

[0058] Some video encoders operate in an "encoding loop" that is readily recognizable to those skilled in the art. As a simple description, the encoding loop may include an encoding portion of an encoder (530) (hereinafter referred to as a "source coder" which is responsible for creating symbols based on an input picture to be encoded and reference pictures), and a (local) decoder (533) embedded in the video encoder (303). The "local" decoder (533) reconstructs the symbols in a manner similar to how a (remote) decoder creates sample data to create sample data (since in the video compression techniques contemplated in this application, any compression between the symbols and the encoded video bitstream is lossless). The reconstructed sample stream is input into a reference picture memory (534). Since the decoding of the symbol stream produces a bit-exact result independent of the decoder location (local or remote), the content in the reference picture memory is also bit-exact corresponding between the local encoder and the remote encoder. In other words, the reference picture samples "seen" by the prediction portion of the encoder are exactly the same as the sample values that the decoder will "see" when using the prediction during decoding. This basic principle of reference picture synchronization (and the drift that occurs in cases where synchronization cannot be maintained, e.g., due to channel errors) is well known to those skilled in the art.

[0059] The operation of the "local" decoder (533) may be the same as that of the "remote" decoder that has been described in detail above in connection with Figure 4 However, briefly referring additionally to Figure 5 When symbols are available and the entropy encoder (545) and the parser (420) can encode / decode the symbols losslessly into the encoded video sequence, the entropy decoding portion of the video decoder (310) (including the channel (412), the receiver (410), the buffer memory (415), and the parser (420)) may not be fully implementable in the local decoder (533).

[0060] At this point, it can be observed that any decoder technology other than parsing / entropy decoding that exists in the decoder must also exist in the corresponding encoder in substantially the same functional form. The description of the encoder technology can be abbreviated because it is inverse to the decoder technology described in full. A more detailed description is only needed in certain areas and is provided below.

[0061] As part of the operation, the source encoder (530) may perform motion-compensated predictive coding. Referring to one or more previously encoded frames designated as "reference frames" in the video sequence, the motion-compensated predictive coding performs predictive coding on the input frame. In this way, the coding engine (532) encodes the difference between the pixel blocks of the input frame and the pixel blocks of the reference frame, and the reference frame can be selected as the prediction reference for the input frame.

[0062] The local video decoder (533) may decode the encoded video data of the frame that can be designated as a reference frame based on the symbols created by the source encoder (530). The operation of the coding engine (532) may be a lossy process. When the encoded video data can be decoded at the video decoder ( Figure 5 not shown), the reconstructed video sequence can generally be a copy of the source video sequence with some errors. The local video decoder (533) replicates the decoding process that can be performed by the video decoder on the reference frame and can store the reconstructed reference frame in the reference picture memory (534). In this way, the encoder (303) can locally store a copy of the reconstructed reference frame, which has the same content (in the absence of transmission errors) as the reconstructed reference frame that will be obtained by the remote video decoder.

[0063] The predictor (535) may perform a prediction search for the coding engine (532). That is, for a new frame to be encoded, the predictor (535) may search in the reference picture memory (534) for sample data (as candidate reference pixel blocks) or some metadata, such as reference picture motion vectors, block shapes, etc., that can be used as an appropriate prediction reference for the new picture. The predictor (535) may operate on a per-pixel block basis of the sample blocks to find a suitable prediction reference. In some cases, according to the search results obtained by the predictor (535), it can be determined that the input picture may have a prediction reference obtained from multiple reference pictures stored in the reference picture memory (534).

[0064] The controller (550) may manage the encoding operations of the source encoder (530), including, for example, setting parameters and subgroup parameters for encoding video data.

[0065] The outputs of all the above functional units can be entropy encoded in an entropy encoder (545). The entropy encoder (545) can perform lossless compression on the symbols generated by various functional units according to techniques well known to those skilled in the art, such as Huffman coding, variable length coding, arithmetic coding, etc., so as to convert the symbols into an encoded video sequence.

[0066] The transmitter (540) can buffer the encoded video sequence created by the entropy encoder (545) to prepare for transmission over a communication channel (560), which can be a hardware / software link to a storage device that will store the encoded video data. The transmitter (540) can combine the encoded video data from the source encoder (530) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (the sources of which are not shown).

[0067] The controller (550) can manage the operation of the video encoder (303). During encoding, the controller (550) can assign a certain encoded picture type to each encoded picture, but this may affect the encoding techniques applicable to the corresponding picture. For example, pictures can generally be assigned to any of the following picture types.

[0068] An intra picture (I picture), which can be a picture that can be encoded and decoded without using any other frames in the sequence as a prediction source. Some video codecs allow different types of intra pictures, including, for example, Independent Decoder Refresh (IDR) pictures. Those skilled in the art are aware of the variants of I pictures and their corresponding applications and characteristics.

[0069] A predictive picture (P picture), which can be a picture that can be encoded and decoded using intra prediction or inter prediction, and the intra prediction or inter prediction uses at most one motion vector and a reference index to predict the sample values of each block.

[0070] A bi - predictive picture (B picture), which can be a picture that can be encoded and decoded using intra prediction or inter prediction, and the intra prediction or inter prediction uses at most two motion vectors and reference indexes to predict the sample values of each block. Similarly, multiple predictive pictures can use more than two reference pictures and associated metadata for reconstructing a single block.

[0071] Source pictures can typically be spatially subdivided into multiple sample blocks (e.g., blocks of 4×4, 8×8, 4×8, or 16×16 samples), and encoded block by block. These blocks can be predictively encoded with reference to other (already encoded) blocks, and the other blocks are determined according to the coding assignment of the corresponding picture applied to the block. For example, blocks of an I picture can be non-predictively encoded, or the blocks can be predictively encoded with reference to already encoded blocks of the same picture (spatial prediction or intra prediction). Pixel blocks of a P picture can be predictively encoded with reference to a previously encoded reference picture by spatial prediction or by temporal prediction. Blocks of a B picture can be predictively encoded with reference to one or two previously encoded reference pictures by spatial prediction or by temporal prediction.

[0072] The video encoder (303) can perform encoding operations according to a predetermined video coding technique or standard such as the ITU-T H.265 recommendation. In operation, the video encoder (303) can perform various compression operations, including predictive coding operations that utilize the temporal and spatial redundancy in the input video sequence. Thus, the encoded video data can conform to the syntax specified by the video coding technique or standard used.

[0073] In an embodiment, the transmitter (540) can transmit additional data and the encoded video. The source encoder (530) can include such data as part of, for example, an encoded video sequence. The additional data can include other forms of redundant data such as temporal / spatial / SNR enhancement layers, redundant pictures, and slices, SEI messages, VUI parameter set fragments, etc.

[0074] In one embodiment, the PH can contain the parameters (or syntax elements) for all slices applied to the encoded picture as NAL units. For example, the PH NAL unit can contain information about the picture type, picture order count, and coding / decoding mode, and their values are the same for all slices of the encoded picture.

[0075] Figure 6 is a schematic diagram of a NAL unit header (601) according to an embodiment.

[0076] Reference Figure 6 , the NAL unit can include a body (602) and a corresponding descriptor (603), and the body includes information about the picture type, picture sequence number, and coding / decoding mode, and their values are the same for all slices of the encoded picture.

[0077] In the same or another embodiment, a PU is a group of NAL units that are associated with each other according to specified classification rules, are consecutive in the decoding order, and exactly contain one encoded picture. A PU can consist of one PH NAL unit, one encoded picture including one or more VCL NAL units, and zero or more non-VCL NAL units.

[0078] In the same or another embodiment, an AU may consist of zero or one AU delimiter NAL unit and one or more PUs in increasing order of nuh_layer_id. The first AU in the bitstream may start with the first NAL unit of the bitstream. There may be at most one AU delimiter NAL unit in an AU.

[0079] In the same embodiment, the first VCL NAL unit of a picture may be the first VCL NAL unit after the PHNAL unit in the decoding order of the picture. When the VCL NAL unit is the first VCL NAL unit of a picture and one or more conditions are true, the VCL NAL unit may be the first VCL NAL unit of an AU (and thus, the picture containing the first VCL NAL unit is the first picture of the AU). The one or more conditions include:

[0080] - The value of nuh_layer_id of the VCL NAL unit is less than that of the previous picture in decoding order.

[0081] - The value of slice_pic_order_cnt_lsb of the VCL NAL unit is different from the PicOrderCntVal of the previous picture in decoding order.

[0082] - The PicOrderCntVal derived for the VCL NAL unit is different from the PicOrderCntVal of the previous picture in decoding order.

[0083] Figure 8A is a schematic diagram of the NAL unit structure in which each PU in an AU has one PH VCL unit according to an embodiment.

[0084] Refer to Figure 8A , in the same or another embodiment, an AU may consist of one or more non-VCL NAL units and VCL NAL units in the following order:

[0085] 1. Access unit delimiter (AUD) NAL unit (805) (when present),

[0086] 2. DPS NAL unit (810) (when present),

[0087] 3. VPS NAL unit (815) (when present),

[0088] 4. SPS NAL unit (820) (when present),

[0089] 5. PPS NAL unit (825) (when present),

[0090] 6. Prefix APS NAL unit (830) (when present),

[0091] 7. PH NAL unit (835),

[0092] 8. Prefix SEI NAL unit (840) (when present),

[0093] 9. NAL unit with nal_unit_type equal to RSV_NVCL_26 (when present),

[0094] 10. NAL unit with nal_unit_type in the range of UNSPEC28 to UNSPEC29 (when present),

[0095] 11. One or more VCL NAL units (845),

[0096] 12. Suffix APS NAL unit (850) (when present),

[0097] 13. Suffix SEI NAL unit (855) (when present),

[0098] 14. End-of-Sequence (EOS) NAL unit (860) (when present)

[0099] 15. End-of-Byte-Stream (EOB) NAL unit (865) (when present)

[0100] When the bitstream consists of multiple layers, the NAL units from the DPS NAL unit to the EOS NAL unit (except VPS) in the above order will be repeated as many times as the number of layers (if present).

[0101] In the same embodiment, the order of the coded pictures and non-VCL NAL units within a PU or AU can follow the following constraints:

[0102] – When an AU delimiter NAL unit is present in an AU, it can be the first NAL unit of that AU.

[0103] – The PH NAL unit in a PU can be before the first VCL NAL of that PU.

[0104] – When any DPS NAL unit, VPS NAL unit, SPS NAL unit, PPS NAL unit, prefix APS NAL unit, or prefix SEI NAL unit is present in a PU, they cannot be after the last VCL NAL unit of that PU.

[0105] – When there are any DPS NAL units, VPS NAL units, SPS NAL units, or PPS NAL units in a PU, they can be before the PH NAL unit of that PU.

[0106] – NAL units with nal_unit_type equal to SUFFIX_APS_NUT, SUFFIX_SEI_NUT, or FD_NUT in a PU may not be before the first VCL NAL unit of that PU.

[0107] – When there is an EOS NAL unit in a PU, it can be the last NAL unit among all NAL units in that PU except the EOB NAL unit.

[0108] – When there is an EOB NAL unit in an AU, it can be the last NAL unit in that AU.

[0109] Figure 7 Schematic diagram of a picture header with independent mode and dependent mode according to an embodiment.

[0110] In one embodiment, PH is divided into two types: independent PH and dependent PH. The PH of the independent layer can be an independent PH. The PH of the dependent layer can be an independent PH or a dependent PH. The dependent PH of the dependent layer k (where k > 0) can refer to the PH of layer l, which is the direct reference layer of the dependent layer k, where l < k.

[0111] In an embodiment, the encoded enhanced layer picture can be reconstructed in the decoder. Specifically, it includes: decoding at least one independent PH of the encoded picture of the reference layer l, and the encoded picture of the reference layer l can be referred to by the encoded enhanced layer picture; decoding the dependent PH of the encoded enhanced layer picture, and performing PH prediction. This PH prediction includes the processing process described in the following embodiments.

[0112] In the same embodiment, when the dependent PH of the dependent layer k refers to the PH of the direct reference layer l of the dependent layer k, one or more syntax elements that are usually present in the independent PH may not be signaled in the dependent PH of the dependent layer k. The value of the syntax element skipped in the dependent PH of the dependent layer k is inferred to be equal to the value of the same syntax element in the PH of the direct reference layer l. In an example, according to at least one value of at least one corresponding syntax element of at least one independent PH, at least one value of at least one syntax element of the dependent PH can be predicted, and at least one value of the non-existent syntax element of the dependent PH is inferred to be equal to at least one value of the existent syntax element of the at least one independent PH.

[0113] In the same or another embodiment, when the dependent PH of dependent layer k references the PH of the direct reference layer l of dependent layer k, one or more syntax elements that are typically present in an independent PH may be signaled in the dependent PH of dependent layer k. The value of the syntax element in the dependent PH of dependent layer k is predicted from the value of the same syntax element in the PH of the direct reference layer l.

[0114] In the same embodiment, when predicting the value of a syntax element in a dependent PH from the value of the corresponding syntax element in a reference PH, the delta value between the predicted syntax element in the dependent PH and the reference syntax element in the reference PH may be signaled in the dependent PH. In one example, the delta value between at least one value of at least one syntax element of a predicted dependent picture header and at least one corresponding syntax element of at least one independent picture header may be signaled.

[0115] In the same embodiment, when dependent layer k has multiple direct reference layers, the PH of the highest reference layer immediately preceding dependent layer k in decoding order may be the reference PH.

[0116] In the same embodiment, the reference PH may be an independent PH in an independent layer.

[0117] In the same embodiment, the reference PH may be an independent PH or a dependent PH in a dependent layer.

[0118] In the same embodiment, a dependent PH and its reference PH may belong to the same access unit.

[0119] In the same embodiment, a dependent PH and its reference PH may belong to different layers that have a direct dependency.

[0120] Reference Figure 7 , in the same embodiment, the flag ph_dependent_flag in the PH may indicate whether the current PH is dependent or independent. When the value of ph_dependent_flag is equal to 1, the following syntax elements may not be present in the current dependent PH. The value of the skipped syntax element may be inferred from the value of the same syntax element signaled in the reference PH. When the value of ph_dependent_flag is equal to 0, all of the following syntax elements may be present in the current independent PH.

[0121] In the same embodiment, the value ph_pic_parameter_set_id of the PPS ID in the dependent PH may be different from the value of the PPS ID in the reference PH. When the values of the PPS ID are different, the values of the parameters derived from the syntax elements in the PH may be different because of the different reference PPSs.

[0122] In the same or another embodiment, when a dependent layer refers to multiple reference layers, the index in the dependent PH of the dependent layer may indicate the PH of which reference layer the current dependent PH refers to.

[0123] In the same or another embodiment, the NAL unit type ( Figure 6 The nal_unit_type in the field may indicate that the corresponding NAL unit contains a dependent PH.

[0124] In the same or another embodiment, the NAL unit type may indicate that the corresponding NAL unit contains an independent PH.

[0125] In one embodiment, the VCL NAL units following the PH NAL unit in the decoding order of the picture reference the previous PH NAL unit. When there is only one PH NAL unit in a PU, any VCL NAL unit references the PH NAL unit preceding all VCL NAL units. When there are multiple PH NAL units in a PU, any VCL NAL unit references the PH NAL immediately preceding the current VCL NAL unit.

[0126] Figure 8B is a diagram of a NAL unit structure having one PH VCL unit and one or more subordinate PH VCL units in a PU of an AU according to an embodiment.

[0127] refer to Figure 8B In the same or another embodiment, an access unit may consist of one or more non-VCL NAL units and VCL NAL units, in the following order:

[0128] 1. AUD NAL unit (805) (when present),

[0129] 2. DPS NAL unit (810) (when present),

[0130] 3. VPS NAL unit (815) (when present),

[0131] 4. SPS NAL unit (820) (when present),

[0132] 5. PPS NAL unit (825) (when present),

[0133] 6. Prefix APS NAL unit (830) (when present),

[0134] 7. Prefix SEI NAL unit (840) (when present),

[0135] 8. The NAL unit with nal_unit_type equal to RSV_NVCL_26 (when present),

[0136] 9. The NAL unit with nal_unit_type in the range of UNSPEC28 to UNSPEC29 (when present),

[0137] 10. PH NAL unit (835),

[0138] - Option A: Independent PH NAL unit (835a)

[0139] - Option B: Dependent PH NAL unit (835b)

[0140] 11. One or more VCL NAL units (845),

[0141] 12. Suffix APS NAL unit (850) (when present),

[0142] 13. Suffix SEI NAL unit (855) (when present),

[0143] 14. EOS NAL unit (860) (when present)

[0144] 15. EOB NAL unit (865) (when present)

[0145] When the bitstream can consist of multiple layers, the NAL units from the DPS NAL unit to the EOS NAL unit (except for the VPS) in the above order will be repeated as many times as the number of layers (if present). For each layer, there can be one PH VCL NAL unit for each PU. Depending on the value of the NAL unit type or the flag in the PH, the PH VCL NAL unit contains an independent PH NAL unit or a dependent PH NAL unit. In one example, based on the flag of at least one independent picture header, it can be determined that the encoded picture includes the at least one independent picture header; based on the flag of the dependent picture header, it can be determined that the encoded enhancement layer picture includes the dependent picture header. In another example, based on the unit type of the encoded picture, it can be determined that the encoded picture includes at least one independent picture header; based on the unit type of the encoded enhancement layer picture, it can be determined that the encoded enhancement layer picture includes the dependent picture header.

[0146] Figure 8C is a schematic diagram of the NAL unit structure in which each PU in the AU has one or more PH VCL units according to an embodiment.

[0147] Reference Figure 8C, for each layer, the grouped PH NAL units (835) and VCL NAL units (845) can be repeated.

[0148] In the same or another embodiment, one or more NAL units containing the same PH can appear continuously before one or more VCL NAL units.

[0149] In the same embodiment, the order of the coded pictures and non-VCL NAL units within a PU or AU can follow the following constraints:

[0150] – When an AU delimiter NAL unit exists in an AU, it can be the first NAL unit of that AU.

[0151] – The PH NAL unit in a PU can be before at least one VCL NAL of that PU.

[0152] – When any DPS NAL unit, VPS NAL unit, SPS NAL unit, PPS NAL unit, prefix APS NAL unit, or prefix SEI NAL unit exists in a PU, they cannot be after the last VCL NAL unit of that PU.

[0153] – When any DPS NAL unit, VPS NAL unit, SPS NAL unit, or PPS NAL unit exists in a PU, they can be before the PH NAL unit of that PU.

[0154] – A NAL unit with nal_unit_type equal to SUFFIX_APS_NUT, SUFFIX_SEI_NUT, or FD_NUT in a PU cannot be before the first VCL NAL unit of that PU.

[0155] – When an EOS NAL unit exists in a PU, it can be the last NAL unit among all NAL units in that PU except the EOB NAL unit.

[0156] – When an EOB NAL unit exists in an AU, it can be the last NAL unit in that AU.

[0157] In the same or another embodiment, the PPS identifier (PPS_ID) specifying the reference PPS can be signaled in the PH. When a PPS_ID exists in the PH, any slice or any VCL NAL unit in the same PU or AU after the PH can refer to the PPS for which the PPS_ID is signaled in the PH.

[0158] In the same embodiment, when signaling the PPS_ID in the corresponding PH, the PPS_ID may not be present in the slice header, and the PH is before the slices in the same PU or AU. In this case, all slices in the same PU may refer to the same PPS without changing the values of the syntax elements in the PPS.

[0159] Figure 8D is a flowchart of a method (870) for signaling a dependent picture header and an independent picture header according to an embodiment. In some embodiments, Figure 8D one or more processing blocks of may be performed by a decoder (310). In some embodiments, Figure 8D one or more processing blocks of may be performed by another device or a set of devices separate from or including the decoder (310), such as an encoder (303).

[0160] participate Figure 8D , in a first step (871), the method (870) includes decoding at least one independent picture header of an encoded picture of a reference layer, the encoded picture of the reference layer being referenced by the encoded enhancement layer picture.

[0161] In a second step (872), the method (870) includes decoding a dependent picture header of the encoded enhancement layer picture.

[0162] In a third step (873), the method (870) includes performing picture header prediction, which includes one or both of: predicting at least one value of at least one syntax element of the dependent picture header according to at least one value of at least one corresponding syntax element of the at least one independent picture header; and inferring that at least one value of a non-existent syntax element of the dependent picture header is equal to at least one value of an existent syntax element of the at least one independent picture header.

[0163] The method (870) may further include signaling a delta value between at least one value of at least one syntax element of the predicted dependent picture header and at least one value of at least one corresponding syntax element of the at least one independent picture header.

[0164] The method (870) may further include determining that the encoded picture includes the at least one independent picture header based on a flag of the at least one independent picture header; and determining that the encoded enhancement layer picture includes the dependent picture header based on a flag of the dependent picture header.

[0165] The method (870) may further include determining that the encoded picture includes the at least one independent picture header based on the unit type of the encoded picture; and determining that the encoded enhancement layer picture includes the dependent picture header based on the unit type of the encoded enhancement layer picture.

[0166] The at least one independent picture header is directly before the video coding / decoding layer in the encoded picture.

[0167] The dependent picture header is directly before the video coding / decoding layer in the encoded enhancement layer picture.

[0168] Each of the encoded picture and the encoded enhancement layer picture includes a plurality of video coding / decoding layers and a plurality of picture headers directly before the plurality of video coding / decoding layers.

[0169] Although Figure 8D illustrates example steps of the method (870), in some embodiments, the method (870) may include more steps, fewer steps, different steps, or steps in a different arrangement than Figure 8D shown. Additionally or alternatively, two or more steps of the method (870) may be executed in parallel.

[0170] Figure 8E is a block diagram of a device (880) for signaling a dependent picture header and an independent picture header according to an embodiment.

[0171] Referring to Figure 8E , the device (880) includes a first decoding code (881), a second decoding code (882), and an execution code (883).

[0172] The first decoding code (881) is for causing the at least one processor to decode at least one independent picture header of an encoded picture of a reference layer, the encoded picture of the reference layer being referenced by the encoded enhancement layer picture.

[0173] The second decoding code (882) is for causing the at least one processor to decode the dependent picture header of the encoded enhancement layer picture.

[0174] The execution code (883) is for causing the at least one processor to perform picture header prediction, which includes one or both of the following: predicting at least one value of at least one syntax element of the dependent picture header according to at least one value of at least one corresponding syntax element of the at least one independent picture header; and inferring that at least one value of a non-existent syntax element of the dependent picture header is equal to at least one value of an existent syntax element of the at least one independent picture header.

[0175] The apparatus (880) may further include signaling code for causing the at least one processor to signal a delta value between at least one value of at least one syntax element of the predicted dependent picture header and at least one corresponding value of at least one syntax element of the at least one independent picture header.

[0176] The apparatus (880) may further include first determination code for causing the at least one processor to determine that the coded picture includes the at least one independent picture header based on a flag of the at least one independent picture header; and to determine that the coded enhancement layer picture includes the dependent picture header based on a flag of the dependent picture header.

[0177] The apparatus (880) may further include second determination code for causing the at least one processor to determine that the coded picture includes the at least one independent picture header based on a unit type of the coded picture; and to determine that the coded enhancement layer picture includes the dependent picture header based on a unit type of the coded enhancement layer picture.

[0178] The at least one independent picture header is directly before a video coding / decoding layer in the coded picture.

[0179] The dependent picture header is directly before a video coding / decoding layer in the coded enhancement layer picture.

[0180] Each of the coded picture and the coded enhancement layer picture includes a plurality of video coding / decoding layers and a plurality of picture headers directly before the plurality of video coding / decoding layers.

[0181] The above techniques may be implemented as computer software by computer-readable instructions and physically stored in one or more computer-readable media. Figure 9 A schematic diagram of a computer system (900) according to an embodiment is shown.

[0182] The computer software may be encoded in any suitable machine code or computer language, creating code including instructions through mechanisms such as assembly, compilation, and linking, and the instructions may be directly executed by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc., or executed through decoding, microcode, etc.

[0183] The instructions may be executed on various types of computers or their components, including for example personal computers, tablets, servers, smartphones, gaming devices, Internet of Things devices, etc.

[0184] Figure 9The components shown for the computer system (900) are exemplary in nature and are not intended to impose any limitation on the scope of use or functionality of the computer software implementing the embodiments of the present application. Nor should the configuration of the components be construed as having any dependency or requirement on any one component or combination thereof shown in the embodiments of the computer system (900).

[0185] The computer system (900) may include certain human - machine interface input devices. Such human - machine interface input devices can respond to the input of one or more human users through tactile input (such as keyboard input, swiping, data glove movement), audio input (such as voice, applause), visual input (such as gestures), and olfactory input (not shown). The human - machine interface device can also be used to capture certain media, which need not be directly related to conscious human input, such as audio (e.g., speech, music, ambient sound), images (e.g., scanned images, photographic images obtained from a still - image camera), and video (e.g., two - dimensional video, three - dimensional video including stereoscopic video).

[0186] The human - machine interface input device may include one or more of the following (only one is shown): keyboard (901), mouse (902), touchpad (903), touchscreen (910), data glove, joystick (905), microphone (906), scanner (907), camera (908).

[0187] The computer system (900) may also include certain human - machine interface output devices. Such human - machine interface output devices can stimulate the senses of one or more human users through, for example, tactile output, sound, light, and smell / taste. Such human - machine interface output devices may include tactile output devices (e.g., tactile feedback through the touchscreen (910), data glove, or joystick (905), but there may also be tactile feedback devices that do not serve as input devices), audio output devices (e.g., speakers (909), headphones (not shown)), visual output devices (e.g., screens (910) including cathode - ray tube (CRT) screens, liquid - crystal (LCD) screens, plasma screens, organic light - emitting diode (OLED) screens, each of which may or may not have touchscreen input functionality, each of which may or may not have tactile feedback functionality - some of which can output two - dimensional visual output or output above three - dimensions through means such as stereoscopic picture output; virtual reality glasses (not shown), holographic displays, and smoke boxes (not shown)), and printers (not shown). The graphics adapter (950) generates images and outputs the images to the touchscreen (910).

[0188] The computer system (900) may also include human-accessible storage devices and their associated media, such as optical media including high-density read-only / rewritable optical discs with CD / DVD (CD / DVD ROM / RW) (920) or similar media (921), thumb drives (922), removable hard disk drives or solid state drives (923), conventional magnetic media such as tapes and floppy disks (not shown), ROM / ASIC / PLD-based dedicated devices such as security software protectors (not shown), and so on.

[0189] Those skilled in the art should also understand that the term "computer-readable medium" used in connection with the disclosed subject matter does not include transmission media, carrier waves, or other transient signals.

[0190] The computer system (900) may also include an interface to one or more communication networks (955). For example, the network (955) may be wireless, wired, or optical. The network (955) may also be a local area network, wide area network, metropolitan area network, vehicular network, and industrial network, real-time network, delay-tolerant network, and so on. The network (955) also includes local area networks such as Ethernet, wireless local area networks, cellular networks (Global System for Mobile Communications (GSM), Third Generation (3G), Fourth Generation (4G), Fifth Generation (5G), Long Term Evolution (LTE), etc.), television cable or wireless wide area digital networks (including cable television, satellite television, and terrestrial broadcast television), vehicular and industrial networks (including CANBus), and so on. Some networks (955) typically require an external network interface adapter for connection to certain common data ports or peripheral buses (949) (e.g., the Universal Serial Bus (USB) port of the computer system (900)); other systems are typically integrated into the core of the computer system (900) by connecting to the system bus as described below (e.g., an Ethernet interface is integrated into a PC computer system or a cellular network interface (954) is integrated into a smart phone computer system). By using any of these networks (955), the computer system (900) can communicate with other entities. The communication may be one-way, only for receiving (e.g., wireless television), one-way only for sending (e.g., CAN bus to certain CAN bus devices), or two-way, such as through a local or wide area digital network to other computer systems. Each of the above-mentioned networks (955) and network interfaces (954) may use certain protocols and protocol stacks.

[0191] The above-mentioned human-machine interface devices, human-accessible storage devices, and network interfaces (954) may be connected to the core (940) of the computer system (900).

[0192] The core (940) may include one or more central processing units (CPUs) (941), a graphics processing unit (GPU) (942), a dedicated programmable processing unit in the form of a field programmable gate array (FPGA) (943), a hardware accelerator for specific tasks (944), etc. These devices, as well as a read-only memory (ROM) (945), a random access memory (RAM) (946), an internal mass storage (such as an internal non-user accessible hard disk drive, a solid state drive (SSD), etc.) (947), etc. may be connected through a system bus (948). In some computer systems, the system bus (948) can be accessed in the form of one or more physical plugs so as to be expandable through additional central processing units, graphics processing units, etc. Peripheral devices can be directly attached to the system bus (948) of the core or connected through a peripheral bus (949). The architecture of the peripheral bus includes Peripheral Component Interconnect (PCI), Universal Serial Bus (USB), etc.

[0193] The CPU (941), GPU (942), FPGA (943), and accelerator (944) can execute certain instructions, which, when combined, can constitute the above-mentioned computer code. The computer code can be stored in the ROM (945) or RAM (946). Transitional data can also be stored in the RAM (946), while permanent data can be stored in, for example, the internal mass storage (947). Fast storage and retrieval of any memory device can be achieved by using a cache memory, which can be closely associated with one or more CPUs (941), GPUs (942), mass storage (947), ROM (945), RAM (946), etc.

[0194] The computer-readable medium may have computer code for performing various computer-implemented operations. The medium and the computer code can be specially designed and constructed for the purposes of the embodiments or can be those well-known and available to those skilled in the computer software art.

[0195] By way of example and not limitation, a computer system having an architecture (900), and in particular a core (940), can provide the functionality of a processor (including a CPU, GPU, FPGA, accelerator, etc.) to execute software contained in one or more tangible computer-readable media. Such computer-readable media can be media associated with the above-mentioned user-accessible mass storage, as well as specific memories having a non-volatile core (940), such as on-core mass storage (947) or ROM (945). The software implementing the various embodiments of the present application can be stored in such devices and executed by the core (940). Depending on specific needs, the computer-readable media can include one or more storage devices or chips. The software can cause the core (940), and in particular the processors therein (including the CPU, GPU, FPGA, etc.), to execute specific processes or specific portions of specific processes described herein, including defining data structures stored in RAM (946) and modifying such data structures according to software-defined processes. Additionally or alternatively, the computer system can provide functionality that is logically hardwired or otherwise included in circuitry (e.g., accelerator (944)) that can operate in place of or in conjunction with the software to execute specific processes or specific portions of specific processes described herein. In appropriate cases, references to software can include logic and vice versa. In appropriate cases, references to computer-readable media can include circuitry (such as an integrated circuit (IC)) that stores software for execution, circuitry that contains execution logic, or both. The present application encompasses any suitable combination of hardware and software.

[0196] Although the present application has described a number of exemplary embodiments, various changes, permutations, and various equivalent substitutions of the embodiments are within the scope of the present application. It should thus be understood that those skilled in the art can design a variety of systems and methods that, although not explicitly shown or described herein, embody the principles of the present application and are thus within the spirit and scope of the present application.

Claims

1. A method for reconstructing a coded enhanced layer picture, characterized in that, The method includes: decoding at least one independent picture header network abstraction layer (NAL) unit of an encoded picture of a reference layer, the encoded picture of the reference layer being referenced by the encoded enhancement layer picture; decoding a dependent picture header NAL unit of the encoded enhancement layer picture; and performing picture header prediction, the picture header prediction including both: predicting at least one value of at least one syntax element of the dependent picture header NAL unit according to at least one value of at least one corresponding syntax element of the at least one independent picture header NAL unit; and inferring at least one value of non-existent syntax elements of the dependent picture header NAL unit to be equal to at least one value of existent syntax elements of the at least one independent picture header NAL unit.

2. The method according to claim 1, characterized in that It further includes: signaling an incremental value between at least one value of at least one syntax element of the predicted dependent picture header NAL unit and at least one value of at least one corresponding syntax element of the at least one independent picture header NAL unit.

3. The method according to claim 1, wherein It further includes: determining that the encoded picture includes the at least one independent picture header NAL unit based on a flag of the at least one independent picture header NAL unit; and determining that the encoded enhancement layer picture includes the dependent picture header NAL unit based on a flag of the dependent picture header NAL unit.

4. The method according to claim 1, characterized in that, It further includes: determining that the encoded picture includes the at least one independent picture header NAL unit based on a unit type of the encoded picture; and determining that the encoded enhancement layer picture includes the dependent picture header NAL unit based on a unit type of the encoded enhancement layer picture.

5. The method according to any one of claims 1-4, characterized in that, The at least one independent picture header NAL unit is located before a video coding / decoding layer NAL unit in the encoded picture.

6. The method according to any one of claims 1 to 4, characterized in that, The dependent picture header NAL unit is located before a video coding / decoding layer NAL unit in the encoded enhancement layer picture.

7. The method according to any one of claims 1-4, characterized in that, Each of the encoded picture and the encoded enhancement layer picture includes a plurality of video coding / decoding layer NAL units and a plurality of picture header NAL units before the plurality of video coding / decoding layers.

8. A video encoding method, applied to an encoder including a local decoder, characterized in that, The method includes: decoding at least one independent picture header network abstraction layer (NAL) unit of an encoded picture of a reference layer, the encoded picture of the reference layer being referenced by an encoded enhancement layer picture; decoding a dependent picture header NAL unit of the encoded enhancement layer picture; and performing picture header prediction, the picture header prediction including both: predicting at least one value of at least one syntax element of the dependent picture header NAL unit according to at least one value of at least one corresponding syntax element of the at least one independent picture header NAL unit; and inferring at least one value of non-existent syntax elements of the dependent picture header NAL unit to be equal to at least one value of existent syntax elements of the at least one independent picture header NAL unit.

9. An apparatus for reconstructing a picture of an encoded enhancement layer, characterized in that, The device includes: at least one memory for storing program code; and at least one processor, configured to access the at least one memory and operate according to the program code to perform the method according to any one of claims 1 to 7.

10. An apparatus for reconstructing a coded enhanced layer picture, characterized in that, The apparatus comprises: a first decoding module, configured to decode at least one independent picture header network abstraction layer (NAL) unit of an encoded picture in a reference layer, the encoded picture in the reference layer being referenced by the encoded enhancement layer picture; a second decoding module, configured to decode a dependent picture header NAL unit of the encoded enhancement layer picture; and an execution module, configured to perform picture header prediction, comprising both: predicting at least one value of at least one syntax element of the dependent picture header NAL unit according to at least one value of at least one corresponding syntax element of the at least one independent picture header NAL unit; and inferring at least one value of at least one non-existing syntax element of the dependent picture header NAL unit to be equal to at least one value of at least one existing syntax element of the at least one independent picture header NAL unit.

11. A non-volatile computer-readable storage medium, characterized in that, for storing instructions which, when executed by one or more processors, cause the one or more processors to perform the method according to any one of claims 1 to 7.

12. A method for storing a video bitstream, characterized in that, performing the video encoding method according to claim 8 to generate the video bitstream; and storing the video bitstream.

13. A method for transmitting a video bitstream, characterized in that, performing the video encoding method according to claim 8 to generate the video bitstream; and transmitting the video bitstream.

14. A non-volatile computer-readable storage medium, characterized in that, for storing instructions and the video bitstream which, when executed by one or more processors, cause the one or more processors to perform the steps of the video encoding method according to claim 8 to generate the video bitstream.

Images