Method and device for encoding and decoding video bit stream

Abstract

The embodiment of the invention provides a video bit stream coding and decoding method and device, and the method comprises the steps: executing first confirmation, so as to determine whether a current layer of a current picture is an independent layer or not; performing a second acknowledgement to determine whether to enable reference picture resampling for the current layer; disabling wrap-around compensation for the current layer based on the first acknowledgement and the second acknowledgement; and encoding the current layer without the wrap-around compensation.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Patent Application No. 62 / 955,520, filed on December 31, 2019, and U.S. Patent Application No. 17 / 064,172, filed on October 6, 2020, the entire contents of which are incorporated herein. Technical Field

[0003] The disclosed subject matter relates to video encoding and decoding, and more particularly, to enabling and disabling surround motion compensation. Background Art

[0004] The use of inter-picture prediction with motion compensation for video encoding and decoding is well known. An uncompressed digital video may include a series of pictures, each picture having a spatial dimension of, for example, 1920×1080 luma samples and associated chroma samples. The series of pictures may have a fixed or variable picture rate (also informally referred to as a frame rate), for example, 60 pictures per second or 60 Hz. Uncompressed video has very high bitrate requirements. For example, 1080p60 4:2:0 video (1920x1080 luma sample resolution at 60 Hz frame rate) with 8 bits per sample requires close to 1.5 Gbit / s of bandwidth. One hour of such video would require over 600 GB of storage space.

[0005] One goal of video encoding and decoding is to reduce the redundancy of the input video signal through compression. Compression can help reduce the bandwidth or storage space requirements mentioned above, in some cases by two or more orders of magnitude. Both lossless and lossy compression, and combinations of the two, can be used. Lossless compression refers to the technique of reconstructing an exact copy of the original signal from the compressed original signal. When lossy compression is used, the reconstructed signal may not be exactly the same as the original signal, but the distortion between the original and the reconstructed signal is small enough that the reconstructed signal can be used for the intended application. Lossy compression is widely used in video. The amount of distortion allowed depends on the application. For example, users of some consumer streaming applications can tolerate higher distortion than users of television applications. The achievable compression ratio reflects that higher allowed / tolerable distortion produces higher compression ratios.

[0006] Video encoders and decoders can utilize several broad categories of techniques, including, for example, motion compensation, transforms, quantization, and entropy coding, some of which are described below.

[0007] Historically, video encoders and decoders have tended to operate on a given picture size, which in most cases is defined for a coded video sequence (CVS), a group of pictures (GOP) or a similar multi-picture time frame and remains unchanged. For example, in MPEG-2, system designs are known to change the horizontal resolution (and thus the picture size) according to factors such as the activity of the scene, but only on I pictures, so it is usually used for GOPs. For example, according to ITU-T Rec. H.263 Annex P, it is known to resample reference pictures with different resolutions within a CVS. However, here the picture size is not changed, only the reference pictures are resampled, resulting in the possibility that only part of the picture canvas is used (in the case of downsampling), or only part of the scene is captured (in the case of upsampling). In addition, H.263 Annex Q allows a single macroblock to be resampled up or down by a factor of 2 (in each dimension). Again, the picture size remains unchanged. The size of the macroblock is fixed in H.263, so no signaling is required.

[0008] In modern video codecs, changes in picture size in predicted pictures are becoming more and more mainstream. For example, VP9 allows reference pictures to be resampled and the resolution of the entire picture to be changed. Similarly, certain proposals have been made for VVC (including, for example, Hendry et al., "On adaptive resolution change (ARC) for VVC", Joint Video Group document JVET-M0135-v1, January 9-19, 2019, which is incorporated herein in its entirety) that allow the entire reference picture to be resampled to a different higher or lower resolution. In this document, it is proposed to encode different candidate resolutions in the sequence parameter set and referenced by per-picture syntax elements in the picture parameter set. Summary of the invention

[0009] In an embodiment, a method for generating an encoded video bitstream using at least one processor is provided, comprising: performing a first confirmation to determine whether a current layer of a current picture is an independent layer; performing a second confirmation to determine whether reference picture resampling is enabled for the current layer; disabling surround compensation for the current layer based on the first confirmation and the second confirmation; and encoding the current layer without performing the surround compensation.

[0010] In an embodiment, a device for generating a coded video bitstream is provided, comprising: at least one memory configured to store program code; and at least one processor configured to read the program code and operate according to the instructions of the program code, wherein the program code comprises: a first confirmation code configured to cause the at least one processor to perform a first confirmation to determine whether a current layer of a current picture is an independent layer; a second confirmation code configured to cause the at least one processor to perform a second confirmation to determine whether to enable reference picture resampling for the current layer; a disabling code configured to cause the at least one processor to disable surround compensation for the current layer based on the first confirmation and the second confirmation; and an encoding code configured to cause the at least one processor to encode the current layer without performing the surround compensation.

[0011] In an embodiment, a non-volatile computer-readable medium storing instructions is provided, comprising one or more instructions, which, when executed by one or more processors of a device for generating a coded video bitstream, causes the one or more processors to perform a first confirmation to determine whether a current layer of a current picture is an independent layer; perform a second confirmation to determine whether to enable reference picture resampling for the current layer; based on the first confirmation and the second confirmation, disable surround compensation for the current layer; and encode the current layer without performing the surround compensation. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Other features, properties and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings, in which

[0013] Figure 1 is a schematic diagram of a simplified block diagram of a communication system according to an embodiment.

[0014] Figure 2 is a schematic diagram of a simplified block diagram of a communication system according to an embodiment.

[0015] Figure 3 is a schematic diagram of a simplified block diagram of a decoder according to an embodiment.

[0016] Figure 4 is a schematic diagram of a simplified block diagram of an encoder according to an embodiment.

[0017] Figures 5A-5E is a diagram of options for signaling ARC parameters according to an embodiment.

[0018] Figure 6A-6B is a diagram of an example of a syntax table according to an embodiment.

[0019] Figure 7 is a diagram of an example of a syntax table according to an embodiment.

[0020] Figures 8A-8C is a flow chart of an example process for generating an encoded video bitstream according to an embodiment.

[0021] Fig. 9 is a schematic diagram of a computer system according to an embodiment. DETAILED DESCRIPTION

[0022] Figure 1 A simplified block diagram of a communication system (100) according to an embodiment of the present disclosure is shown. The communication system (100) may include at least two terminal devices (110, 120) interconnected via a network (150). For unidirectional data transmission, a first terminal device (110) may encode video data at a local location for transmission to another terminal device (120) via the network (150). A second terminal device (120) may receive encoded video data from another terminal device from the network (150), decode the encoded video data, and display the recovered video data. Unidirectional data transmission is more common in applications such as media services.

[0023] Figure 1 A second pair of terminal devices (130, 140) is shown that supports bidirectional transmission of encoded video, which can occur, for example, during a video conference. For bidirectional data transmission, each terminal device (130, 140) can encode video data collected at a local location for transmission to the other terminal device via a network (150). Each terminal device (130, 140) can also receive encoded video data transmitted by the other terminal device, and can decode the encoded video data and display the recovered video data on a local display device.

[0024] exist Figure 1 In the embodiment of the present disclosure, the terminal devices (110-140) may be servers, personal computers, and smart phones, but the principles of the present disclosure may not be limited thereto. Embodiments of the present disclosure are applicable to laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. The network (150) represents any number of networks that transmit encoded video data between the terminal devices (110-140), including, for example, wired and / or wireless communication networks. The communication network (150) may exchange data in circuit switching and / or packet switching channels. Representative networks include telecommunication networks, local area networks, wide area networks, and / or the Internet. For the purposes of this discussion, unless explained below, the architecture and topology of the network (150) may be irrelevant to the operation of the present disclosure.

[0025] As an application example of the disclosed subject matter, Figure 2The placement of the video decoder and encoder in a streaming environment is shown. The disclosed subject matter is equally applicable to other video-enabled applications including, for example, video conferencing, digital TV, storing compressed video on digital media including CDs, DVDs, memory sticks, etc., and the like.

[0026] The streaming system may include an acquisition subsystem (213), which may include a video source (201), such as a digital camera, which creates, for example, an uncompressed video sample stream (202). The sample stream (202) is depicted as a thick line compared to an encoded video bitstream to emphasize that it is a high data volume video sample stream, and the sample stream (202) may be processed by an encoder (203) coupled to the camera (201). The encoder (203) may include hardware, software, or a combination of hardware and software to implement or implement various aspects of the disclosed subject matter as described in more detail below. The encoded video bitstream (204) is depicted as a thin line compared to the sample stream (202) to emphasize the lower data volume of the encoded video bitstream, which may be stored on a streaming server (205) for future use. One or more streaming clients (206, 208) may access the streaming server (205) to retrieve a copy (207, 209) of the encoded video bitstream (204). The client (206) may include a video decoder (210). The video decoder (210) decodes an incoming copy of an encoded video bitstream (207) and produces an output video sample stream (211) that can be presented on a display (212) or another presentation device (not shown). In some streaming systems, the video bitstream (204, 207, 209) may be encoded according to certain video codec / compression standards. Examples of these standards include ITU-T Recommendation H.265. The video codec standard under development is informally referred to as Versatile Video Codec (VVC). The disclosed subject matter may be used in the context of VVC.

[0027] Figure 3 is a functional block diagram of a video decoder (210) according to an embodiment of the present disclosure.

[0028] The receiver (310) may receive one or more encoded video sequences to be decoded by the video decoder (210); in the same or another embodiment, one encoded video sequence is received at a time, wherein the decoding of each encoded video sequence is independent of the other encoded video sequences. The encoded video sequence may be received from a channel (312), which may be a hardware / software link to a storage device storing the encoded video data. The receiver (310) may receive the encoded video data as well as other data, such as encoded audio data and / or auxiliary data streams that may be forwarded to their respective consuming entities (not shown). The receiver (310) may separate the encoded video sequence from the other data. To prevent network jitter, a buffer memory (315) may be coupled between the receiver (310) and an entropy decoder / parser (320) (hereinafter referred to as "parser"). When the receiver (310) receives data from a storage / forward device with sufficient bandwidth and controllability or from an isochronous synchronous network, it may not be necessary to configure the buffer memory (315), or the buffer memory may be made smaller. Of course, for use over a packet network such as the Internet, a buffer memory (315) may also be required, which may be relatively large and may advantageously be of an adaptive size.

[0029] The video decoder (210) may include a parser (320) to reconstruct symbols (321) from an entropy coded video sequence. The categories of these symbols include information for managing the operation of the video decoder (210) and potentially information for controlling a display device such as display 212 that is not part of the decoder but may be coupled to the decoder, such as Figure 3As shown in . The control information for the display device may be in the form of a parameter set fragment (not indicated) of Supplementary Enhancement Information (SEI message) or Video Usability Information (VUI). The parser (320) may parse / entropy decode the received encoded video sequence. The encoding and decoding of the encoded video sequence may be performed according to a video encoding and decoding technique or standard, and may follow principles well known to those skilled in the art, including variable length encoding and decoding, Huffman coding, arithmetic coding and decoding with or without context sensitivity, and the like. The parser (320) may extract a subgroup parameter set for at least one subgroup of the subgroups of pixels in the video decoder from the encoded video sequence based on at least one parameter corresponding to the group. Subgroups may include Group of Pictures (GOP), pictures, sub-pictures, tiles, slices, bricks, macroblocks, Coding Tree Units (CTU), Coding Units (CU), blocks, Transform Units (TU), Prediction Units (PU), and the like. A tile may indicate a rectangular area of ​​a CU / CTU within a specific tile column and row in a picture. A brick may indicate a rectangular area of ​​a CU / CTU row within a specific tile. A slice may indicate one or more tiles of a picture, which are contained in a NAL unit. A sub-picture may indicate a rectangular area of ​​one or more slices in a picture. The entropy decoder / parser may also extract information such as transform coefficients, quantizer parameter values, motion vectors, and the like from an encoded video sequence.

[0030] The parser (320) may perform entropy decoding / parsing operations on the video sequence received from the buffer memory (315) to create symbols (321).

[0031] Depending on the type of coded video picture or portion of coded video picture (e.g., inter- and intra-pictures, inter- and intra-blocks) and other factors, the reconstruction of the symbol (321) may involve multiple different units. Which units are involved and how they are involved may be controlled by subgroup control information parsed by the parser (320) from the coded video sequence. For the sake of brevity, such subgroup control information flow between the parser (320) and the multiple units below is not described.

[0032] In addition to the functional blocks already mentioned, the video decoder (210) can be conceptually subdivided into several functional units as described below. In actual implementations operating under commercial constraints, many of these units interact closely with each other and may be at least partially integrated with each other. However, for the purposes of describing the disclosed subject matter, the conceptual subdivision into the following functional units is appropriate.

[0033] The first unit is a sealer / inverse transform unit (351). The sealer / inverse transform unit (351) receives quantized transform coefficients as symbols (321) from the parser (320) and control information, including which transform method to use, block size, quantization factor, quantization scaling matrix, etc. The sealer / inverse transform unit (351) can output a block including sample values, which can be input into an aggregator (355).

[0034] In some cases, the output samples of the scaler / inverse transform unit (351) may belong to an intra-coded block; that is, a block that does not use predictive information from a previously reconstructed picture, but may use predictive information from a previously reconstructed portion of the current picture. Such predictive information may be provided by an intra-picture prediction unit (352). In some cases, the intra-picture prediction unit (352) generates a block of the same size and shape as the block being reconstructed using surrounding reconstructed information extracted from the (partially reconstructed) current picture (358). In some cases, the aggregator (355) adds the prediction information generated by the intra-prediction unit (352) to the output sample information provided by the scaler / inverse transform unit (351) on a per-sample basis.

[0035] In other cases, the output samples of the scaler / inverse transform unit (351) may belong to an inter-frame coded and potentially motion compensated block. In this case, the motion compensated prediction unit (353) may access the reference picture memory (357) to extract samples for prediction. After the extracted samples are motion compensated according to the symbols (321), these samples may be added to the output of the scaler / inverse transform unit (in this case referred to as residual samples or residual signals) by the aggregator (355) to generate output sample information. The motion compensated prediction unit's retrieval of predicted samples from addresses within the reference picture memory may be controlled by motion vectors, and the motion vectors are provided to the motion compensated prediction unit in the form of the symbols (321), for example, including X, Y and reference picture components. Motion compensation may also include interpolation of sample values ​​extracted from the reference picture memory when using sub-sample accurate motion vectors, motion vector prediction mechanisms, and the like.

[0036] The output samples of the aggregator (355) may be employed by various loop filtering techniques in a loop filter unit (356). The video compression techniques may include in-loop filter techniques controlled by parameters included in the encoded video bitstream and available to the loop filter unit (356) as symbols (321) from the parser (320). However, the video compression techniques may also be responsive to meta-information obtained during decoding of a previous (in decoding order) portion of an encoded picture or encoded video sequence, and to previously reconstructed and loop filtered sample values.

[0037] The output of the loop filter unit (356) may be a sample stream that may be output to a display device (212) and stored in a reference picture memory for subsequent inter-picture prediction.

[0038] Once fully reconstructed, certain coded pictures may be used as reference pictures for future prediction. Once a coded picture is fully reconstructed, and the coded picture is identified as a reference picture (e.g., by the parser (320)), the current picture (358) may become part of the reference picture memory (357), and a new current picture memory may be reallocated before starting reconstruction of a subsequent coded picture.

[0039] The video decoder (210) may perform decoding operations according to a predetermined video compression technique as documented, for example, in the ITU-T H.265 standard. The encoded video sequence may conform to the syntax specified by the video compression technique or standard used, in the sense that the encoded video sequence follows the syntax of the video compression technique or standard as specified in the video compression technique document or standard, in particular, in a profile. For compliance, the complexity of the encoded video sequence is also required to be within the limits defined by the hierarchy of the video compression technique or standard. In some cases, the hierarchy limits the maximum picture size, maximum frame rate, maximum reconstruction sampling rate (measured in, for example, mega samples per second), maximum reference picture size, etc. In some cases, the limits set by the hierarchy may be further limited by the Hypothetical Reference Decoder (HRD) specification and metadata of the HRD buffer management signaled in the encoded video sequence.

[0040] In an embodiment, the receiver (310) may receive additional (redundant) data along with the encoded video. The additional data may be part of the encoded video sequence. The additional data may be used by the video decoder (210) to properly decode the data and / or more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0041] Figure 4 is a functional block diagram of a video encoder (203) according to an embodiment of the present disclosure.

[0042] The video encoder (203) may receive video samples from a video source (201) (not a part of the encoder) that may capture video images to be encoded by the video encoder (203).

[0043] The video source (201) may provide a source video sequence in the form of a digital video sample stream to be encoded by the video encoder (203), wherein the digital video sample stream may have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, ...), any color space (e.g., BT.601 Y CrCB, RGB, ...), and any suitable sampling structure (e.g., Y CrCb 4:2:0, Y CrCb 4:4:4). In a media service system, the video source (201) may be a storage device storing previously prepared videos. In a video conferencing system, the video source (201) may be a camera that captures local image information as a video sequence. The video data may be provided as a plurality of separate pictures that are given motion when viewed sequentially. The pictures themselves may be constructed as a spatial pixel array, wherein each pixel may include one or more samples depending on the sampling structure, color space, etc. used. The relationship between pixels and samples may be easily understood by those skilled in the art. The following description focuses on samples.

[0044] According to an embodiment, the video encoder (203) may encode and compress pictures of a source video sequence into an encoded video sequence (443) in real time or under any other time constraints required by the application. Implementing an appropriate encoding speed is a function of a controller (450). The controller controls other functional units as described below and is functionally coupled to these units. For the sake of brevity, couplings are not shown in the figure. Parameters set by the controller may include rate control related parameters (e.g., picture skipping, quantizer, lambda value of rate-distortion optimization techniques, etc.), picture size, group of pictures (GOP) layout, maximum motion vector search range, etc. Those skilled in the art can easily identify other functions of the controller (450) because they may belong to a video encoder (203) optimized for a specific system design.

[0045] Some video encoders operate in a manner that is readily recognizable to those skilled in the art as a "coding loop". As a simple description, the coding loop may include an encoding portion of an encoder (430) (hereinafter referred to as a "source encoder", which is responsible for creating symbols based on an input picture to be encoded and a reference picture), and a (local) decoder (433) embedded in the video encoder (203). The "local" decoder (433) reconstructs the symbols to create sample data in a manner similar to how the (remote) decoder creates sample data (because in the video compression techniques contemplated by the disclosed subject matter, any compression between the symbols and the encoded video code stream is lossless). The reconstructed sample stream is input to a reference picture memory (434). Since the decoding of the symbol stream produces bit-accurate results that are independent of the decoder location (local or remote), the contents of the reference picture memory also correspond bit-by-bit between the local encoder and the remote encoder. In other words, the reference picture samples "seen" by the prediction portion of the encoder are exactly the same sample values ​​that the decoder will "see" when using the prediction during decoding. Such basic principles of reference picture synchronization (and the drift that occurs when synchronization cannot be maintained, eg due to channel errors) are well known to those skilled in the art.

[0046] The operation of the "local" decoder (433) can be combined with the above Figure 3 The "remote" decoder (210) is identical to the one described in detail. However, additional brief reference is made to Figure 3 , when symbols are available and the entropy encoder (445) and the parser (320) are capable of losslessly encoding / decoding the symbols into an encoded video sequence, the entropy decoding portion of the video decoder (210) (including the channel (312), the receiver (310), the buffer memory (315), and the parser (320)) may not be fully implemented in the local decoder (433).

[0047] At this point it can be observed that any decoder techniques other than parsing / entropy decoding present in the decoder must also be present in the corresponding encoder in substantially the same functional form. For this reason, the disclosed subject matter focuses on decoder operation. The description of encoder techniques can be abbreviated because they are inverse to the fully described decoder techniques. A more detailed description is only required in certain areas and is provided below.

[0048] As part of the operation, the source encoder (430) may perform motion compensated predictive coding. The motion compensated predictive coding predictively encodes the input frame with reference to one or more previously encoded frames from the video sequence designated as "reference frames." In this manner, the encoding engine (432) encodes the differences between the pixel blocks of the input frame and the pixel blocks of the reference frame, which may be selected as a prediction reference for the input frame.

[0049] The local video decoder (433) may decode the encoded video data of the frame that may be designated as the reference frame based on the symbol created by the source encoder (430). The operation of the encoding engine (432) may advantageously be a lossy process. When the encoded video data is available at the video decoder ( Figure 4 When the video sequence is decoded at a remote location (not shown), the reconstructed video sequence may typically be a copy of the source video sequence with some errors. The local video decoder (433) replicates the decoding process that may be performed by the video decoder on the reference frame and may cause the reconstructed reference frame to be stored in the reference picture memory (434). In this way, the encoder (203) may store a copy of the reconstructed reference frame locally that has common content (absent transmission errors) with the reconstructed reference frame to be obtained by the remote video decoder.

[0050] The predictor (435) may perform a prediction search for the encoding engine (432). That is, for a new frame to be encoded, the predictor (435) may search the reference picture memory (434) for sample data (as candidate reference pixel blocks) or certain metadata, such as reference picture motion vectors, block shapes, etc., that may serve as appropriate prediction references for the new picture. The predictor (435) may operate pixel-by-pixel based on sample blocks to find a suitable prediction reference. In some cases, based on the search results obtained by the predictor (435), it may be confirmed that the input picture may have prediction references taken from a plurality of reference pictures stored in the reference picture memory (434).

[0051] The controller (450) may manage encoding operations of the source encoder (430), including, for example, setting parameters and subgroup parameters for encoding video data.

[0052] The outputs of all the above functional units may be entropy encoded in an entropy encoder (445). The entropy encoder may perform lossless compression on the symbols generated by the various functional units according to techniques known to those skilled in the art, such as Huffman coding, variable length coding, arithmetic coding, etc., thereby converting the symbols into a coded video sequence.

[0053] The transmitter (440) may buffer the encoded video sequence created by the entropy encoder (445) in preparation for transmission over a communication channel (460), which may be a hardware / software link to a storage device where the encoded video data will be stored. The transmitter (440) may combine the encoded video data from the source encoder (430) with other data to be transmitted, such as encoded audio data and / or ancillary data streams (source not shown).

[0054] The controller (450) can manage the operation of the video encoder (203). During encoding, the controller (450) can assign a certain coded picture type to each coded picture, but this may affect the coding techniques that can be applied to the corresponding picture. For example, a picture can generally be assigned to any of the following picture types.

[0055] An intra picture (I picture) may be a picture that can be encoded and decoded without using any other frame in the sequence as a prediction source. Some video codecs allow different types of intra pictures, including, for example, Independent Decoder Refresh (IDR) pictures. Those skilled in the art are aware of those variants of I pictures and their respective applications and features.

[0056] A predictive picture (P picture) may be a picture that can be encoded and decoded using intra prediction or inter prediction, which uses at most one motion vector and a reference index to predict sample values ​​for each block.

[0057] Bidirectional predictive pictures (B pictures), which can be pictures that can be encoded and decoded using intra prediction or inter prediction, which uses up to two motion vectors and reference indices to predict the sample values ​​of each block. Similarly, multiple predictive pictures can use more than two reference pictures and associated metadata for reconstruction of a single block.

[0058] The source picture may typically be spatially subdivided into blocks of samples (e.g., blocks of 4×4, 8×8, 4×8, or 16×16 samples), and coded block by block. These blocks may be predictively coded with reference to other (already coded) blocks, identified according to the coding assignments applied to the blocks' respective pictures. For example, blocks of I pictures may be non-predictively coded, or they may be predictively coded (spatial prediction or intra prediction) with reference to already coded blocks of the same picture. Blocks of P pictures may be predictively coded by spatial prediction with reference to one previously coded reference picture or by temporal prediction. Blocks of B pictures may be predictively coded by spatial prediction with reference to one or two previously coded reference pictures or by temporal prediction.

[0059] The video encoder (203) may perform encoding operations according to a predetermined video encoding technique or standard, such as ITU-T H.265 Recommendation. In operation, the video encoder (203) may perform various compression operations, including predictive encoding operations that exploit temporal and spatial redundancy in an input video sequence. Thus, the encoded video data may conform to the syntax specified by the video encoding technique or standard used.

[0060] In an embodiment, the transmitter (440) may transmit additional data along with the encoded video. The source encoder (430) may include such data as a portion of the encoded video sequence. The additional data may include other forms of redundant data such as temporal / spatial / SNR enhancement layers, redundant pictures and slices, SEI messages, VUI parameter set fragments, etc.

[0061] Recently, there has been some interest in the compressed domain aggregation or extraction of multiple semantically independent picture parts into a single video picture. Specifically, in the context of, for example, 360 encoding or certain surveillance applications, multiple semantically independent source pictures (e.g., six cube surfaces of a 360 scene of a cube projection, or individual camera inputs in a multi-camera surveillance setup) may require separate adaptive resolution settings to cope with different scene activities at a given point in time. In other words, the encoder can choose to use different resampling factors for different semantically independent pictures that make up the entire 360 ​​scene or surveillance scene at a given point in time. This in turn requires reference picture resampling to be performed on parts of the encoded picture when combined into a single picture, and adaptive resolution encoding signaling is available.

[0062] The following is an introduction to some terms that will be referred to in the rest of this specification.

[0063] In some cases, a sub-picture may refer to a rectangular arrangement of samples, blocks, macroblocks, coding units, or similar entities that are semantically grouped and can be independently encoded at varying resolutions. One or more sub-pictures may form a picture. One or more encoded sub-pictures may form a coded picture. One or more sub-pictures may be combined into a picture, and one or more sub-pictures may be extracted from a picture. In some environments, one or more coded sub-pictures may be combined in a compressed domain without transcoding to the sample level to become a coded picture, and in the same or other cases, one or more coded sub-pictures may be extracted from a coded picture in the compressed domain.

[0064] Adaptive resolution change (ARC) may refer to a mechanism that allows the resolution of a picture or sub-picture within a coded video sequence to be changed, for example, by resampling a reference picture. Hereinafter, ARC parameters refer to control information required to perform adaptive resolution change, which may include, for example, filter parameters, scaling factors, resolution of output and / or reference pictures, various control flags, etc.

[0065] In an embodiment, encoding and decoding may be performed on a single semantically independent coded video picture.Before describing the implications of encoding / decoding of multiple sub-pictures with independent ARC parameters and the additional complexity it implies, options for signaling ARC parameters will be described.

[0066] refer to Figures 5A-5E , several embodiments of signaling ARC parameters are shown. As described in each embodiment, they may have certain advantages and disadvantages from the perspective of codec efficiency, complexity, and architecture. Video codec standards or technologies may select one or more of these embodiments, or options known in the relevant art, to signal ARC parameters. These embodiments may not be mutually exclusive and may be interchangeable based on application needs, the standard technologies involved, or the selection of encoders.

[0067] Categories of ARC parameters can include:

[0068] - Up / down sampling factors in X and Y dimensions separately or combined

[0069] -Add the up / down sampling factor of the time dimension, which means that a given number of pictures are enlarged / reduced at a constant speed

[0070] - Either of the above two categories may involve the encoding of one or more possibly shorter syntax elements, which may point to a table containing one or more factors.

[0071] - The combined or individual resolutions of the input picture, output picture, reference picture, coded picture in the X or Y dimension in units of samples, blocks, macroblocks, coding units (CUs), or any other suitable granularity. If there is more than one resolution (e.g., one for the input picture and one for the reference picture), then in some cases one set of values ​​can be inferred from the other. This can be gated, for example, using a flag. See below for more detailed examples.

[0072] - "Warping" coordinates are similar to those used in H.263 Annex P, which are also in units of the appropriate granularity mentioned above. H.263 Annex P defines an efficient way to encode such warping coordinates, but other, potentially more efficient ways can also be envisioned. For example, the variable-length reversible "Huffman"-style encoding of the warping coordinates of Annex P can be replaced by a binary encoding of appropriate length, where the length of the binary codeword can be derived, for example, from the maximum picture size, possibly multiplied by some factor and offset by some value to allow "warping" outside the boundaries of the maximum picture size.

[0073] - Upsampling or downsampling filter parameters. In embodiments, there may be only a single filter for upsampling and / or downsampling. However, in embodiments, it may be desirable to allow greater flexibility in filter design, and this may require signaling of filter parameters. These parameters may be selected by an index into a list of possible filter designs, the filter may be fully specified (e.g., by a list of filter coefficients, using appropriate entropy coding techniques), the filter may be implicitly selected by the upsampling / downsampling rate, which in turn is signaled according to any of the above mechanisms, etc.

[0074] Hereinafter, this specification assumes that a limited set of upsampling / downsampling factors are encoded (using the same factors in both the X and Y dimensions), represented by codewords. The codewords may be variable length encoded, for example, using Ext-Golomb codes common to certain syntax elements in video codec specifications such as H.264 and H.265. For example, a suitable mapping from values ​​to upsampling / downsampling factors may be according to Table 1:

[0075] Table 1

[0076] Codeword Ext-Golomb Code Original / target resolution 0 1 1 / 1 1 010 1 / 1.5 (50% zoom) 2 011 1.5 / 1 (50% smaller) 3 00100 1 / 2 (100% zoom) 4 00101 2 / 1 (reduced by 100%)

[0077] Many similar mappings can be designed depending on the needs of the application and the capabilities of the upscaling and downscaling mechanisms available in the video compression technology or standard. The table can be extended to more values. The values ​​can also be represented by entropy coding mechanisms other than Ext-Golomb codes, for example using binary encoding. This may have certain advantages when, for example, the MANE is interested in the resampling factors outside the video processing engine (most importantly the encoder and decoder) itself. It should be noted that for cases where no resolution change is required, a shorter Ext-Golomb code can be chosen; in the table above, only one bit. In the most common cases, this has the advantage of codec efficiency over using binary codes.

[0078] The number of entries in the table and their semantics may be fully or partially configurable. For example, a basic outline of the table may be conveyed in a "high" parameter set such as a sequence or decoder parameter set. In an embodiment, one or more such tables may be defined in a video codec technology or standard and may be selected by, for example, a decoder or sequence parameter set.

[0079] The following describes how to include the upsampling / downsampling factors (ARC information) encoded as described above in a video codec or standard syntax. Similar considerations can be applied to one or more codewords that control upsampling / downsampling filters. When the filter or other data structure requires a relatively large amount of data, please refer to the discussion below.

[0080] like Figure 5A As shown, H.263 Annex P includes ARC information (502) in the picture header (501) in the form of four warp coordinates, specifically in the H.263PLUSPTYPE (503) header extension. This can be a wise design choice when a) picture headers are available and b) the ARC information is expected to change frequently. However, when H.263 style signaling is used, the overhead can be quite high, and because picture headers can be transient in nature, the scaling factors may not apply across picture boundaries.

[0081] like Figure 5B As shown, JVCET-M135-v1 includes ARC reference information (505) (index) located in a picture parameter set (504), which indexes a table (506) including target resolutions, which is located in a sequence parameter set (507). By using the SPS as an interoperability negotiation point during capability exchange, the placement of possible resolutions in the table (506) of the sequence parameter set (507) can be demonstrated. By referencing the appropriate picture parameter set (504), the resolution can be changed as the picture changes within the limits set by the values ​​in the table (506).

[0082] refer to Figure 5C-5E , there may be the following embodiments to transmit ARC information in a video bitstream. Each of these options has certain advantages over the above embodiments. Embodiments may exist simultaneously in the same video codec technology or standard.

[0083] In an embodiment, for example Figure 5C In the illustrated embodiment, ARC information (509) such as a resampling (scaling) factor may be present in a slice header, a GOP header, a tile header, or a tile group header. Figure 5C An embodiment using a tile group header (508) is shown. This may be sufficient, for example, if the ARC information is small, such as a single variable length ue(v) or a fixed length codeword of a few bits, as described above. Placing the ARC information directly in the tile group header has the additional advantage that the ARC information may apply, for example, to a sub-picture represented by the tile group, rather than to the entire picture, see also below. In addition, even if the video compression technique or standard only envisions adaptive resolution changes for the entire picture (e.g., as compared to adaptive resolution changes based on tile groups), placing the ARC information in the tile group header has certain advantages from an error recovery perspective compared to placing it in the picture header of the H.263 format.

[0084] In an embodiment, for example Figure 5D In the illustrated embodiment, the ARC information (512) itself may be present in an appropriate parameter set, such as a picture parameter set, a header parameter set, a tile parameter set, an adaptation parameter set, or the like. Figure 5D An embodiment using an adaptive parameter set (511) is shown. Advantageously, the scope of the parameter set may be no larger than a picture, such as a group of tiles. The use of ARC information is implicit by activating the relevant parameter set. For example, when a video codec technology or standard only considers picture-based ARC, then a picture parameter set or equivalent parameters may be applicable.

[0085] In an embodiment, for example Figure 5E In the illustrated embodiment, ARC reference information (513) may be present in a tile group header (514) or similar data structure. The reference information (513) may refer to a subset of ARC information (515) available in a parameter set (516) that extends beyond a single picture, such as a sequence parameter set or a decoder parameter set.

[0086] Since picture parameter sets, like sequence parameter sets, can (and in some standards (e.g. RFC3984) are) used for capability negotiation or announcement, implicitly activating additional levels of PPS from tile group headers, PPS, SPS (as used in JVET-M0135-v1) seems unnecessary. However, if the ARC information should also apply to, for example, sub-pictures represented by tile groups, parameter sets whose activation scope is limited to tile groups (e.g., adaptation parameter sets or header parameter sets) may be a better choice. Furthermore, if the size of the ARC information exceeds a negligible size - for example containing filter control information such as multiple filter coefficients - then from a codec efficiency point of view, parameters may be a better choice than using the header (508) directly, since these settings can be reused by future pictures or sub-pictures by referring to the same parameter set.

[0087] When using a sequence parameter set or another higher parameter set whose range spans multiple pictures, certain considerations may apply:

[0088] 1. In some cases, the parameter set storing the ARC information table (516) may be a sequence parameter set, but in other cases a decoder parameter set is more advantageous. A decoder parameter set may have an activation range of multiple CVSs, i.e., coded video bitstreams, i.e., all coded video bits from session start to session teardown. Such a range may be more appropriate because possible ARC factors may be decoder features, which may be implemented in hardware, and hardware features tend not to change with any CVS (which in at least some entertainment systems is a set of pictures, one second or less in length). That is, placing the table in a sequence parameter set is explicitly included in the placement options described herein, particularly in conjunction with point 2 below.

[0089] 2. The ARC reference information (513) can be advantageously placed directly in the picture / slice / tile / GOP / tile group header, such as the tile group header (514), rather than in the picture parameter set as in JVCET-M0135-v1. For example, when the encoder wants to change a single value in the picture parameter set (such as the ARC reference information), it must create a new PPS and reference the new PPS. Assume that only the ARC reference information changes, while other information (such as the quantization matrix information in the PPS) is retained. Such information may be large and requires retransmission to make the new PPS complete. Since the ARC reference information (513) can be a single codeword, such as an index in a table, and this is the only value that changes, it would be cumbersome and wasteful to retransmit all of the quantization matrix information, such as the quantization matrix information. In this regard, from the perspective of codec efficiency, indirect access through the PPS can be better avoided, as suggested in JVET-M0135-v1. Similarly, placing the ARC reference information in the PPS has another disadvantage, namely, since the scope of picture parameter set activation is a picture, the ARC information referred to by the ARC reference information (513) may be applicable to the entire picture rather than a sub-picture.

[0090] In the same or another embodiment, the signaling of ARC parameters may follow the following: Figure 6A-6B A detailed example is outlined in . Figure 6A-6B Syntax diagrams are depicted in a representation using notation that roughly follows C-style programming, such as used in video codec standards since at least 1993. Lines in boldface represent syntax elements present in the bitstream, and lines that are not in boldface typically represent control flow or variable settings.

[0091] like Fig. 6A As shown, as an exemplary syntax structure for a header that may be applicable to a portion of a picture (possibly rectangular), a tile group header (601) may conditionally contain a variable-length, Exp-Golomb coded syntax element dec_pic_size_idx (602) (in boldface). The presence of this syntax element in the tile group header may be gated based on the use of adaptive resolution (603) - here, the value of the flag is not in boldface, meaning that the flag is present in the bitstream at the point where it appears in the syntax diagram. Whether adaptive resolution is used for this picture or portion thereof may be signaled in any higher-level syntax structure, either internal or external to the bitstream. In the example shown, it is signaled in a sequence parameter set, as described below.

[0092] refer to Figure 6B, an excerpt of a sequence parameter set (610) is also shown. The first syntax element shown is adaptive_pic_resolution_change_flag (611). When true, this flag may indicate the use of adaptive resolution, which in turn may require certain control information. In this example, such control information appears conditionally based on the value of the flag, which is based on the parameter set (612) and the if() statement in the tile group header (601).

[0093] When adaptive resolution is used, in this example, what is encoded is the output resolution (613) in units of samples. Number 613 refers to output_pic_width_in_luma_samples and output_pic_height_in_luma_samples, which can jointly define the resolution of the output picture. Elsewhere in the video codec technology or standard, certain restrictions on any one value can be defined. For example, the level definition can limit the number of total output samples, which can be the product of the values ​​of these two syntax elements. Moreover, some video codec technologies or standards, or external technologies or standards (such as system standards) may limit the number range (for example, one or two dimensions must be divisible by powers of 2) or aspect ratio (for example, width and height must be, for example, 4:3 or 16:9). Such restrictions can be introduced to facilitate hardware implementation, or for other reasons, and are well known in the art.

[0094] In some applications, it may be advisable for the encoder to instruct the decoder to use a certain reference picture size, rather than implicitly assuming that size as the output picture size. In this example, the syntax element reference_pic_size_present_flag (614) enables the conditional presence of the reference picture dimensions (615) (again, the numbers refer to width and height).

[0095] Finally, a table of possible decoded picture widths and heights is shown. Such a table may be indicated, for example, by a table indication (num_dec_pic_size_in_luma_samples_minus1) (616). "Minus1" may refer to the interpretation of the value of this syntax element. For example, if the coded value is zero, there is one table entry. If the value is 5, there are six table entries. For each "row" in the table, the decoded picture width and height are included in the syntax (617).

[0096] The presented table entry (617) may be indexed using the syntax element dec_pic_size_idx (602) in the tile group header, allowing each tile group to have a different decoded size - effectively a scaling factor.

[0097] In the related technical implementation of VVC, when the reference picture width is different from the current picture width, there may be a problem that the surround motion compensation does not work correctly. In an embodiment, when the layer of the current picture is a dependent layer (dependentlayer), or RPR is enabled for the current layer, the surround motion compensation can be disabled in the high-level syntax. In an embodiment, when the reference picture width is different from the current picture width, the surround processing can be disabled during the interpolation process of motion compensation.

[0098] Surround motion compensation can be a useful feature for encoding, for example, 360 projection pictures with equirectangular projection (ERP) format. It can reduce some visual artifacts at the seams and improve the codec gain. In the current VVC specification draft JVET-P2001 (edited and updated by JVET-Q0041), sps_ref_wraparound_offset_minus1 in SPS specifies the offset used to calculate the horizontal wraparound position.

[0099] A problem may arise that the surround offset value is determined relative to the picture width. If the picture width of the reference picture is different from the current picture width, the surround offset value should change proportionally with the scaling ratio between the current picture and the reference picture. However, in practice, adjusting the offset value according to the picture width of each reference picture may significantly increase the implementation and computational complexity compared to the advantages of surround motion compensation. Inter-layer prediction and reference picture resampling (RPR) with different picture sizes may result in very different combinations of different picture resolutions across layers and across temporal pictures.

[0100] Embodiments may address this issue. For example, in an embodiment, when the layer of the current picture is a dependent layer, or RPR is enabled for the current layer, wraparound motion compensation may be disabled by setting sps_ref_wraparound_enabled_flag to 0. Therefore, wraparound motion compensation may be used only when the current layer is an independent layer and RPR is disabled. In this case, the reference picture size should be equal to the current picture size. In addition, in an embodiment, when the reference picture width is different from the current picture width, the wraparound motion compensation process may be disabled during the interpolation process of motion compensation.

[0101] The embodiments may be used alone or in any order. In addition, each of the method (or embodiment), encoder, and decoder may be implemented by a processing circuit (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored in a non-volatile computer-readable medium.

[0102] Figure 7An example syntax table according to an embodiment is shown. In an embodiment, sps_ref_wraparound_enabled_flag(701) equal to 1 may specify that horizontal wrap motion compensation is applied in inter prediction. sps_ref_wraparound_enabled_flag(701) equal to 0 may specify that horizontal wrap motion compensation is not applied. When the value of (CtbSizeY / MinCbSizeY+1) is less than or equal to (pic_width_in_luma_samples / MinCbSizeY-1), the value of sps_ref_wraparound_enabled_flag(701) may be equal to 0, where pic_width_in_luma_samples is the value of pic_width_in_luma_samples in any PPS that references the SPS. A requirement for bitstream consistency may be that when vps_independent_layer_flag[GeneralLayerIdx[nuh_layer_id]] is equal to 0, the value of sps_ref_wraparound_enabled_flag(701) shall be equal to 0. When not present, the value of sps_ref_wraparound_enabled_flag (701) may be inferred to be equal to 0.

[0103] In an embodiment, refPicWidthInLumaSamples may be the pic_width_in_luma_samples of the current reference picture of the current picture. In an embodiment, if refPicWidthInLumaSamples is equal to the pic_width_in_luma_samples of the current picture, refWraparoundEnabledFlag may be set equal to sps_ref_wraparound_enabled_flag. Otherwise, refWraparoundEnabledFlag may be set equal to 0.

[0104] For i = 0..1, the brightness position in the full sample unit (xInti, yInti) can be derived as follows:

[0105] If subpic_treated_as_pic_flag[SubPicIdx] is equal to 1, the following may apply:

[0106] xInt i =Clip3(SubPicLeftBoundaryPos,SubPicRightBoundaryPos,xIntL +i) (Equation 1)

[0107] yInt i =Clip3(SubPicTopBoundaryPos,SubPicBotBoundaryPos,yInt L +i) (Equation 2)

[0108] Otherwise (subpic_treated_as_pic_flag[SubPicIdx] is equal to 0), the following may apply:

[0109] xInt i =Clip3(0,picW-1,refWraparoundEnabledFlag?

[0110] ClipH((sps_ref_wraparound_offset_minus1+1)*MinCbSizeY,picW,(xInt L +i)):xInt L +i) (Equation 3)

[0112] yInt i =Clip3(0,picH-1,yInt L +i) (Equation 4)

[0113] The brightness position in full sample units (xInt, yInt) can be derived as follows:

[0114] If subpic_treated_as_pic_flag[SubPicIdx] is equal to 1, the following applies:

[0115] xInt=Clip3(SubPicLeftBoundaryPos,SubPicRightBoundaryPos,xInt L ) (Equation 5)

[0116] yInt=Clip3(SubPicTopBoundaryPos,SubPicBotBoundaryPos,yInt L ) (Equation 6)

[0117] Otherwise, the following may apply:

[0118] xInt=Clip3(0,picW-1,

[0119] refWraparoundEnabledFlag?

[0120] ClipH((sps_ref_wraparound_offset_minus1+1)*MinCbSizeY,picW,xInt L ):

[0121] xInt L ) (Equation 7)

[0122] yInt=Clip3(0,picH-1,yInt L ) (Equation 8)

[0123] The predicted brightness sample value predSampleLXL can be derived as follows:

[0124] predSampleLXL=refPicLXL[xInt][yInt]< <shift3

[0125] For i = 0..3, the chrominance position in the full sample unit (xInti, yInti) can be derived as follows:

[0126] If subpic_treated_as_pic_flag[SubPicIdx] is equal to 1, the following may apply:

[0127] xInt i =Clip3(SubPicLeftBoundaryPos / SubWidthC,SubPicRightBoundaryPos / SubWidthC,xInt C +i-1)(Equation 9)

[0129] yInt i =Clip3(SubPicTopBoundaryPos / SubHeightC,SubPicBotBoundaryPos / SubHeightC,yInt C +i-1)(Equation 10)

[0131] Otherwise (subpic_treated_as_pic_flag[SubPicIdx] is equal to 0), the following may apply:

[0132] xInt i =Clip3(0,picW C -1,

[0133] refWraparoundEnabledFlag? ClipH(xOffset,picWC ,xInt C +i-1):xInt C +i-1) (Equation 11)

[0134] yInt i =Clip3(0,picH C -1,yInt C +i-1)(Equation 12)

[0135] For i = 0..3, the chrominance position in the full sample unit (xInti, yInti) can be derived as follows:

[0136] xInt i =Clip3(xSbIntC-1,xSbIntC+sbWidth+2,xInt i ) (Equation 13)

[0137] yInt i =Clip3(ySbIntC-1,ySbIntC+sbHeight+2,yInt i ) (Equation 14)

[0138] Figures 8A-8C 800A, 800B, and 800C are flow charts of example processes 800A, 800B, and 800C for generating encoded video bitstreams according to an embodiment. In an embodiment, any one of the processes 800A, 800B, and 800C, or any portion of the processes 800A, 800B, and 800C, may be combined in any combination or arrangement and in any order as desired. In some implementations, Figures 8A-8C One or more process blocks of may be performed by decoder 210. In some implementations, Figures 8A-8C One or more process blocks of may be performed by another device or a group of devices separate from or including the decoder 210 , such as the encoder 203 .

[0139] like Fig. 8A As shown, process 800A may include performing a first check to determine whether a current layer of a current picture is an independent layer (block 811 ).

[0140] like Fig. 8A As further shown, process 800A may include performing a second determination to determine whether reference picture resampling is enabled for the current layer (block 812).

[0141] like Fig. 8A As further shown, process 800A may include disabling surround compensation for the current layer based on the first confirmation and the second confirmation (block 813).

[0142] like Fig. 8A As further shown, process 800A may include encoding the current layer without performing the surround compensation (block 814).

[0143] In an embodiment, the first confirmation may be made based on a first flag signaled in a first syntax structure, and the second confirmation may be made based on a second flag signaled in a second syntax structure, the second syntax structure being lower than the first syntax structure.

[0144] In an embodiment, the first flag may be signaled in a video parameter set and the second flag may be signaled in a sequence parameter set.

[0145] In an embodiment, surround compensation may be disabled based on the absence of a second flag in the sequence parameter set.

[0146] like Figure 8B As shown, process 800B may include determining whether a current layer of a current picture is an independent layer (block 821 ).

[0147] like Figure 8B As further shown, if it is determined that the current layer is not an independent layer ("No" at block 821), process 800B may proceed to block 822 where surround motion compensation may be disabled.

[0148] like Figure 8B As further shown, if it is determined that the current layer is an independent layer (“yes” at block 821 ), process 800B may proceed to block 823 .

[0149] like Figure 8B As further shown, process 800B may include determining whether reference picture resampling is enabled (block 823).

[0150] like Figure 8B As further shown, if it is determined that reference picture resampling is enabled ("yes" at block 823), process 800B may proceed to block 822 where surround motion compensation may be disabled.

[0151] like Figure 8B As further shown, if it is determined that reference picture resampling is not enabled ("NO" at block 823), process 800B may proceed to block 824 where surround motion compensation may be enabled.

[0152] like Figure 8C As shown, process 800C may include confirming that a current layer of a current picture is an independent layer (block 831).

[0153] like Figure 8C As further shown, process 800C may include confirming that reference picture resampling is enabled (block 832).

[0154] like Figure 8C As further shown, process 800C may include determining whether the width of the current picture is different from the width of the current reference picture (block 833).

[0155] like Figure 8C As further shown, if it is determined that the width of the current picture is different from the width of the current reference picture ("yes" at block 833), process 800C may proceed to block 834 where surround motion compensation may be disabled.

[0156] like Figure 8C As further shown, if the width of the current picture is determined to be the same as the width of the current reference picture ("No" at block 833), process 800C may proceed to block 835 where surround motion compensation may be enabled.

[0157] In an embodiment, block 833 may be performed during a motion compensated interpolation process.

[0158] Although Figures 8A-8C Example blocks of processes 800A, 800B, and 800C are shown, but in some implementations, processes 800A, 800B, and 800C may include more Figures 8A-8C More blocks, fewer blocks, different blocks, or differently arranged blocks as depicted in the process 800A, 800B, and 800C may be performed in parallel, or alternatively.

[0159] In addition, the proposed method can be implemented by a processing circuit (eg, one or more processors or one or more integrated circuits). In an example, the one or more processors execute a program stored in a non-transitory computer-readable medium to perform one or more of the proposed methods.

[0160] The above techniques may be implemented as computer software via computer-readable instructions and physically stored in one or more computer-readable media. Fig. 9 A computer system 900 is shown that is suitable for implementing some embodiments of the disclosed subject matter.

[0161] The computer software may be encoded in any suitable machine code or computer language, and may be assembled, compiled, linked, or the like to create a code comprising instructions, which may be directly executed by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc., or executed by decoding, microcode, or the like.

[0162] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablets, servers, smartphones, gaming devices, IoT devices, etc.

[0163] Fig. 9 The components shown for computer system 900 are exemplary in nature and are not intended to impose any limitations on the scope of use or functionality of computer software implementing the disclosed embodiments. The configuration of the components should not be interpreted as having any dependency or requirement on any component or combination of components shown in the exemplary embodiment of computer system 900.

[0164] The computer system 900 may include certain human-computer interface input devices. Such human-computer interface input devices may respond to input from one or more human users through tactile input (e.g., keyboard input, sliding, data glove movement), audio input (e.g., sound, applause), visual input (e.g., gestures), and olfactory input (not shown). The human-computer interface devices may also be used to capture certain media that are not necessarily directly related to conscious human input, such as audio (e.g., speech, music, ambient sound), images (e.g., scanned images, photographic images obtained from a still image camera), and videos (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0165] The human-machine interface input devices may include one or more of the following (only one of which is depicted): keyboard 901 , mouse 902 , touchpad 903 , touch screen 910 and associated graphics adapter 950 , data gloves, joystick 905 , microphone 906 , scanner 907 , camera 908 .

[0166] The computer system 900 may also include certain human-computer interface output devices. Such human-computer interface output devices may stimulate the senses of one or more human users through, for example, tactile output, sound, light, and smell / taste. Such human-computer interface output devices may include tactile output devices (e.g., tactile feedback through a touch screen 910, a data glove, or a joystick 905, but there may also be tactile feedback devices that are not used as input devices), audio output devices (e.g., speakers 909, headphones (not shown)), visual output devices (e.g., screens 910 including cathode ray tube (CRT) screens, liquid crystal display (LCD) screens, plasma screens, organic light emitting diode (OLED) screens, each of which has or does not have a touch screen input function, each of which has or does not have a tactile feedback function - some of which may output two-dimensional visual outputs or outputs of more than three dimensions by means such as stereoscopic picture output; virtual reality glasses (not shown), holographic displays, and smoke boxes (not shown)) and printers (not shown).

[0167] The computer system 900 may also include human-accessible storage devices and their associated media, such as optical media including high-density read-only / rewritable optical disks (CD / DVD ROM / RW) 920 with CD / DVD or similar media 921, a thumb drive 922, a removable hard disk drive or solid state drive 923, traditional magnetic media such as tapes and floppy disks (not shown), special-purpose ROM / ASIC / PLD-based devices such as security software protectors (not shown), and the like.

[0168] Those skilled in the art will also appreciate that the term "computer-readable media" used in connection with the presently disclosed subject matter does not include transmission media, carrier waves, or other transient signals.

[0169] The computer system 900 may also include an interface to one or more communication networks (955). For example, the network may be wireless, wired, or optical. The network may also be a local area network, a wide area network, a metropolitan area network, an in-vehicle network, an industrial network, a real-time network, a delay-tolerant network, and the like. The network also includes local area networks such as Ethernet, wireless local area networks, cellular networks (including global mobile communication systems (GSM), third generation (3G), fourth generation (4G), fifth generation (5G), long term evolution (LTE), etc.), television wired or wireless wide area digital networks (including cable television, satellite television, and terrestrial broadcast television), in-vehicle and industrial networks (including CANBus), and the like. Some networks typically require an external network interface adapter (954) for connecting to some universal data ports or peripheral buses (949) (e.g., a universal serial bus (USB) port of the computer system 900); other systems are typically integrated into the core of the computer system 900 by connecting to a system bus as described below (e.g., an Ethernet interface integrated into a PC computer system or a cellular network interface integrated into a smart phone computer system). As an example, the network 955 can be connected to the peripheral bus 949 using the network interface 954. By using any of these networks, the computer system 900 can communicate with other entities. The communication can be one-way, only for receiving (e.g., wireless television), one-way only for sending (e.g., CAN bus to certain CAN bus devices), or two-way, such as through a local or wide area digital network to other computer systems. Each of the above networks and network interfaces (954) can use certain protocols and protocol stacks.

[0170] The above-mentioned human-machine interface devices, human-accessible storage devices, and network interfaces can be connected to the core 940 of the computer system 900 .

[0171] The core 940 may include one or more central processing units (CPUs) 941, graphics processing units (GPUs) 942, dedicated programmable processing units in the form of field programmable gate arrays (FPGAs) 943, hardware accelerators 944 for specific tasks, and the like. These devices, as well as read-only memory (ROM) 945, random access memory (RAM) 946, internal mass storage (e.g., internal non-user accessible hard disk drive, solid state drive (SSD)), and the like) 947, and the like may be connected via a system bus 948. In some computer systems, the system bus 948 may be accessed in the form of one or more physical plugs so that it may be expanded by additional central processing units, graphics processing units, and the like. Peripheral devices may be attached directly to the system bus 948 of the core, or connected via a peripheral bus 949. The architecture of the peripheral bus includes a peripheral component interconnect (PCI), a universal serial bus USB, and the like.

[0172] The CPU 941, GPU 942, FPGA 943, and accelerator 944 may execute certain instructions, which in combination may constitute the above-mentioned computer code. The computer code may be stored in ROM 945 or RAM 946. Transient data may also be stored in RAM 946, while permanent data may be stored in, for example, internal mass storage 947. Fast storage and retrieval of any memory device may be achieved by using a cache memory, which may be closely associated with one or more CPUs 941, GPUs 942, mass storage 947, ROM 945, RAM 946, etc.

[0173] The computer readable medium may have computer code thereon for performing various computer-implemented operations. The medium and computer code may be specially designed and constructed for the purposes of the present disclosure, or may be the medium and code well known and available to those skilled in the art of computer software.

[0174] As an embodiment and not limitation, a computer system having architecture 900, in particular core 940, can provide the function of executing software contained in one or more tangible computer-readable media as a processor (including CPU, GPU, FPGA, accelerator, etc.). Such computer-readable media can be a medium associated with the above-mentioned user-accessible mass storage, and a specific memory of the core 940 having non-volatile properties, such as core internal mass storage 947 or ROM 945. The software implementing various embodiments of the present application can be stored in such a device and executed by the core 940. According to specific needs, the computer-readable medium may include one or more storage devices or chips. The software can enable the core 940, in particular the processor therein (including CPU, GPU, FPGA, etc.) to perform a specific process or a specific part of a specific process described herein, including defining a data structure stored in RAM 946 and modifying such a data structure according to a software-defined process. In addition or as an alternative, the computer system can provide a function that is logically hardwired or otherwise contained in a circuit (e.g., accelerator 944), which can replace the software or run with the software to perform a specific process or a specific part of a specific process described herein. References to software may include logic, and vice versa, where appropriate. References to computer-readable media may include circuits (such as integrated circuits (ICs)) storing the executing software, circuits containing the executing logic, or both, where appropriate. The present disclosure includes any suitable combination of hardware and software.

[0175] Although the present disclosure has described a number of exemplary embodiments, various changes, arrangements and various equivalent substitutions of the embodiments are within the scope of the present disclosure. It should therefore be understood that those skilled in the art are capable of designing a variety of systems and methods, which, although not explicitly shown or described herein, embody the principles of the present disclosure and are therefore within the spirit and scope of the present disclosure.

Claims

1. A method for generating a coded video bitstream, characterized in that: The method comprises: Encoding a current layer of a current picture with or without surround compensation; The surround compensation is disabled only when the width of the current picture is different from the width of the current reference picture.

2. The method according to claim 1, characterized in that A third confirmation is performed to determine whether a width of the current picture is different from a width of the current reference picture when reference picture resampling is enabled, and surround compensation is enabled when reference picture resampling is not enabled.

3. The method according to claim 2, characterized in that Whether the current layer enables reference picture resampling is determined based on a second flag signaled in a second syntax structure.

4. The method according to claim 3, characterized in that The second flag is signaled in a sequence parameter set.

5. The method according to claim 4, characterized in that Based on the absence of the second flag in the sequence parameter set, the surround compensation is disabled.

6. The method according to any one of claims 1 to 5, characterized in that The third validation is performed during the motion compensated interpolation process.

7. A method for decoding a coded video bit stream, characterized in that: The method comprises: Performing a decoding operation with or without surround compensation on a current layer of a current picture; The surround compensation is disabled only when the width of the current picture is different from the width of the current reference picture.

8. A device for generating a coded video bitstream, characterized in that The device comprises: at least one memory configured to store program code; and At least one processor is configured to read the program code and operate according to instructions of the program code to execute the method according to any one of claims 1 to 6.

9. A device for decoding a coded video bit stream, characterized in that The device comprises: at least one memory configured to store program code; and At least one processor is configured to read the program code and operate according to instructions of the program code to execute the method of claim 7.

10. A non-volatile computer readable medium, characterized in that: Used to store instructions, the instructions include one or more instructions, and the one or more instructions are configured to, when executed by one or more processors, cause the one or more processors to perform the method according to any one of claims 1 to 7.

11. A method for processing a video bitstream, characterized in that: A video bit stream is stored on a non-volatile computer-readable medium, wherein the video bit stream is generated according to the video encoding method according to any one of claims 1 to 6, or is decoded based on the video decoding method according to claim 7.

Images