Methods, apparatus, and computer programs for wrap-around motion compensation when reference picture resampling is present.

Abstract

This invention provides a method, apparatus, and computer program for wrap-around motion compensation in cases where reference picture resampling is present. [Solution] A method for generating an encoded video bitstream using at least one processor, comprising: making a first determination of whether the current layer of the current picture is an independent layer; making a second determination of whether reference picture resampling is enabled for the current layer; disabling wrap-around compensation for the current layer based on the first and second determinations; and encoding the current layer without wrap-around compensation.

Description

Technical Field

[0004] , , , , ,

[0003] ,

[0001] Cross - reference to Related Applications This application claims priority from U.S. Provisional Patent Application No. 62 / 955,520, filed on December 31, 2019, and U.S. Patent Application No. 17 / 064,172, filed on October 6, 2020, the entire contents of which are incorporated herein by reference.

[0002] Field The disclosed subject matter relates to video encoding and decoding, and more particularly to enabling and disabling wraparound motion compensation.

Background Art

[0003] Video encoding and decoding using inter - picture prediction with motion compensation is known. Uncompressed digital video can be composed of a series of pictures, each picture having spatial dimensions of, for example, 1920×1080 luminance samples and related chrominance samples. A series of pictures can have a fixed or variable picture rate (informally also known as the frame rate), for example, 60 pictures per second or 60 Hz. Uncompressed video has significant bit - rate requirements. For example, 1080p60 4:2:0 video with 8 bits per sample (1920×1080 luminance sample resolution at a frame rate of 60 Hz) requires a bandwidth close to 1.5 Gbit / s. One hour of such video requires a storage space of more than 600 GB.

[0004] One purpose of video encoding and decoding can be to reduce the redundancy of the input video signal through compression. Compression can help reduce the aforementioned bandwidth or storage space requirements by more than two orders of magnitude, in some cases. Both lossless and lossy compression, as well as combinations thereof, can be used. Lossless compression is a technique in which an exact copy of the original signal can be reconstructed from the compressed original signal. When using lossy compression, the reconstructed signal may not be identical to the original signal, but the distortion between the original and reconstructed signals is small enough to make the reconstructed signal useful for its intended purpose. In the case of video, lossy compression is widely used. The amount of distortion that is acceptable depends on the application; for example, users of certain consumer streaming applications may tolerate higher distortion than users of television distribution applications. The achievable compression ratio can reflect the fact that higher acceptable / acceptable distortion can result in a higher compression ratio.

[0005] Video encoders and decoders can utilize techniques from several broad categories, including motion compensation, transformation, quantization, and entropy coding, some of which are described below.

[0006] Historically, video encoders and decoders have tended to operate on a given picture size, which in most cases remained constant and was defined for coded video sequences (CVS), group of pictures (GOP), or similar multi-picture timeframes. For example, in MPEG-2, the system design is known to vary the horizontal resolution (and thus the picture size) depending on factors such as scene activity, but this is only in the picture, and therefore typically for the GOP. Resampling of the reference picture to use different resolutions within a CVS is known, for example, from ITU-T Rec. H.263 Annex P. However, here the picture size does not change, only the reference picture is resampled, and as a result, potentially only a portion of the picture canvas is used (in the case of downsampling), or only a portion of the scene is captured (in the case of upsampling). Furthermore, H.263 Annex Q allows for resampling of individual macroblocks up or down by a factor of two (in each dimension). Here again, the picture size remains the same. The size of macroblocks is fixed in H.263, and therefore does not need to be transmitted via signals.

[0007] Changes in picture size within a predicted picture have become more prevalent in modern video coding. For example, VP9 allows for resampling and resolution changes of the reference picture for the entire picture. Similarly, certain proposals made for VVC (including, for example, Non-Patent Document 1, which is entirely incorporated herein) allow for resampling of the entire reference picture to different—higher or lower—resolutions. That document proposes that different candidate resolutions are encoded in a sequence parameter set and referenced by per-picture syntactic elements in a picture parameter set. [Non-Patent Document 1] Hendry, et. al, "On adaptive resolution change (ARC) for VVC", Joint Video Team document JVET-M0135-v1, Jan 9-19, 2019 [Overview of the project] [Means for solving the problem]

[0008] In one embodiment, a method is provided for generating an encoded video bitstream using at least one processor. The method includes: making a first determination of whether the current layer of the current picture is an independent layer; making a second determination of whether reference picture resampling is enabled for the current layer; disabling wrap-around compensation for the current layer based on the first and second determinations; and encoding the current layer without wrap-around compensation.

[0009] In one embodiment, an apparatus is provided for generating an encoded video bitstream, comprising at least one memory configured to store program code, and at least one processor configured to read the program code and operate as directed by the program code. The program code includes: a first decision code configured to cause the at least one processor to make a first decision about whether the current layer of the current picture is an independent layer; a second decision code configured to cause the at least one processor to make a second decision about whether reference picture resampling is enabled for the current layer; a disabling code configured to cause the at least one processor to disable wrap-around compensation for the current layer based on the first and second decisions; and an encoding code configured to cause the at least one processor to encode the current layer without wrap-around compensation.

[0010] In one embodiment, a non-temporary computer-readable medium storing instructions is provided. The instructions include, when executed by one or more processors of a device that generates an encoded video bitstream, one or more instructions causing the one or more processors to: make a first determination about whether the current layer of the current picture is an independent layer; make a second determination about whether reference picture resampling is enabled for the current layer; disable wrap-around compensation for the current layer based on the first and second determinations; and encode the current layer without wrap-around compensation. [Brief explanation of the drawing]

[0011] Further features, properties, and various advantages of the disclosed subject matter will become clearer from the detailed description and accompanying drawings below.

[0012] [Figure 1] This is a schematic diagram of a simplified block diagram of a communication system according to one embodiment.

[0013] [Figure 2] This is a schematic diagram of a simplified block diagram of a communication system according to one embodiment.

[0014] [Figure 3] This is a schematic diagram of a simplified block diagram of a decoder according to one embodiment.

[0015] [Figure 4] This is a schematic diagram of a simplified block diagram of an encoder according to one embodiment.

[0016] [Figure 5] Figures A through E are schematic diagrams of options for signaling ARC parameters according to one embodiment.

[0017] [Figure 6] A to B is a schematic diagram of an example of a syntax table according to an embodiment.

[0018] [Figure 7] It is a schematic diagram of an example of a syntax table according to an embodiment.

[0019] [Figure 8] A to C is a flowchart of an exemplary process for generating an encoded video bitstream according to an embodiment.

[0020] [Figure 9] It is a schematic diagram of a computer system according to an embodiment.

Mode for Carrying Out the Invention

[0021] FIG. 1 shows a simplified block diagram of a communication system (100) according to an embodiment of the present disclosure. The system (100) may include at least two terminals (110 to 120) interconnected via a network (150). For one-way data transmission, the first terminal (110) can encode video data at a local location for transmission to the other terminal (120) via the network (150). The second terminal (120) can receive the encoded video data of the other terminal from the network (150), decode the encoded data, and display the recovered video data. One-way data transmission may be common in media delivery applications and the like.

[0022] Figure 1 shows a second pair of terminals (130, 140) provided to support the bidirectional transmission of encoded video that may occur, for example, during a video conference. For bidirectional data transmission, each terminal (130, 140) can encode video data captured at its local location for transmission to the other terminal over the network (150). Each terminal (130, 140) can also receive encoded video data transmitted by the other terminal, decode the encoded data, and display the recovered video data on a local display device.

[0023] In Figure 1, terminals (110-140) may be illustrated as servers, personal computers, and smartphones, but the principles of this disclosure are not limited thereto. Embodiments of this disclosure find applications with laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. Network (150) represents any number of networks that transmit encoded video data between terminals (110-140), including, for example, wired and / or wireless communication networks. Communication network (150) can exchange data over circuit-switched and / or packet-switched channels. Typical networks include telecommunication networks, local area networks, wide area networks, and / or the Internet. For the purposes of this discussion, the architecture and topology of network (150) may not be important to the operation of this disclosure unless described below.

[0024] Figure 2 shows the arrangement of a video encoder and decoder in a streaming environment as an example of the application of the disclosed subject matter. The disclosed subject matter may be equally applicable to other video-enabled applications, such as video conferencing, digital TV, and the storage of compressed video on digital media including CDs, DVDs, and memory sticks.

[0025] The streaming system may include a video source (201), such as a digital camera, and may include a capture subsystem (213) that generates, for example, an uncompressed video sample stream (202). The sample stream (202) is drawn with a thick line to highlight its high data volume compared to an encoded video bitstream and can be processed by an encoder (203) coupled to the camera (201). The encoder (203) may include hardware, software, or a combination thereof to enable or realize aspects of the subject disclosed, as will be described in more detail below. An encoded video bitstream (204), drawn with a thin line to highlight its lower data volume compared to the sample stream, can be stored in a streaming server (205) for future use. One or more streaming clients (206, 208) can access the streaming server (205) and retrieve a copy (207, 209) of the encoded video bitstream (204). A client (206) may include a video decoder (210). The video decoder decodes an incoming copy of the encoded video bitstream (207) and generates an outgoing video sample stream (211) that can be rendered on a display (212) or other rendering device (not shown). In some streaming systems, the video bitstreams (204, 207, 209) may be encoded according to certain video coding / compression standards. Examples of these standards include ITU-T Recommendation H.265. Video coding standards informally known as multipurpose video coding or VVC are also under development. The subject matter disclosed may be used in the context of VVC.

[0026] Figure 3 may be a functional block diagram of a video decoder (210) according to one embodiment of the present disclosure.

[0027] The receiver (310) may receive one or more encoded video sequences to be decoded by the decoder (210); in the same or another embodiment, one encoded video sequence at a time, and the decoding of each encoded video sequence is independent of other encoded video sequences. The encoded video sequences may be received from a channel (312), which may be a hardware / software link to a storage device that stores the encoded video data. The receiver (310) may receive the encoded video data together with other data, such as encoded audio data and / or auxiliary data streams, which may be transmitted in their respective usage entities (not shown). The receiver (310) can isolate the encoded video sequences from other data. As a measure against network jitter, a buffer memory (315) may be coupled between the receiver (310) and the entropy decoder / parser (320) (hereinafter referred to as the "parser"). If the receiver (310) is receiving data from a storage / transmission device with sufficient bandwidth and controllability, or from an isochronous network, the buffer (315) may not be required or may be small. For use in best-effort packet networks such as the Internet, the buffer (315) may be required, may be relatively large, and may be advantageously adaptive in size.

[0028] The video decoder (210) may include a parser (320) for reconstructing symbols (321) from an entropy-encoded video sequence. These categories of symbols include information used to manage the operation of the decoder (210) and potentially information for controlling rendering devices, such as a display (212) that is not an integral part of the decoder but can be coupled to it, as shown in Figure 3. The control information for rendering devices (one or more) may be in the form of Supplementary Enhancement Information (SEI messages) or Video Usability Information (VUI) parameter set fragments (not shown). The parser (320) can parse / entropy-decode the received encoded video sequence. The encoding of the encoded video sequence may follow video coding techniques or standards, and may follow a variety of principles well known to those skilled in the art, including variable-length coding, Huffman coding, context-sensitive or non-context-sensitive arithmetic coding, etc. The parser (320) can extract from the encoded video sequence a set of subgroup parameters for at least one of the subgroups of pixels in the video decoder, based on at least one parameter corresponding to the group. Subgroups can include picture groups (GOP), pictures, subpictures, tiles, slices, macroblocks, coding tree units (CTU), coding units (CU), blocks, transform units (TU), prediction units (PU), and the like. A tile may represent a rectangular area of ​​CU / CTU within a particular tile column and row in a picture. A brick may represent a rectangular area of ​​CU / CTU rows within a particular tile. A slice may represent one or more bricks in a picture that are contained within a NAL unit.A sub-picture may represent a rectangular region of one or more slices within the picture. The entropy decoder / parser can also extract information such as transformation coefficients, quantizer parameter values, and motion vectors from the encoded video sequence.

[0029] The parser (320) can perform an entropy decode / parse operation on the video sequence received from the buffer (315), thereby generating a symbol (321).

[0030] The reconstruction of the symbol (321) can involve multiple different units, depending on the type of the encoded video picture or its parts (e.g., intra-block) and other factors. Which units are involved and how can be controlled by subgroup control information parsed from the encoded video sequence by the parser (320). The flow of such subgroup control information between the parser (320) and the multiple units described below is not depicted for clarity.

[0031] In addition to the functional blocks already described, the decoder 210 can be conceptually divided into several functional units, as described below. In a practical implementation operating under commercial constraints, many of these units can interact closely with each other and be at least partially integrated. However, for the purpose of describing the subject matter being disclosed, the conceptual subdivision into functional units described below is appropriate.

[0032] The first unit is the scaler / inverse unit (351). The scaler / inverse unit (351) receives quantized transformation coefficients and control information from the parser (320) as symbols (singular or plural) (321). The control information includes which transformation to use, block size, quantization factor, quantization scaling matrix, etc. The scaler / inverse unit can output a block containing sample values ​​that can be input to the aggregater (355).

[0033] In some cases, the output samples of the scaler / inverse transform (351) may relate to intra-encoded blocks, i.e., blocks that do not use predictive information from previously reconstructed pictures, but can use predictive information from previously reconstructed portions of the current picture. Such predictive information may be provided by an intra-picture predictive unit (352). In some cases, the intra-picture predictive unit (352) generates blocks of the same size and shape as the block being reconstructed, using surrounding already reconstructed information taken from the current (partially reconstructed) picture (358). The tallyer (355) may, for each sample, add the predictive information generated by the intra-predictive unit (352) to the output sample information provided by the scaler / inverse transform unit (351).

[0034] In other cases, the output samples of the scaler / inverse unit (351) may relate to intercoded and potentially motion-compensated blocks. In such cases, a motion-compensated prediction unit (353) may access a reference picture memory (357) to retrieve samples to be used for prediction. After motion-compensating the retrieved samples according to symbols (321) relating to the blocks, these samples can be added by an aggregater (355) to the output of the scaler / inverse unit (in this case, called residual samples or residual signals) to generate output sample information. The addresses in the reference picture memory from which the motion-compensated unit retrieves prediction samples can be controlled by motion vectors available to the motion-compensated unit in the form of symbols (321). These symbols may have, for example, X, Y, and reference picture components. Motion compensation may include interpolation of sample values ​​retrieved from reference picture memory when accurate motion vectors less than or equal to a sample are used, motion vector prediction mechanisms, etc.

[0035] The output samples of the tallyer (355) can undergo various loop filtering techniques within the loop filter unit (356). The video compression technique may include an in-loop filtering technique. The in-loop filtering technique is controlled by parameters contained in the encoded video bitstream and made available to the loop filter unit (356) as symbols (321) from the parser (320), but can also respond to metadata obtained during decoding of earlier parts (in decoding order) of the encoded picture or encoded video sequence, as well as to previously reconstructed and loop-filtered sample values.

[0036] The output of the loop filter unit (356) can be a sample stream, which can be output to the renderer (212) and can also be stored in reference picture memory for use in future interpicture predictions.

[0037] Once an encoded picture is fully reconstructed, it can be used as a reference picture for future predictions. For example, once an encoded picture is fully reconstructed and identified as a reference picture (e.g., by a parser (320)), the current reference picture (358) can become part of the reference picture buffer (357), and fresh current picture memory can be reallocated before starting the reconstruction of subsequent encoded pictures.

[0038] The video decoder (210) can perform decoding operations according to a given video compression technique, which may be documented in a standard such as ITU-T Recommendation H.265. The encoded video sequence may conform to the syntax specified by the video compression technique or standard used, in the sense that it conforms to the syntax of the video compression technique or standard specified in the video compression technique documentation or standard, particularly in the profile documentation therewith. Conformance may also require that the complexity of the encoded video sequence be within the range defined by the level of the video compression technique or standard. In some cases, the level may constrain the maximum picture size, maximum frame rate, maximum reconstruction sample rate (e.g., measured in megasamples per second), maximum reference picture size, etc. The limits set by the level may, in some cases, be further constrained through the Hypothetical Reference Decoder (HRD) specification and metadata for HRD buffer management, which are signaled in the encoded video sequence.

[0039] In one embodiment, the receiver (310) may receive additional (redundant) data along with the encoded video. The additional data may be included as part of the encoded video sequence(single or multiple). The additional data may be used by the video decoder (210) to properly decode the data and / or to more accurately reconstruct the original video data. The additional data may take the form of, for example, a temporal, spatial, or SNR enhancement layer, redundant slices, redundant pictures, forward error correction codes, etc.

[0040] Figure 4 may be a functional block diagram of a video encoder (203) according to one embodiment of the present disclosure.

[0041] The encoder (203) can receive video samples from a video source (201) (which is not part of the encoder) that can capture video images to be encoded by the encoder (203).

[0042] The video source (201) can provide a source video sequence to be encoded by the encoder (203) in the form of a digital video sample stream, which can be any preferred bit depth (e.g., 8-bit, 10-bit, 12-bit, ...), any color space (e.g., BT.601 YCrCB, RGB, ...), and any preferred sampling structure (e.g., YCrCb 4:2:0, YCrCb 4:4:4). In a media service system, the video source (201) may be a storage device that stores pre-prepared video. In a video conferencing system, the video source (203) may be a camera that captures local image information as a video sequence. The video data may be provided as a plurality of individual pictures that give motion when viewed in sequence. The picture itself may be organized as a spatial array of pixels, each pixel may contain one or more samples, depending on the sampling structure, color space, etc., in use. Those skilled in the art will readily understand the relationship between pixels and samples. The following description will focus on samples.

[0043] According to one embodiment, the encoder (203) can encode and compress the pictures of a source video sequence in real time or under any other temporal constraints required by the application to obtain an encoded video sequence (443). Enforcing an appropriate encoding rate is one function of the controller (450). The controller controls and is functionally coupled to other functional units, such as those described below. Such couplings are not depicted for clarity. Parameters set by the controller may include parameters related to rate control (picture skip, quantizer, lambda value of rate-distortion optimization technique, ...), picture size, picture group (GOP) layout, maximum motion vector search range, etc. Those skilled in the art will readily identify other functions of the controller (450) that may relate to a video encoder (203) optimized for certain system designs.

[0044] Some video encoders operate in what is readily recognizable to those skilled in the art as an "encoding loop." In a drastically simplified explanation, in one example, the encoding loop may consist of an encoding unit of an encoder (430) (hereinafter referred to as the "source encoder") (responsible for generating symbols based on the input picture to be encoded and a reference picture(s)) and a (local) decoder (433) embedded in the encoder (203). The decoder reconstructs the symbols to generate sample data that a (remote) decoder would also generate (in the video compression techniques considered in the disclosed subject, any compression between the symbols and the encoded video bitstream is lossless). The reconstructed sample stream is input to a reference picture memory (434). Since decoding the symbol stream yields bit-accurate results regardless of the decoder location (local or remote), the contents of the reference picture buffer are also bit-accurate between the local encoder and the remote encoder. In other words, the encoder's prediction unit "sees" the exact same sample values ​​as the reference picture samples that the decoder "sees" when using the prediction during decoding. This fundamental principle of reference picture synchronization (and the resulting drift, for example, when synchronization cannot be maintained due to channel errors) is well known to those skilled in the art.

[0045] The operation of the “local” decoder (433) may be the same as that of the “remote” decoder (210), which has already been described in detail above in relation to Figure 3. However, referring to Figure 4 for the moment, since symbols are available and the encoding / decoding of symbols to an encoded video sequence by the entropy encoder (445) and parser (320) may be reversible, the entropy decoding section of the decoder (210), which includes the channel (312), receiver (310), buffer (315), and parser (320), does not have to be fully implemented in the local decoder (433).

[0046] An observation that can be made at this point is that any decoder technique present within a decoder, other than parse / entropy decoding, must necessarily exist in substantially the same functional form within the corresponding encoder. For this reason, the subject matter disclosed will focus on decoder operation. The description of encoder techniques can be abbreviated, as it is the inverse of the comprehensively described decoder techniques. More detailed explanations are necessary only in certain areas, which are provided below.

[0047] As part of its operation, the source encoder (430) can perform motion-compensated predictive coding, predictively coding the input frame by referencing one or more previously coded frames from the video sequence, designated as “reference frames”. In this way, the coding engine (432) codes the difference between the pixel blocks of the input frame and the pixel blocks of the reference frame(s) that may be selected as predictive references for the input frame.

[0048] The local video decoder (433) can decode the encoded video data of a frame that may be designated as a reference frame, based on the symbols generated by the source encoder (430). The operation of the encoding engine (432) can, advantageously, be a lossy process. When the encoded video data can be decoded by the video decoder (not shown in Figure 4), the reconstructed video sequence can typically be a copy of the source video sequence with some errors. The local video decoder (433) can replicate the decoding process that the video decoder may perform on the reference frame and have the reconstructed reference frame stored in the reference picture cache (434). In this way, the encoder (203) can locally store a copy of the reconstructed reference frame that has common content (if there are no transmission errors) as the reconstructed reference frame that would be obtained by the far-end video decoder.

[0049] The predictor (435) can perform a predictive search on the encoding engine (432). That is, for a new frame to be encoded, the predictor (435) can search the reference picture memory (434) for sample data (as candidate reference pixel blocks) or certain metadata, such as reference picture motion vectors, block shapes, etc., that can act as appropriate predictive references for the new picture. The predictor (435) may operate on a sample block-by-pixel-block basis to find appropriate predictive references. Depending on the search results obtained by the predictor (435), the input picture may have predictive references drawn from multiple reference pictures stored in the reference picture memory (434).

[0050] The controller (450) may manage the encoding operation of the video encoder (430), including, for example, setting parameters and subgroup parameters used to encode video data.

[0051] The outputs of all the above functional units can undergo entropy coding in the entropy encoder (445). The entropy encoder converts the symbols generated by the various functional units into coded video sequences by lossless compression of the symbols according to techniques known to those skilled in the art, such as Huffman coding, variable-length coding, and arithmetic coding.

[0052] The transmitter (440) may buffer the encoded video sequence generated by the entropy encoder (445) and prepare it for transmission over the communication channel (460). The communication channel may be a hardware / software link to a storage device that stores the encoded video data. The transmitter (440) may merge the encoded video data from the video encoder (430) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (sources not shown).

[0053] The controller (450) may manage the operation of the encoder (203). During encoding, the controller (450) may assign a certain encoded picture type to each encoded picture. The encoded picture type may influence the encoding technique that can be applied to each picture. For example, a picture may often be assigned as one of the following frame types:

[0054] An intra-picture (I-picture) can be encoded and decoded without using other pictures in the sequence as a source of prediction. Some video codecs allow different types of intra-pictures, including, for example, Independent Decoder Refresh pictures. Those skilled in the art will recognize these variations of I-pictures, as well as their respective uses and characteristics.

[0055] A prediction picture (P-picture) may be encoded and decoded using intra-prediction or inter-prediction, which uses up to one motion vector and reference index to predict the sample values ​​for each block.

[0056] A bidirectional predictive picture (B-picture) may be encoded and decoded using intra-prediction or inter-prediction with up to two motion vectors and reference indices to predict the sample values ​​of each block. Similarly, a multi-predictive picture may use three or more reference pictures and associated metadata for the reconstruction of a single block.

[0057] A source picture is typically divided spatially into multiple sample blocks (for example, blocks of 4x4, 8x8, 4x8, or 16x16 samples each), and each block can be encoded. Blocks can be predictively encoded by referencing other (already encoded) blocks, as determined by the encoding assignment applied to each picture in the block. For example, a block of picture I may be non-predictively encoded, or it may be predictively encoded by referencing an already encoded block of the same picture (spatial prediction or intra-prediction). A pixel block of picture P may be predictively encoded by referencing one previously encoded reference picture via spatial prediction or temporal prediction. A block of picture B may be predictively encoded by referencing one or two previously encoded reference pictures via spatial prediction or temporal prediction.

[0058] The video encoder (203) can perform encoding operations in accordance with a specified video encoding technique or standard, such as ITU-T Recommendation H.265. In this operation, the video encoder (203) can perform various compression operations, including predictive encoding operations that leverage the temporal and spatial redundancy in the input video sequence. Thus, the encoded video data may conform to the syntax specified by the video encoding technique or standard used.

[0059] In one embodiment, the transmitter (440) may transmit additional data along with the encoded video. The video encoder (430) may include such data as part of the encoded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant pictures and slices, Supplemental Enhancement Information (SEI) messages, Visual Usability Information (VUI) parameter set fragments, and the like.

[0060] In recent years, the aggregation and extraction of multiple semantically independent picture segments into a single video picture has attracted some attention. Particularly in the context of 360 encoding or certain surveillance applications, multiple semantically independent source pictures (e.g., six cubic surfaces of a cubically projected 360 scene, or individual camera inputs in a multi-camera surveillance setup) may require separate adaptive resolution settings to address different scene-specific activities at a given point in time. In other words, an encoder can choose to use different resampling factors for different semantically independent pictures constituting the entire 360 ​​or surveillance scene at a given point in time. When combined into a single picture, this requires that reference picture resampling be performed and that adaptive resolution encoding signaling be available for the segments of the encoded picture.

[0061] Below are some terms that will be mentioned in the remainder of this paper.

[0062] A subpicture may, in some cases, refer to a rectangular arrangement of samples, blocks, macroblocks, coding units, or similar entities that are semantically grouped and can be independently coded at a modified resolution. One or more subpictures may form a picture. One or more coded subpictures may form a coded picture. One or more subpictures may be assembled into a single picture, or one or more subpictures may be extracted from a single picture. In certain environments, one or more coded subpictures may be assembled into a coded picture in a compressed region without transcoding to the sample level, and in the same or other cases, one or more coded subpictures may be extracted from a coded picture in a compressed region.

[0063] Adaptive Resolution Change (ARC) can refer to a mechanism that allows for changes in the resolution of a picture or sub-picture within an encoded video sequence, for example, through reference picture resampling. Hereinafter, ARC parameters refer to the control information required to perform Adaptive Resolution Change, and may include, for example, filter parameters, scaling factors, output and / or reference picture resolutions, and various control flags.

[0064] In some embodiments, encoding and decoding may be performed on a single semantically independent encoded video picture. Before describing the implications of encoding / decoding multiple subpictures with independent ARC parameters and the additional complexity that is implied, the options for signaling the ARC parameters are described.

[0065] Referring to Figures 5A to 5E, several embodiments for signaling ARC parameters are shown. As described for each embodiment, they may have certain advantages and disadvantages in terms of coding efficiency, complexity, and architecture. A video coding standard or technique may select one or more known options from these embodiments or related techniques for signaling ARC parameters. These embodiments may not be mutually exclusive and may be interchangeable based on application needs, relevant standards, or encoder selection.

[0066] The class of the ARC parameter may include the following:

[0067] • Separate or combined upsampling / downsampling factors in the X and Y dimensions.

[0068] This involves adding a time dimension, which represents a constant speed of zooming in / out on a given number of pictures, to the upsampling / downsampling factor.

[0069] Both of the above may include the encoding of one or more possibly short syntactic elements that can point to a table containing the aforementioned factors (one or more).

[0070] The X-dimensional or Y-dimensional resolution of the input picture, output picture, reference picture, and encoded picture, in units of samples, blocks, macroblocks, coding units (CUs), or any other preferred granularity. If there are multiple resolutions (e.g., one for the input picture and one for the reference picture), in certain cases one set of values ​​may be inferred from another set of values. This can be gated, for example, by the use of flags. See below for more detailed examples.

[0071] • As mentioned above, a “warping” coordinate system with a suitable granularity, similar to that used in Annex P of H.263. While Annex P of H.263 defines one efficient way to encode such warping coordinates, other, potentially more efficient methods may be devised. For example, the variable-length, reversible “Huffman” encoding of warping coordinates in Annex P can be replaced by a binary encoding of a suitable length, where the length of the binary codeword can be derived, for example, from the maximum picture size, possibly by multiplying the maximum picture size by a factor to allow for “warping” outside the boundaries of the maximum picture size, and offsetting by a certain value.

[0072] • Upsampling or downsampling filter parameters. In some embodiments, only a single filter for upsampling and / or downsampling may exist. However, in some embodiments, it is desirable to allow greater flexibility in the filter design, which may require signaling of filter parameters. Such parameters may be selected through an index in a list of possible filter designs, the filter may be fully specified (e.g., through a list of filter coefficients using a suitable entropy coding technique), or the filter may be implicitly selected through an upsampling / downsampling ratio signaled according to one of the mechanisms described above.

[0073] This paper assumes the coding of a finite set of upsampling / downsampling factors (the same factors are used in both the X and Y dimensions) represented by a codeword. The codeword may be variable-length coded, for example, using Ext-Golomb coding common to certain syntactic elements in video coding specifications such as H.264 and H.265. One preferred mapping of values ​​to upsampling / downsampling factors may be, for example, as shown in Table 1: [Table 1]

[0074] Many similar mappings can be devised according to the needs of the application and the capabilities of the upscaling and downscaling mechanisms available in the video compression technology or standard. This table can be extended to more values. The values ​​may be represented by an entropy coding mechanism other than Ext-Golomb coding, for example, using binary coding. This may have certain advantages when the resampling factor is of interest outside of the video processing engine (encoder and decoder first) itself, as in MANE, for example. For situations where resolution changes are not required, a short Ext-Golomb code can be chosen, and it should be noted that in the table above, this is only 1 bit. This may be more coding efficient than using binary coding in the most common cases.

[0075] The number of items in a table and their meanings may be fully or partially configurable. For example, the basic outline of the table may be communicated in a “higher-level” parameter set, such as a sequence or decoder parameter set. In some embodiments, one or more such tables may be defined in a video coding technique or standard, or selected, for example, through a decoder or sequence parameter set.

[0076] The following describes how the encoded upsampling / downsampling factors (ARC information), as described above, may be incorporated into video coding techniques or standard syntax. Similar considerations may apply to one or a few codewords controlling upsampling filters. For a discussion of cases where relatively large amounts of data are required for filters or other data structures, see below.

[0077] As shown in Figure 5A, H.263 Annex P includes ARC information (502) in the form of four distortion coordinates within the picture header (501), particularly in the H.263 PLUSPTYPE (503) header extension. This can be a reasonable design choice when a) there are available picture headers and b) frequent changes to the ARC information are expected. However, the overhead when using H.263-style signaling can be very high, and because picture headers may be transient in nature, scaling factors may not be relevant between picture boundaries.

[0078] As shown in Figure 5B, JVCET-M135-v1 includes ARC reference information (505) (index) located within the picture parameter set (504), which indexes a table (506) containing target resolutions located within the sequence parameter set (507). Including possible resolutions in table (506) within the sequence parameter set (507) can be justified, according to oral statements made by the authors, by using SPS as an interoperability point of compromise during capability exchange. Resolutions can vary per picture, within the limits set by the values ​​in table (506), by referencing the appropriate picture parameter set (504).

[0079] Referring to Figures 5C to 5E, the following embodiments may exist for transmitting ARC information in a video bitstream. Each of these options has certain advantages over the embodiments described above. The embodiments may coexist simultaneously in the same video encoding technique or standard.

[0080] In various embodiments, for example, the embodiment shown in Figure 5C, ARC information (509), such as a resampling (zoom) factor, may reside in a slice header, GOP header, tile header, or tile group header. Figure 5C shows an embodiment in which a tile group header (508) is used. This may be sufficient if the ARC information is small, for example, a single variable-length ue(v) or a fixed-length codeword of several bits, as shown above. Having the ARC information directly in the tile group header has the additional advantage that the ARC information may be applicable, for example, to subpictures represented by the tile group rather than the entire picture. See also below. Furthermore, even if the video compression technique or standard assumes only adaptive resolution changes of the entire picture (and not, for example, tile group-based adaptive resolution changes), putting the ARC information in the tile group header has certain advantages in terms of error tolerance compared to putting it in an H.263-style picture header.

[0081] In various embodiments, for example, in the embodiment shown in Figure 5D, the ARC information (512) itself may reside in an appropriate parameter set, such as a picture parameter set, a header parameter set, a tile parameter set, an adaptive parameter set, etc. Figure 5D shows an embodiment in which an adaptive parameter set (511) is used. The scope of the parameter set may, advantageously, not be larger than a picture, such as a tile group. The use of the ARC information is done implicitly through the activation of the relevant parameter set. For example, if the video coding technique or standard considers only picture-based ARC, then a picture parameter set or equivalent may be appropriate.

[0082] In embodiments, for example, in the embodiment shown in Figure 5E, the ARC reference information (513) may reside within a tile group header (514) or a similar data structure. The reference information (513) may refer to a subset of ARC information (515) available within a parameter set (516) that has a scope beyond a single picture, such as a sequence parameter set or a decoder parameter set.

[0083] The activation implied by additional levels of indirect referencing of the tile group header, PPS, and PPS from SPS used in JVET-M0135-v1 appears unnecessary. This is because picture parameter sets, like sequence parameter sets, can be used (and can be in certain standards such as RFC3984) for capability negotiation or announcements. However, if the ARC information should be applicable to subpictures that are also represented by tile groups, for example, then a parameter set with an activation scope limited to tile groups, such as an adaptive parameter set or header parameter set, might be a better choice. Also, if the ARC information includes filter control information that is not negligible, such as a large number of filter coefficients, then parameters may be a better choice from the standpoint of encoding efficiency than directly using the header (508), because their settings can be reused by future pictures or subpictures by referencing the same parameter set.

[0084] When using sequence parameter sets, or other higher-level parameter sets with a scope spanning multiple pictures, certain considerations may apply.

[0085] 1. The parameters set to store the ARC information table (516) may in some cases be a sequence parameter set, but in other cases, more favorably, be a decoder parameter set. The decoder parameter set may have an activation scope for multiple CVSs, i.e., encoded video streams, i.e., all encoded video bits from session start to session end. Possible ARC factors may also be decoder functions implemented in hardware, and such a scope may be more appropriate because hardware functions tend not to change with any CVS (at least in some entertainment systems, a CVS is a picture group of less than 1 second in length). Nevertheless, placing the table in a sequence parameter set is explicitly included in the arrangement options described herein, particularly in relation to point 2 below.

[0086] 2. The ARC reference information (513) may, advantageously, be placed directly in the picture / slice tile / GOP / tile group header, for example, the tile group header (514), rather than in the picture parameter set, as in JVCET-M0135-v1. For example, if the encoder wants to change a single value in the picture parameter set, such as the ARC reference information, it must create a new PPS and reference that new PPS. Assume that only the ARC reference information changes, and other information, such as the quantization matrix information in the PPS, does not. Such information can be quite large and would need to be retransmitted to complete the new PPS. Since the ARC reference information (513) may be a single codeword, such as an index to a table, and the only value that will change, retransmitting all of the quantization matrix information, for example, would be cumbersome and wasteful. Therefore, avoiding indirect referencing through the PPS, as proposed in JVET-M0135-v1, may be considerably better from the standpoint of coding efficiency. Similarly, including ARC reference information in the PPS has the additional drawback that the ARC information referenced by the ARC reference information (513) may apply to the entire picture rather than a sub-picture because the scope of the picture parameter set activation is the picture.

[0087] In the same or a different embodiment, the signaling of ARC parameters can follow the detailed examples outlined in Figures 6A–6B. Figures 6A–6B show syntax diagrams in representational forms using notation that closely follows C-style programming, as used, for example, in video coding standards since at least 1993. Thick lines indicate syntactic elements present in the bitstream, while non-thick lines often indicate control flow or variable settings.

[0088] As shown in Figure 6A, the tile group header (601) can conditionally contain a variable-length Exp-Golomb coded syntax element dec_pic_size_idx (602) (shown in bold) as an exemplary syntactic structure of a header applicable to a portion of a picture (potentially a rectangle). The presence of this syntax element in the tile group header can be gated using adaptive resolution (603)—a flag value not shown in bold here. This means that the flag is present in the bitstream at the point where it appears in the syntax diagram. Whether or not adaptive resolution is used for this picture or a portion of it can be signaled in any high-level syntactic structure inside or outside the bitstream. In the example shown, it is signaled in the sequence parameter set, as outlined below.

[0089] Referring to Figure 6B, an excerpt of the sequence parameter set (610) is also shown. The first syntactic element shown is adaptive_pic_resolution_change_flag (611). If true, the flag can indicate the use of adaptive resolution, which may require some kind of control information. In this example, such control information exists conditionally based on the value of the flag, based on the if() statement in the parameter set (612) and the tile group header (601).

[0090] When using adaptive resolution, in this example, the output resolution (613) in units of samples is encoded. The code 613 refers to both output_pic_width_in_luma_samples and output_pic_height_in_luma_samples, which together can define the resolution of the output picture. Some constraints on either value may be defined elsewhere in the video encoding technique or standard. For example, a level definition may limit the total number of output samples, which may be the product of the values ​​of these two syntactic elements. Also, certain video encoding techniques or standards, or external techniques or standards such as system standards, may restrict numbering ranges (e.g., one or both dimensions must be divisible by a power of 2) or aspect ratios (e.g., width and height must be in a relationship such as 4:3 or 16:9). Such constraints may be introduced to facilitate hardware implementation or for other reasons, and are well known in the art.

[0091] In certain applications, it may be desirable to instruct the encoder to use a reference picture size present in the decoder, rather than implicitly assuming that size is the output picture size. In this example, the syntax element reference_pic_size_present_flag(614) gates the conditional existence of the reference picture dimensions(615) (again, the numbers refer to both width and height).

[0092] Finally, a table of possible decoded picture widths and heights is shown. Such a table can be represented, for example, by the table directive (num_dec_pic_size_in_luma_samples_minus1)(616). "minus1" (minus 1) can refer to the interpretation of the value of that syntactic element. For example, if the encoded value is zero, there is one table entry, and if the value is 5, there are six table entries. For each "row" in the table, the width and height of the decoded picture are included in the syntax (617).

[0093] The presented table entry (617) can be indexed using the syntax element dec_pic_size_idx (602) in the tile group header, thereby allowing for different decoded sizes—effectively zoom factors—for each tile group.

[0094] In implementations of VVC-related technologies, a problem may arise where wrap-around motion compensation may not function correctly if the reference picture width differs from the current picture width. In some embodiments, wrap-around motion compensation may be disabled at the high-level syntax if the current picture layer is a dependent layer or if RPR is enabled for the current layer. In some embodiments, if the reference picture width differs from the current picture width, wrap-around processing may be disabled during the interpolation process for motion compensation.

[0095] Wrap-around motion compensation can be a useful feature for encoding 360 projection pictures, for example, those with equirectangular projection (ERP) format. This can reduce some visual artifacts at seams and improve encoding gain. In the current VVC specification draft JVET-P2001 (updated by JVET-Q0041 for editorial reasons), sps_ref_wraparound_offset_minus1 in SPS specifies the offset used to calculate the horizontal wrap-around position.

[0096] A problem can arise where the wrap-around offset value is determined in relation to the picture width. If the picture width of the reference picture differs from the current picture width, the wrap-around offset value should be changed proportionally to the scaling ratio between the current picture and the reference picture. However, in practice, adjusting the offset value according to the picture width of each reference picture can significantly increase the complexity of implementation and computation compared to the benefits of wrap-around motion compensation. Interlayer prediction and reference picture resampling (RPR) with different picture sizes can result in an astonishingly diverse range of different picture resolutions across layers and temporal pictures.

[0097] Embodiments can address this problem. For example, in embodiments, if the current picture layer is a dependent layer, or if RPR is enabled for the current layer, wrap-around motion compensation may be disabled by setting sps_ref_wraparound_enabled_flag to equal to 0. Thus, wrap-around motion compensation can only be used when the current layer is an independent layer and RPR is disabled. Under these conditions, the reference picture size is equal to the current picture size. Furthermore, in some embodiments, if the reference picture width is different from the current picture width, the wrap-around motion compensation process may be disabled during the interpolation process for motion compensation.

[0098] The embodiments may be used separately or in any order. Furthermore, each of the method (or embodiment), encoder, and decoder may be implemented by processing circuitry (e.g., one or more processors or one or more integrated circuits). In one example, the one or more processors execute a program stored on a non-temporary computer-readable medium.

[0099] Figure 7 shows exemplary syntax tables for various embodiments. In some embodiments, sps_ref_wraparound_enabled_flag(701) being equal to 1 may specify that horizontal wraparound motion compensation is applied to interpretation. sps_ref_wraparound_enabled_flag(701) being equal to 0 may specify that horizontal wraparound motion compensation is not applied. sps_ref_wraparound_enabled_flag(701) may be equal to 0 if the value of (CtbSizeY / MinCbSizeY+1) is less than or equal to (pic_width_in_luma_samples / MinCbSizeY-1), where pic_width_in_luma_samples is the value of pic_width_in_luma_samples in any PPS that references the SPS. If vps_independent_layer_flag[GeneralLayerIdx[nuh_layer_id]] is equal to 0, then the value of sps_ref_wraparound_enabled_flag(701) being equal to 0 may be a bitstream compatibility requirement. If it does not exist, the value of sps_ref_wraparound_enabled_flag(701) may be assumed to be equal to 0.

[0100] In some embodiments, refPicWidthInLumaSamples may be the pic_width_in_luma_samples of the current reference picture of the current picture. In some embodiments, if refPicWidthInLumaSamples is equal to the pic_width_in_luma_samples of the current picture, refWraparoundEnabledFlag may be set to equal to sps_ref_wraparound_enabled_flag. Otherwise, refWraparoundEnabledFlag may be set to equal to 0.

[0101] The rumor position (xInti, yInti) in a complete sample unit may be derived for i=0..1 as follows:

[0102] If subpic_treated_as_pic_flag[SubPicIdx] is equal to 1, the following may apply:

number

[0103] Otherwise (subpic_treated_as_pic_flag[subPicIdx] is equal to 0), the following may apply:

number

[0104] The lumer position (xInt, yInt) in a complete sample unit may be derived as follows:

[0105] If subpic_treated_as_pic_flag[SubPicIdx] is equal to 1, the following may apply:

number

[0106] Otherwise, the following may apply:

number

[0107] The predicted rumor sample value preSampleLXL may also be derived as follows: preSampleLXL=refPicLXL[xInt][yInt]< <shift3。

[0108] The chroma position (xInti, yInti) at the complete sample level may also be derived for i=0..3 as follows.

[0109] If subpic_treated_as_pic_flag[SubPicIdx] is equal to 1, the following may apply:

number

[0110] Otherwise (subpic_treated_as_pic_flag[subPicIdx] is equal to 0), the following may apply:

number

[0111] The chroma position (xInti, yInti) at the complete sample level may be further modified for i=0..3 as follows:

number

[0112] Figures 8A–8C are flowcharts of exemplary processes 800A, 800B, and 800C for generating an encoded video bitstream according to various embodiments. In various embodiments, any one of processes 800A, 800B, and 800C, or any part of processes 800A, 800B, and 800C, may be combined in any combination or permutation, in any desired order. In some implementations, one or more process blocks in Figures 8A–8C may be executed by the decoder 210. In some implementations, one or more process blocks in Figures 8A–8C may be executed by a separate device or group of devices, such as an encoder 203, which is separate from or includes the decoder 210.

[0113] As shown in Figure 8A, process 800A may include making a first decision about whether the current layer of the current picture is an independent layer (block 811).

[0114] As further shown in Figure 8A, process 800A may include making a second decision on whether reference picture resampling is enabled for the current layer (block 812).

[0115] As further shown in Figure 8A, process 800A may include disabling wrap-around compensation for the current picture based on the first and second decisions (block 813).

[0116] As further shown in Figure 8A, process 800A may also include encoding the current picture without wrap-around compensation (block 814).

[0117] In one embodiment, the first decision may be based on a first flag signaled in a first syntactic structure, and the second decision may be based on a second flag signaled in a second syntactic structure that is lower in rank than the first syntactic structure.

[0118] In one embodiment, the first flag may be signaled in the video parameter set, and the second flag may be signaled in the sequence parameter set.

[0119] In one embodiment, wrap-around compensation may be disabled based on the absence of a second flag in the sequence parameter set.

[0120] As shown in Figure 8B, process 800B may include determining whether the current layer of the current picture is an independent layer (block 821).

[0121] As further shown in Figure 8B, if it is determined that the current layer is not an independent layer (NO in block 821), process 800B may proceed to block 822, where wrap-around motion compensation may be disabled.

[0122] As further shown in Figure 8B, if the current layer is determined to be an independent layer (YES in block 821), process 800B may proceed to block 823.

[0123] As further shown in Figure 8B, process 800B may include determining whether reference picture resampling is enabled (block 823).

[0124] As further shown in Figure 8B, if it is determined that reference picture resampling is enabled (YES in block 823), process 800B may proceed to block 822, where wrap-around motion compensation may be disabled.

[0125] As further shown in Figure 8B, if it is determined that reference picture resampling is not enabled (NO in block 823), process 800B may proceed to block 824, where wrap-around motion compensation may be enabled.

[0126] As shown in Figure 8C, process 800C may include determining that the current layer of the current picture is an independent layer (block 831).

[0127] As further shown in Figure 8C, process 800C may include determining whether reference picture resampling is enabled (block 832).

[0128] As further shown in Figure 8C, process 800C may include determining whether the width of the current picture is different from the width of the current reference picture (block 833).

[0129] As further shown in Figure 8C, if it is determined that the width of the current picture is different from the width of the current reference picture (YES in block 833), process 800C may proceed to block 834, where wrap-around motion compensation may be disabled.

[0130] As further shown in Figure 8C, if it is determined that the width of the current picture is the same as the width of the current reference picture (NO in block 821), process 800C may proceed to block 835, where wrap-around motion compensation may be enabled.

[0131] In one embodiment, block 833 may be executed during the interpolation process for motion compensation.

[0132] Figures 8A–8C show exemplary blocks for processes 800A, 800B, and 800C, but in some implementations, process 800 may include additional blocks, fewer blocks, different blocks, or blocks in different arrangements compared to the blocks shown in Figures 8A–8C. Additionally or alternatively, two or more blocks of processes 800A, 800B, and 800C may be executed in parallel.

[0133] Furthermore, the proposed method may be implemented by a processing circuit (for example, one or more processors or one or more integrated circuits). In one example, the one or more processors execute a program stored on a non-temporary computer-readable medium in order to perform one or more of the proposed methods.

[0134] The techniques described above can be implemented as computer software using computer-readable instructions and can be physically stored on one or more computer-readable media. For example, Figure 9 shows a computer system 900 suitable for implementing certain embodiments of the disclosed subject matter.

[0135] Computer software can be coded using any suitable machine code or computer language, and can be subjected to assembly, compilation, linking, or similar mechanisms to create code containing instructions that can be executed directly by a computer's central processing unit (CPU), graphics processing unit (GPU), etc., or through interpretation, microcode execution, etc.

[0136] The instructions can be executed on various types of computers or their components, including, for example, personal computers, tablet computers, servers, smartphones, game consoles, and Internet of Things devices.

[0137] The components shown in Figure 9 for computer system 900 are illustrative and not intended to imply any limitation on the scope of use or functionality of computer software implementing embodiments of this disclosure. The configuration of the components should not be construed as having any dependence or requirement on any one or combination of components shown in the exemplary embodiments of computer system 900.

[0138] The computer system 900 may include certain types of human interface input devices. Such human interface input devices may respond to input from one or more human users through, for example, tactile input (e.g., keystrokes, swipes, data glove movements), voice input (e.g., voice, clapping), visual input (e.g., gestures), or olfactory input (not shown). The human interface devices may also be used to capture certain media that are not necessarily directly related to conscious human input, such as sound (e.g., speech, music, ambient sounds), images (e.g., scanned images, photographic images obtained from a still image camera), or video (e.g., 2D video, 3D video including stereoscopic video).

[0139] The input human interface device may include one or more of the following: keyboard 901, mouse 902, trackpad 903, touchscreen 910 and associated graphics adapter 950, data glove, joystick 905, microphone 906, scanner 907, and camera 908 (only one of each is depicted).

[0140] The computer system 900 may also include certain human interface output devices. Such human interface output devices may stimulate the senses of one or more human users, for example, through tactile output, sound, light, and smell / taste. Such human interface output devices may include tactile output devices (e.g., touchscreen 910, tactile feedback by data glove or joystick 905; however, there may also be tactile feedback devices that do not function as input devices), audio output devices (e.g., speaker 909, headphones (not shown)), visual output devices (e.g., screen 910 including cathode ray tube (CRT) screens, liquid crystal display (LCD) screens, plasma screens, organic light-emitting diode (OLED) screens; each may or may not have touchscreen input functionality, each may or may not have tactile feedback functionality, and some of them may be able to output higher than three dimensions through means such as two-dimensional visual output or stereoscopic output; virtual reality glasses (not shown), holographic displays and smoke tanks (not shown)), and printers (not shown).

[0141] The computer system 900 may also include human-accessible storage devices and associated media, such as optical media including CD / DVD ROM / RW 920 along with CD / DVD or similar media 921, a thumb drive 922, a removable hard drive or solid-state drive 923, legacy magnetic media such as tape and floppy disks (not shown), and specialized ROM / ASIC / PLD-based devices such as security dongles (not shown).

[0142] Those skilled in the art should also understand that the term “computer-readable medium” as used in relation to the subject matter currently disclosed does not include a transmission medium, carrier wave, or other transient signal.

[0143] The computer system 900 may also include interfaces to one or more communication networks (955). These networks may be, for example, wireless, wired, or optical. They may also be local, wide-area, metropolitan, automotive, and industrial, real-time, or latency-tolerant. Examples of networks include cellular networks such as Ethernet®, wireless LAN, Global Mobile Communication System (GSM), third-generation (3G), fourth-generation (4G), fifth-generation (5G), and Long-Term Evolution (LTE); wide-area digital networks for wired or wireless TV, including cable television, satellite television, and terrestrial television; and automotive and industrial networks including CANBus. Certain networks typically require an external network interface adapter (954) that is attached to some kind of general-purpose data port or peripheral bus (949) (for example, a Universal Serial Bus (USB) port on the computer system 900). Others are typically integrated into the core of the computer system 900 by mounting to a system bus, as described later (for example, an Ethernet interface to a PC computer system or a cellular network interface to a smartphone computer system). Using any of these networks, the computer system 900 can communicate with other entities. Such communication may be one-way, receive-only (e.g., broadcast television), one-way transmit-only (e.g., CANbus to certain CANbus devices), or bidirectional to other computer systems using local or wide-area digital networks, for example. Certain protocols and protocol stacks can be used on each of those networks and network interfaces (1154) as described above.

[0144] The aforementioned human interface device, human-accessible storage device, and network interface can be mounted on the core 940 of the computer system 900.

[0145] The core 940 may include one or more central processing units (CPUs) 941, graphics processing units (GPUs) 942, specialized programmable processing units in the form of field-programmable gate arrays (FPGAs) 943, hardware accelerators 944 for certain tasks, etc. These devices may be connected via a system bus 948, along with read-only memory (ROM) 945, random access memory (RAM) 946, internal mass storage devices such as hard drives and solid-state devices (SSDs) 947, etc. In some computer systems, the system bus 948 may be accessible in the form of one or more physical plugs to allow expansion with additional CPUs, GPUs, etc. Peripheral devices may be connected directly to the core's system bus 948 or via a peripheral bus 949. Architectures for peripheral buses include Peripheral Component Interconnect (PCI), USB, etc.

[0146] The CPU 941, GPU 942, FPGA 943, and accelerator 944 can execute certain instructions that, when combined, constitute the aforementioned computer code. This computer code can be stored in ROM 945 or RAM 946. Temporary data can also be stored in RAM 946, while persistent data can be stored, for example, in the internal mass storage device 947. High-speed storage and retrieval to any of the memory devices can be enabled through the use of cache memory that can be closely associated with one or more CPUs 941, GPUs 942, mass storage devices 947, ROM 945, RAM 946, etc.

[0147] A computer-readable medium may have computer code on it for performing various computer-implemented operations. The medium and computer code may be specifically designed and constructed for the purposes of this disclosure, or they may be of a type that is well known and available to those skilled in the computer software field.

[0148] As an example, and not an limitation, a computer system having architecture 900, specifically a core 940, can provide functionality as a result of a processor (including a CPU, GPU, FPGA, accelerator, etc.) executing software embodied in one or more tangible computer-readable media. Such computer-readable media can be user-accessible mass storage as described above, as well as media related to certain types of storage of the core 940 of a non-temporary nature, such as the mass storage device 947 or ROM 945 inside the core. Software implementing various embodiments of this disclosure can be stored in such devices and executed by the core 940. The computer-readable media can include one or more memory devices or chips, depending on the specific needs. The software can cause the core 940 and specifically the processors within it (including a CPU, GPU, FPGA, etc.) to execute certain processes or specific parts of certain processes described herein, including defining data structures stored in RAM 946 and modifying such data structures according to processes defined by the software. Additionally or alternatively, a computer system may provide functionality as a result of logic wired within or otherwise embodied within a circuit (e.g., accelerator 944), which may operate in place of, or in conjunction with, software for performing a particular process or a particular part of a particular process as described herein. References to software include logic, and vice versa, as appropriate. References to computer-readable media may, as appropriate, include circuitry storing software for execution (e.g., integrated circuits (ICs)), circuitry embodying logic for execution, or both. This disclosure encompasses any preferred combination of hardware and software.

[0149] While this disclosure has described several exemplary embodiments, there are many modifications, substitutions, and alternative equivalents that fall within the scope of this disclosure. Therefore, those skilled in the art will understand that many systems and methods can be devised that embody the principles of this disclosure and thus fall within the spirit and scope of this disclosure, even if they are not expressly shown or described herein.

Claims

[Claim 1] A method for generating an encoded video bitstream using at least one processor: This includes a step of encoding the current layer of the current picture with or without wrap-around compensation. Wrap-around compensation is disabled when the width of the current picture differs from the width of the current reference picture. Wrap-around compensation is enabled when the width of the current picture is the same as the width of the current reference picture. method.

Images