Wrap-around offset for reference picture resampling in video coding

Abstract

A video decoder can be configured to obtain, from a picture parameter set (PPS) data structure of video data, a value for a wrap-around offset, determine, in response to a determination that a block of the video data is encoded in an inter prediction mode, a motion vector for the block and a reference picture for the block, determine, based on the value for the wrap-around offset, a horizontal wrap-around position in the reference picture, and determine, based on the horizontal wrap-around position, a prediction block for the block.

Description

[0001] This application claims priority to U.S. Application No. 17 / 131,192, filed December 22, 2020, which claims the benefit of U.S. Provisional Patent Application No. 62 / 954,341, filed December 27, 2019, the entire contents of each of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates to video encoding and video decoding. Background Technology

[0003] Digital video capabilities can be incorporated into a wide variety of devices, including digital televisions, digital live broadcast systems, wireless broadcasting systems, personal digital assistants (PDAs), laptops or desktop computers, tablets, e-book readers, digital cameras, digital recording devices, digital media players, video game devices, video game consoles, cellular or satellite radio phones (so-called "smartphones"), video conferencing equipment, video streaming devices, and more. Digital video devices implement video decoding technologies, such as those described in standards defined by MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4 (Part 10, Advanced Video Decoding (AVC)), ITU-T H.265 / High Efficiency Video Decoding (HEVC), and extensions to such standards. By implementing such video decoding technologies, video devices can more efficiently transmit, receive, encode, decode, and / or store digital video information.

[0004] Video decoding techniques include spatial (intra-picture) prediction and / or temporal (inter-picture) prediction to reduce or remove redundancy inherent in video sequences. For block-based video decoding, video slices (e.g., video pictures or portions of video pictures) can be segmented into video blocks, which may also be referred to as decoding tree units (CTUs), decoding units (CUs), and / or decoding nodes. Video blocks in a slice of a picture that has been intra-decoded (I) are encoded using spatial prediction relative to reference samples in adjacent blocks within the same picture. Video blocks in a slice of a picture that has been inter-decoded (P or B) can use spatial prediction relative to reference samples in adjacent blocks within the same picture or temporal prediction relative to reference samples in other reference pictures. A picture may be referred to as a frame, and a reference picture may be referred to as a reference frame. Summary of the Invention

[0005] The technology disclosed herein relates to inter-frame prediction, and more specifically, to inter-frame prediction utilizing horizontal wrapping motion compensation. In some types of video (such as 360-degree video), a reference image may include filled regions at the left and right boundaries of the image. The filled regions are identical, thus reducing visual artifacts when transitioning from the right to the left of the image, and vice versa. In some decoding scenarios, the motion vector of the block being predicted may point outside the boundaries of the reference image. For example, the motion vector may point to a sample to the left of the leftmost boundary of the reference image. However, in such cases, the video decoder can still determine the predicted block based on the motion vector by calculating the horizontal wrapping position in the half of the reference (e.g., the right half of the reference image if the motion vector points to the left of the leftmost boundary of the reference image). As will be explained in more detail below, to determine the horizontal wrapping position, the video decoder receives values ​​for wrapping offsets in the bitstream. However, in some decoding scenarios, due to reference image resampling (RPR), the reference image may have a different size or resolution than the image being decoded. This can lead to surround offset in the case of existing implementations of horizontal surround motion compensation, causing the video decoder to identify incorrect prediction blocks in the reference image.

[0006] The techniques of this disclosure advantageously enable video decoding systems to utilize RPR with horizontal surround motion compensation in a manner that does not introduce surround offset, which causes the video decoder to misidentify prediction blocks in the reference image. Specifically, by obtaining the value for surround offset from the PPS data structure of the video data, a video decoder configured according to the techniques of this disclosure may be able to support both RPR and horizontal surround motion compensation simultaneously without introducing prediction errors due to the video decoder mislocating prediction blocks in the resampled reference image.

[0007] In one example, a method for decoding video data includes: obtaining a value for the wrap-around offset from a Picture Parameter Set (PPS) data structure for a current image of the video data; determining a motion vector for the block and a reference image for the block in response to determining that a block of the current image is encoded in an inter-frame prediction mode; determining a horizontal wrap-around position in the reference image based on the value for the wrap-around offset; and determining a predicted block for the block based on the horizontal wrap-around position.

[0008] In another example, an apparatus for decoding video data includes: a memory configured to store video data; and one or more processors implemented in circuitry and configured to: obtain a value for a wrap-around offset from a Picture Parameter Set (PPS) data structure for a current image of the video data; determine a motion vector for the block and a reference image for the block in response to determining that a block of the current image is encoded in an inter-frame prediction mode, wherein the block belongs to a slice referencing the PPS data structure; determine a horizontal wrap-around position in the reference image based on the value for the wrap-around offset; and determine a predicted block for the block based on the horizontal wrap-around position.

[0009] In another example, a computer-readable storage medium stores instructions that, when executed by one or more processors, cause one or more processors to: obtain a value for the wrap-around offset from a Picture Parameter Set (PPS) data structure for a current picture of video data; determine a motion vector for the block and a reference picture for the block in response to determining that a block of the current picture is encoded in an inter-frame prediction mode, wherein the block belongs to a slice referencing the PPS data structure; determine a horizontal wrap-around position in the reference picture based on the value for the wrap-around offset; and determine a predicted block for the block based on the horizontal wrap-around position.

[0010] In another example, an apparatus for decoding video data includes: a unit for obtaining a value for a wrap-around offset from a Picture Parameter Set (PPS) data structure for a current picture of the video data; a unit for determining a motion vector for a block and a reference picture for a block in response to determining that a block of the current picture is encoded in an inter-frame prediction mode; a unit for determining a horizontal wrap-around position in the reference picture based on the value for the wrap-around offset; and a unit for determining a predicted block for the block based on the horizontal wrap-around position.

[0011] In one example, a method for encoding video data includes: determining a value for a surround offset; in response to determining that a block of the current image is encoded in an inter-frame prediction mode, determining a motion vector for the block and a reference image for the block, wherein the block belongs to a slice referencing a PPS data structure; determining a horizontal surround position in the reference image based on the value for the surround offset; determining a predicted block for the block based on the horizontal surround position; including the value for the surround offset in a Picture Parameter Set (PPS) data structure for the current image of the video data; and outputting a bitstream of encoded video data including the PPS data structure.

[0012] In another example, an apparatus for encoding video data includes: a memory configured to store video data; and one or more processors implemented in circuitry and configured to: determine a value for a surround offset; determine a motion vector for the block and a reference picture for the block in response to determining that a block of the current picture is encoded in an inter-frame prediction mode; determine a horizontal surround position in the reference picture based on the value for the surround offset; determine a predicted block for the block based on the horizontal surround position; include the value for the surround offset in a Picture Parameter Set (PPS) data structure for the current picture of the video data; and output a bitstream of encoded video data including the PPS data structure.

[0013] In another example, a computer-readable storage medium stores instructions that, when executed by one or more processors, cause one or more processors to: determine a value for the wrap-around offset; in response to determining that a block of the current picture is encoded in an inter-frame prediction mode, determine a motion vector for the block and a reference picture for the block; determine a horizontal wrap-around position in the reference picture based on the value for the wrap-around offset; determine a predicted block for the block based on the horizontal wrap-around position; include the value for the wrap-around offset in a Picture Parameter Set (PPS) data structure for the current picture of the video data; and output a bitstream of encoded video data including the PPS data structure.

[0014] In another example, an apparatus for encoding video data includes: a unit for determining a value for a wrap-around offset; a unit for determining a motion vector for the block and a reference picture for the block in response to determining that a block of the current picture is encoded in an inter-frame prediction mode, wherein the block belongs to a slice referencing a PPS data structure; a unit for determining a horizontal wrap-around position in the reference picture based on the value for the wrap-around offset; a unit for determining a predicted block for the block based on the horizontal wrap-around position; a unit for including the value for the wrap-around offset in a Picture Parameter Set (PPS) data structure for the current picture of the video data; and a unit for outputting a bitstream of encoded video data including the PPS data structure.

[0015] Details of one or more examples are set forth in the accompanying drawings and the following description. Other features, objects, and advantages will be apparent from the description, drawings, and claims. Attached Figure Description

[0016] Figure 1 This is a block diagram illustrating an example video encoding and decoding system that can perform the techniques described in this disclosure.

[0017] Figure 2A and Figure 2BThis is a conceptual diagram showing an example quadtree binary tree (QTBT) structure and its corresponding decoding tree unit (CTU).

[0018] Figure 3 An example of wrap-around offset is shown in the reference image.

[0019] Figure 4 This shows the position of a reference pixel that was mistakenly moved to an unknown location.

[0020] Figure 5 This is a block diagram illustrating an example video encoder that can perform the techniques described in this disclosure.

[0021] Figure 6 This is a block diagram illustrating an example video decoder that can perform the techniques described in this disclosure.

[0022] Figure 7 This is a flowchart illustrating the process of encoding video data.

[0023] Figure 8 This is a flowchart illustrating the process of decoding video data.

[0024] Figure 9 This is a flowchart illustrating the process of encoding video data.

[0025] Figure 10 This is a flowchart illustrating the process of decoding video data. Detailed Implementation

[0026] Video decoding (e.g., video encoding and / or video decoding) typically involves predicting blocks of video data from already decoded blocks of video data in the same frame (e.g., intra-frame prediction) or from already decoded blocks of video data in different frames (e.g., inter-frame prediction). In some cases, the video encoder also computes residual data by comparing the predicted blocks with the original blocks. Therefore, the residual data represents the difference between the predicted and original blocks. To reduce the number of bits required to signal the residual data, the video encoder transforms and quantizes the residual data, and signals the transformed and quantized residual data in the encoded bitstream. Compression achieved through the transform and quantization process can be lossy, meaning that the transform and quantization process may introduce distortion into the decoded video data.

[0027] The video decoder decodes the residual data and adds it to the prediction blocks to produce reconstructed video blocks that match the original video blocks more closely than individual prediction blocks. Due to the loss introduced by transforming and quantizing the residual data, the first reconstructed block may have distortion or artifacts. A common type of artifact or distortion is called block artifacts, where the boundaries of the blocks used to decode the video data are visible.

[0028] To further improve the quality of the decoded video, the video decoder can perform one or more filtering operations on the reconstructed video blocks. Examples of these filtering operations include deblocking filtering, Sample Adaptive Offset (SAO) filtering, and Adaptive Loop Filtering (ALF). The parameters of these filtering operations can be determined by the video encoder and explicitly signaled in the encoded video bitstream, or they can be implicitly determined by the video decoder without needing to be explicitly signaled in the encoded video bitstream.

[0029] The technology disclosed herein relates to inter-frame prediction, and more specifically, to inter-frame prediction utilizing horizontal wrap-around motion compensation. In some types of video (such as 360-degree video), a reference image may include filled regions at the left and right boundaries of the image. These filled regions are identical, thus reducing visual artifacts when transitioning from the right to the left of the image, and vice versa. In some decoding scenarios, the motion vector of the block being predicted may point to samples outside the boundaries of the reference image. For example, the motion vector may point to samples to the left of the leftmost boundary of the reference image. In other decoding scenarios, the motion vector may point to interpolated samples within the boundaries of the reference image, but determined using an interpolation operation that uses samples outside the boundaries of the reference image. However, in such cases, the video decoder can still determine the predicted block based on the motion vector by calculating the horizontal wrap-around position in the half of the reference (e.g., the right half of the reference image if the motion vector points to the left of the leftmost boundary of the reference image). As will be explained in more detail below, in order to determine the horizontal wrap position, the video decoder receives a value for the wrap offset from the bitstream. However, in some decoding scenarios, due to RPR, the reference image may have a different size or resolution than the image being decoded. This can lead to a wrap offset in the existing implementation of horizontal wrap motion compensation, causing the video decoder to identify incorrect prediction blocks in the reference image.

[0030] The techniques of this disclosure advantageously enable video decoding systems to utilize RPR with horizontal surround motion compensation in a manner that does not result in surround offset, which causes the video decoder to misidentify prediction blocks in the reference image. Specifically, by obtaining values ​​for surround offset from the Picture Parameter Set (PPS) data structure of the video data, a video decoder configured according to the techniques of this disclosure may be able to support both RPR and horizontal surround motion compensation simultaneously without introducing prediction errors due to the video decoder mislocating prediction blocks in the resampled reference image.

[0031] Figure 1 This is a block diagram illustrating an example video encoding and decoding system 100 capable of performing the techniques of this disclosure. In general, the techniques of this disclosure relate to decoding (encoding and / or decoding) video data. Typically, video data includes any data used for processing video. Therefore, video data can include raw, unencoded video, encoded video, decoded (e.g., reconstructed) video, and video metadata (such as signaling data).

[0032] like Figure 1 As shown, in this example, system 100 includes source device 102, which provides encoded video data to be decoded and displayed by destination device 116. Specifically, source device 102 provides the video data to destination device 116 via computer-readable medium 110. Source device 102 and destination device 116 can include any of a wide variety of devices, including desktop computers, laptop computers, tablet computers, set-top boxes, mobile phones such as smartphones, televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some cases, source device 102 and destination device 116 may be equipped for wireless communication and may therefore be referred to as wireless communication devices.

[0033] exist Figure 1In the example, source device 102 includes a video source 104, memory 106, video encoder 200, and output interface 108. Destination device 116 includes an input interface 122, video decoder 300, memory 120, and display device 118. According to this disclosure, the video encoder 200 of source device 102 and the video decoder 300 of destination device 116 can be configured to apply techniques for signaling and processing surround offset. Therefore, source device 102 represents an example of a video encoding device, and destination device 116 represents an example of a video decoding device. In other examples, the source and destination devices may include other components or arrangements. For example, source device 102 may receive video data from an external video source such as an external camera. Similarly, destination device 116 may interface with an external display device, rather than including an integrated display device.

[0034] like Figure 1 The system 100 shown is merely an example. Typically, any digital video encoding and / or decoding device can perform techniques for signaling and processing surround offset. Source device 102 and destination device 116 are merely examples of such decoding devices, where source device 102 generates decoded video data for transmission to destination device 116. In this disclosure, "decoding device" refers to a device that performs the decoding (e.g., encoding and / or decoding) of data. Thus, video encoder 200 and video decoder 300 represent examples of decoding devices (specifically, video encoder and video decoder). In some examples, source device 102 and destination device 116 may operate in a substantially symmetrical manner, such that each of source device 102 and destination device 116 includes video encoding and decoding components. Therefore, system 100 can support one-way or two-way video transmission between source device 102 and destination device 116, for example, for video streaming, video playback, video broadcasting, or video telephony.

[0035] Typically, video source 104 represents a source of video data (i.e., raw, unencoded video data) and provides a series of consecutive pictures (also referred to as “frames”) of the video data to video encoder 200, which encodes the data used for the pictures. Video source 104 of source device 102 may include video capture devices such as cameras, video archives containing previously captured raw video, and / or video feed interfaces for receiving video from video content providers. Alternatively, video source 104 may generate computer graphics-based data as source video, or a combination of live video, archived video, and computer-generated video. In each case, video encoder 200 encodes the captured, pre-captured, or computer-generated video data. Video encoder 200 may rearrange the pictures from the received order (sometimes referred to as “display order”) to a decoding order for decoding. Video encoder 200 may generate a bitstream comprising the encoded video data. Then, the source device 102 can output encoded video data to a computer-readable medium 110 via the output interface 108 for reception and / or retrieval by, for example, the input interface 122 of the destination device 116.

[0036] The memory 106 of source device 102 and the memory 120 of destination device 116 represent general-purpose memory. In some examples, memories 106 and 120 may store raw video data, such as raw video from video source 104 and raw decoded video data from video decoder 300. Alternatively, memories 106 and 120 may store software instructions executable by, for example, video encoder 200 and video decoder 300. Although memories 106 and 120 are shown separately from video encoder 200 and video decoder 300 in this example, it should be understood that video encoder 200 and video decoder 300 may also include internal memory for functionally similar or equivalent purposes. Furthermore, memories 106 and 120 may store, for example, encoded video data output from video encoder 200 and input to video decoder 300. In some examples, portions of memories 106 and 120 may be allocated as one or more video buffers, for example, to store raw decoded and / or encoded video data.

[0037] Computer-readable medium 110 can represent any type of medium or device capable of transmitting encoded video data from source device 102 to destination device 116. In one example, computer-readable medium 110 represents a communication medium that enables source device 102 to directly transmit encoded video data to destination device 116 in real time, for example, via a radio frequency network or a computer-based network. According to a communication standard such as a wireless communication protocol, output interface 108 can modulate the transmitted signal including the encoded video data, and input interface 122 can demodulate the received transmitted signal. The communication medium can include any wireless or wired communication medium, such as radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium can form part of a packet-based network such as a local area network, a wide area network, or a global network such as the Internet. The communication medium can include a router, switch, base station, or any other device that may be useful for facilitating communication from source device 102 to destination device 116.

[0038] In some examples, source device 102 can output encoded data from output interface 108 to storage device 112. Similarly, destination device 116 can access encoded data from storage device 112 via input interface 122. Storage device 112 may include any of a variety of distributed or locally accessible data storage media, such as hard disk drives, Blu-ray discs, DVDs, CD-ROMs, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data.

[0039] In some examples, source device 102 may output encoded video data to file server 114 or another intermediate storage device that may store the encoded video generated by source device 102. Destination device 116 may access the stored video data from file server 114 via streaming or downloading. File server 114 may be any type of server device capable of storing and sending encoded video data to destination device 116. File server 114 may represent a web server (e.g., for a website), a file transfer protocol (FTP) server, a content delivery network device, or a network attached storage (NAS) device. Destination device 116 may access the encoded video data from file server 114 via any standard data connection (including an Internet connection). This may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a digital subscriber line (DSL), a cable modem, etc.), or a combination of both, suitable for accessing the encoded video data stored on file server 114. File server 114 and input interface 122 may be configured to operate according to streaming protocols, download protocols, or combinations thereof.

[0040] Output interface 108 and input interface 122 may represent a wireless transmitter / receiver, a modem, a wired networking component (e.g., an Ethernet card), a wireless communication component operating according to any of the various IEEE 802.11 standards, or other physical components. In examples where output interface 108 and input interface 122 include wireless components, output interface 108 and input interface 122 may be configured to transmit data (such as encoded video data) according to cellular communication standards (such as 4G, 4G-LTE (Long Term Evolution), improved LTE, 5G, etc.). In some examples where output interface 108 includes a wireless transmitter, output interface 108 and input interface 122 may be configured to operate according to other wireless standards (such as the IEEE 802.11 specification, the IEEE 802.15 specification (e.g., ZigBee)). TM Bluetooth TM The source device 102 and / or destination device 116 may include corresponding system-on-chip (SoC) devices. For example, source device 102 may include an SoC device for performing the functions assigned to video encoder 200 and / or output interface 108, and destination device 116 may include an SoC device for performing the functions assigned to video decoder 300 and / or input interface 122.

[0041] The technology disclosed herein can be applied to video decoding to support any of a variety of multimedia applications, such as over-the-air television broadcasting, cable television transmission, satellite television transmission, internet streaming video transmission (such as HTTP-based Dynamic Adaptive Streaming (DASH)), digital video encoded onto data storage media, decoding of digital video stored on data storage media, or other applications.

[0042] The input interface 122 of the destination device 116 receives an encoded video bitstream from a computer-readable medium 110 (e.g., a communication medium, storage device 112, file server 114, etc.). The encoded video bitstream may include signaling information defined by the video encoder 200, such as syntax elements (which are also used by the video decoder 300), which have values ​​describing the characteristics and / or processing of video blocks or other decoding units (e.g., slices, pictures, picture groups, sequences, etc.). The display device 118 displays a decoded picture of the decoded video data to the user. The display device 118 may represent any of a variety of display devices, such as a cathode ray tube (CRT), liquid crystal display (LCD), plasma display, organic light-emitting diode (OLED) display, or another type of display device.

[0043] Despite Figure 1 Not shown, but in some examples, the video encoder 200 and video decoder 300 may each be integrated with the audio encoder and / or audio decoder, and may include appropriate MUX-DEMUX units or other hardware and / or software to process multiplexed streams including both audio and video in a common data stream. Where applicable, the MUX-DEMUX unit may comply with the ITU H.223 multiplexer protocol or other protocols (such as User Datagram Protocol (UDP)).

[0044] The video encoder 200 and video decoder 300 can each be implemented as any of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When the technology is implemented in part in software, the device may store instructions for the software in a suitable non-transitory computer-readable medium and use one or more processors to execute the instructions in hardware to perform the technology of this disclosure. Each of the video encoder 200 and video decoder 300 may be included in one or more encoders or decoders, and either encoder or decoder may be integrated as part of a combined encoder / decoder (CODEC) in the respective device. Devices including the video encoder 200 and / or video decoder 300 may include integrated circuits, microprocessors, and / or wireless communication devices (such as cellular phones).

[0045] The video encoder 200 and video decoder 300 can operate according to video decoding standards such as ITU-T H.265 (also known as High Efficiency Video Coding (HEVC)) or extensions thereof such as Multi-View and / or Scalable Video Coding Extensions). Alternatively, the video encoder 200 and video decoder 300 can operate according to other proprietary or industry standards such as the Joint Exploratory Test Model (JEM) or ITU-T H.266 (also known as Universal Video Coding (VVC)). The latest draft of the VVC standard is described in the following document: Bross et al., “Versatile Video Coding (Draft 7)”, Joint Video Experts Group (JVET) of ITU-T SG16WP 3 and ISO / IEC JTC 1 / SC 29 / WG11, 16th meeting: Geneva, Switzerland, 1-11 October 2019, JVET-P2001-v14 (hereinafter referred to as “VVC Draft 7”). However, the technology of this disclosure is not limited to any particular decoding standard.

[0046] Typically, video encoder 200 and video decoder 300 can perform block-based decoding of images. The term "block" generally refers to a structure that includes data to be processed (e.g., encoded, decoded, or otherwise used during encoding and / or decoding). For example, a block may include a two-dimensional matrix of samples of luminance and / or chrominance data. Typically, video encoder 200 and video decoder 300 can decode video data represented in YUV (e.g., Y, Cb, Cr) format. That is, video encoder 200 and video decoder 300 can decode luminance and chrominance components, rather than decoding red, green, and blue (RGB) data samples used for images, where chrominance components may include both red hue and blue hue chrominance components. In some examples, video encoder 200 converts the received RGB-formatted data to a YUV representation before encoding, and video decoder 300 converts the YUV representation to RGB format. Alternatively, preprocessing and post-processing units (not shown) can perform these conversions.

[0047] In summary, this disclosure may relate to the decoding (e.g., encoding and decoding) of images to include the process of encoding or decoding the data of an image. Similarly, this disclosure may relate to the decoding of blocks of an image to include the process of encoding or decoding the data used for the blocks (e.g., prediction and / or residual decoding). Encoded video bitstreams typically include a series of values ​​for representing decoding decisions (e.g., decoding modes) and syntax elements that segment the image into blocks. Therefore, references to decoding images or blocks should generally be understood as decoding the values ​​of the syntax elements used to form images or blocks.

[0048] HEVC defines various blocks, including decoding units (CUs), prediction units (PUs), and transform units (TUs). According to HEVC, a video decoder (such as a video encoder 200) partitions the decoding tree unit (CTU) into CUs based on a quadtree structure. That is, the video decoder partitions the CTU and CU into four equal, non-overlapping squares, and each node of the quadtree has zero or four child nodes. Nodes without child nodes can be called "leaf nodes," and the CU of such leaf nodes can include one or more PUs and / or one or more TUs. The video decoder can further partition the PUs and TUs. For example, in HEVC, a residual quadtree (RQT) represents a partition of a TU. In HEVC, PUs represent inter-frame prediction data, while TUs represent residual data. CUs with intra-frame prediction include intra-frame prediction information, such as intra-frame mode indication.

[0049] As another example, the video encoder 200 and video decoder 300 can be configured to operate according to JEM or VVC. According to JEM or VVC, the video decoder (such as the video encoder 200) segments the image into multiple decoding tree units (CTUs). The video encoder 200 can segment the CTUs according to a tree structure (such as a quadtree-binary tree (QTBT) structure or a multi-type tree (MTT) structure). The QTBT structure eliminates the concept of multiple segmentation types, such as the separation between CUs, PUs, and TUs in HEVC. The QTBT structure includes two levels: a first level segmented according to quadtree segmentation and a second level segmented according to binary tree segmentation. The root node of the QTBT structure corresponds to the CTU. The leaf nodes of the binary tree correspond to the decoding units (CUs).

[0050] In the MTT partitioning structure, blocks can be partitioned using quadtree (QT) partitioning, binary tree (BT) partitioning, and one or more types of ternary tree (TT) partitioning (also known as triplet tree (TT)) partitioning. A ternary tree or triplet tree partitioning is a partition in which a block is divided into three sub-blocks. In some examples, a ternary tree or triplet tree partitioning divides a block into three sub-blocks without partitioning the original block through a center. The partitioning types in MTT (e.g., QT, BT, and TT) can be symmetric or asymmetric.

[0051] In some examples, the video encoder 200 and the video decoder 300 may use a single QTBT or MTT structure to represent each of the luma and chroma components, while in other examples, the video encoder 200 and the video decoder 300 may use two or more QTBT or MTT structures, such as one QTBT / MTT structure for the luma component and another QTBT / MTT structure for the two chroma components (or two QTBT / MTT structures for the respective chroma components).

[0052] The video encoder 200 and video decoder 300 can be configured to use per-HEVC quadtree segmentation, QTBT segmentation, MTT segmentation, or other segmentation structures. For illustrative purposes, a description of the techniques of this disclosure is given with respect to QTBT segmentation. However, it should be understood that the techniques of this disclosure can also be applied to video decoders configured to use quadtree segmentation or other types of segmentation.

[0053] Blocks (e.g., CTUs or CUs) can be grouped in various ways within an image. As an example, a brick can refer to a rectangular area of ​​a row of CTUs within a specific tile in an image. A tile can be a rectangular area of ​​a CTU within a specific tile column or a specific tile row in an image. A tile column refers to a rectangular area of ​​a CTU with a height equal to the height of the image and a width specified by a syntax element (e.g., as in an image parameter set). A tile row refers to a rectangular area of ​​a CTU with a height specified by a syntax element (e.g., as in an image parameter set) and a width equal to the width of the image.

[0054] In some examples, a tile may be divided into multiple brick-shaped zones, each of which may include one or more CTU rows within the tile. A tile that is not divided into multiple brick-shaped zones may still be referred to as a brick-shaped zone. However, a brick-shaped zone that is a true subset of a tile may not be referred to as a tile.

[0055] The brick-shaped regions in an image can also be arranged as slices. A slice can be an integer number of brick-shaped regions in the image, which can be uniquely contained within a single Network Abstraction Layer (NAL) unit. In some examples, a slice includes multiple complete tiles or a continuous sequence of complete brick-shaped regions consisting of only one tile.

[0056] This disclosure uses "NxN" and "N by N" interchangeably to refer to the sample size of a block (such as a CU or other video block) in the vertical and horizontal dimensions, for example, 16x16 samples or 16 by 16 samples. Typically, a 16x16 CU will have 16 samples in the vertical direction (y = 16) and 16 samples in the horizontal direction (x = 16). Similarly, an NxNCU typically has N samples in the vertical direction and N samples in the horizontal direction, where N represents a non-negative integer value. Samples in a CU can be arranged in rows and columns. Furthermore, a CU does not necessarily need to have the same number of samples in the horizontal direction as it does in the vertical direction. For example, a CU can include NxM samples, where M is not necessarily equal to N.

[0057] The video encoder 200 encodes video data for use in predicting and / or residual information, as well as other information, for the CU. The prediction information indicates how the CU will be predicted to form a prediction block for the CU. The residual information typically represents the sample-by-sample difference between a sample of the CU before encoding and the prediction block.

[0058] To predict the Cubic Frame (CU), the video encoder 200 typically forms prediction blocks for the CU using either inter-frame prediction or intra-frame prediction. Inter-frame prediction generally refers to predicting the CU based on data from previously decoded images, while intra-frame prediction generally refers to predicting the CU based on data from previously decoded images of the same frame. To perform inter-frame prediction, the video encoder 200 can generate prediction blocks using one or more motion vectors. The video encoder 200 can typically perform motion search to identify, for example, reference blocks that closely match the CU in terms of the difference between the CU and a reference block. The video encoder 200 can calculate difference metrics using sum of absolute differences (SAD), sum of squared differences (SSD), mean absolute difference (MAD), mean squared difference (MSD), or other such difference calculations to determine whether the reference block closely matches the current CU. In some examples, the video encoder 200 can use unidirectional or bidirectional prediction to predict the current CU.

[0059] Some examples of JEM and VVC also provide affine motion compensation modes, which can be considered as inter-frame prediction modes. In affine motion compensation mode, the video encoder 200 can determine two or more motion vectors representing non-translational motion (such as zooming in or out, rotation, perspective motion, or other irregular types of motion).

[0060] To perform intra-frame prediction, the video encoder 200 can select an intra-frame prediction mode to generate prediction blocks. Some examples of JEM and VVC provide sixty-seven intra-frame prediction modes, including various directional modes, as well as planar and DC modes. Typically, the video encoder 200 selects an intra-frame prediction mode that describes the samples of the current block (e.g., a block of a CU) to be predicted based on, which are the neighboring samples of the current block. Assuming that the video encoder 200 decodes the CTU and CU in raster scan order (from left to right, from top to bottom), such samples can typically be located above, to the upper left, or to the left of the current block within the same image.

[0061] The video encoder 200 encodes data representing the prediction mode used for the current block. For example, for inter-frame prediction modes, the video encoder 200 may encode data indicating which of the various available inter-frame prediction modes is used, as well as motion information for the corresponding mode. For unidirectional or bidirectional inter-frame prediction, for example, the video encoder 200 may use Advanced Motion Vector Prediction (AMVP) or merging modes to encode motion vectors. The video encoder 200 may use similar modes to encode motion vectors used for affine motion compensation modes.

[0062] Following a prediction, such as intra-frame or inter-frame prediction of a block, the video encoder 200 can compute residual data for that block. The residual data (such as a residual block) represents the sample-by-sample difference between the block and the prediction block used to form the block using the corresponding prediction mode. The video encoder 200 can apply one or more transforms to the residual block to produce transformed data in the transform domain rather than the sample domain. For example, the video encoder 200 can apply a Discrete Cosine Transform (DCT), an integer transform, a wavelet transform, or a conceptually similar transform to the residual video data. Additionally, the video encoder 200 can apply a second transform after the first transform, such as a Mode-dependent Inseparable Quadratic Transform (MDNSST), a Signal-dependent Transform, a Karhunen-Loeve Transform (KLT), etc. The video encoder 200 produces transform coefficients after applying one or more transforms.

[0063] As described above, after any transformation to produce transform coefficients, the video encoder 200 can perform quantization of the transform coefficients. Quantization generally refers to the process in which the transform coefficients are quantized to potentially reduce the amount of data used to represent them, thereby providing further compression. By performing the quantization process, the video encoder 200 can reduce the bit depth associated with some or all of the transform coefficients. For example, the video encoder 200 can round an n-bit value down to an m-bit value during quantization, where n is greater than m. In some examples, to perform quantization, the video encoder 200 can perform a bitwise right shift of the values ​​to be quantized.

[0064] After quantization, the video encoder 200 can scan the transform coefficients to generate a one-dimensional vector from a two-dimensional matrix including the quantized transform coefficients. The scan can be designed to place higher-energy (and therefore lower-frequency) transform coefficients before the vector and lower-energy (and therefore higher-frequency) transform coefficients after the vector. In some examples, the video encoder 200 can utilize a predefined scan order to scan the quantized transform coefficients to produce a serialized vector, and then entropy-encode the quantized transform coefficients of the vector. In other examples, the video encoder 200 can perform an adaptive scan. After scanning the quantized transform coefficients to form a one-dimensional vector, the video encoder 200 can entropy-encode the one-dimensional vector, for example, according to context-adaptive binary arithmetic decoding (CABAC). The video encoder 200 can also entropy-encode the values ​​of syntax elements used to describe metadata associated with the encoded video data for use by the video decoder 300 when decoding the video data.

[0065] To perform CABAC, the video encoder 200 can assign context within a context model to the symbols to be transmitted. The context may involve, for example, whether the neighboring values ​​of a symbol are zero. Probability determination can be based on the context assigned to the symbol.

[0066] The video encoder 200 can also generate, for example, syntax data (such as block-based syntax data, image-based syntax data, and sequence-based syntax data) or other syntax data (such as Adaptive Parameter Set (APS), Image Header (PH), Sequence Parameter Set (SPS), PPS, or Video Parameter Set (VPS)) destined for the video decoder 300 in the image header, block header, and slice header. An APS typically refers to a syntax structure containing syntax elements applied to zero or more slices, as determined by zero or more syntax elements found in the slice header. A PH refers to a syntax structure containing syntax elements applied to all slices of the decoded image. A PPS refers to a syntax structure containing syntax elements applied to zero or more entire decoded images, as determined by syntax elements found in each slice header. An SPS refers to a syntax structure containing syntax elements applied to zero or more entire CVSs, as determined by the content of syntax elements found in the PPS referenced by syntax elements found in each slice header. Syntax structure typically refers to zero or more syntax elements that exist together in a bitstream in a specified order.

[0067] Similarly, the video decoder 300 can decode such syntax data to determine how to decode the corresponding video data. In this way, the video encoder 200 can generate a bitstream that includes encoded video data, such as syntax elements describing the segmentation of an image into blocks (e.g., CUs) and prediction and / or residual information for those blocks. Finally, the video decoder 300 can receive the bitstream and decode the encoded video data.

[0068] Typically, the video decoder 300 performs a process reciprocal to that performed by the video encoder 200 to decode the encoded video data of the bitstream. For example, the video decoder 300 may use CABAC to decode the values ​​of syntax elements used for the bitstream in a manner substantially similar to, but reciprocal to, the CABAC encoding process of the video encoder 200. Syntax elements may define the segmentation information for dividing a picture into CTUs, and for segmenting each CTU according to a corresponding segmentation structure (such as a QTBT structure) to define the segmentation information of the CUs of the CTU. Syntax elements may also define prediction and residual information for blocks (e.g., CUs) of the video data.

[0069] The residual information can be represented, for example, by quantized transform coefficients. The video decoder 300 can inversely quantize and inversely transform the quantized transform coefficients of the block to regenerate a residual block for that block. The video decoder 300 uses a signal-informed prediction mode (intra-frame prediction or inter-frame prediction) and associated prediction information (e.g., motion information for inter-frame prediction) to form a prediction block for that block. The video decoder 300 can then combine the prediction block and the residual block (on a sample-by-sample basis) to regenerate the original block. The video decoder 300 can perform additional processing, such as performing a deblocking process to reduce visual artifacts along the block boundaries.

[0070] In summary, this disclosure may involve "signaling" certain information (such as syntax elements). The term "signaling" can generally refer to the transmission of values ​​for syntax elements and / or other data for decoding encoded video data. That is, video encoder 200 can signal values ​​for syntax elements in the bitstream. Generally, signaling refers to generating values ​​in the bitstream. As described above, source device 102 can transmit the bitstream to destination device 116 substantially in real time or not in real time (such as when syntax elements are stored in storage device 112 for later retrieval by destination device 116).

[0071] Figure 2A and Figure 2B This is a conceptual diagram illustrating an example Quadtree Binary Tree (QTBT) structure 130 and its corresponding Decoding Tree Unit (CTU) 132. Solid lines represent quadtree splits, while dashed lines indicate binary tree splits. In each split (i.e., non-leaf) node of the binary tree, a flag is signaled to indicate which split type (i.e., horizontal or vertical) is used, where, in this example, 0 indicates a horizontal split and 1 indicates a vertical split. For quadtree splits, since the quadtree node splits the block horizontally and vertically into four sub-blocks of equal size, there is no need to indicate the split type. Therefore, the video encoder 200 can encode the following, and the video decoder 300 can decode the following: syntax elements (such as split information) for the region tree level (i.e., solid lines) of the QTBT structure 130, and syntax elements (such as split information) for the prediction tree level (i.e., dashed lines) of the QTBT structure 130. The video encoder 200 can encode video data (such as prediction and transform data) for a CU represented by the terminal leaf nodes of the QTBT structure 130, while the video decoder 300 can decode the video data.

[0072] generally, Figure 2BThe CTU 132 can be associated with parameters that define the size of the blocks corresponding to the nodes at the first and second levels of the QTBT structure 130. These parameters can include the CTU size (representing the size of the CTU 132 in the sample), the minimum quadtree size (MinQTSize, which represents the minimum allowed size of the leaf nodes of the quadtree), the maximum binary tree size (MaxBTSize, which represents the maximum allowed size of the root node of the binary tree), the maximum binary tree depth (MaxBTDepth, which represents the maximum allowed depth of the binary tree), and the minimum binary tree size (MinBTSize, which represents the minimum allowed size of the leaf nodes of the binary tree).

[0073] The root node corresponding to the CTU in a QTBT structure can have four child nodes at the first level of the QTBT structure, where each child node can be partitioned according to a quadtree. That is, the nodes at the first level are leaf nodes (without child nodes) or have four child nodes. An example of QTBT structure 130 represents such a node as including a parent node and child nodes with solid-line branches. If the nodes at the first level are not larger than the maximum allowed binary tree root node size (MaxBTSize), these nodes can be further partitioned by the corresponding binary tree. The binary tree split of a node can be iterated until the nodes resulting from the split reach the minimum allowed binary tree leaf node size (MinBTSize) or the maximum allowed binary tree depth (MaxBTDepth). An example of QTBT structure 130 represents such a node as having dashed-line branches. The binary tree leaf nodes are called decoding units (CUs), which are used for prediction (e.g., intra-image or inter-image prediction) and transformation without any further partitioning. As discussed above, CUs can also be referred to as “video chunks” or “blocks”.

[0074] In one example of a QTBT segmentation structure, the CTU size is set to 128x128 (luminance samples and two corresponding 64x64 chrominance samples), MinQTSize is set to 16x16, MaxBTSize is set to 64x64, MinBTSize (for both width and height) is set to 4, and MaxBTDepth is set to 4. First, a quadtree segmentation is applied to the CTU to generate quadtree leaf nodes. Quadtree leaf nodes can have sizes ranging from 16x16 (i.e., MinQTSize) to 128x128 (i.e., the CTU size). If a leaf quadtree node is 128x128, it will not be further split by the binary tree because this size exceeds MaxBTSize (i.e., 64x64 in this example). Otherwise, the leaf quadtree node will be further split by the binary tree. Therefore, the quadtree leaf node also serves as the root node of the binary tree and has a binary tree depth of 0. When the depth of the binary tree reaches MaxBTDepth (4 in this example), further splitting is not allowed. A binary tree node with a width equal to MinBTSize (4 in this example) means that further horizontal splitting is not allowed. Similarly, a binary tree node with a height equal to MinBTSize means that further vertical splitting is not allowed for that binary tree node. As mentioned above, the leaf nodes of the binary tree are called CUs and are further processed according to prediction and transformation without further splitting.

[0075] This disclosure describes potential modifications to portions of VVC Draft 7 that could enable RPR to use horizontal orbital motion compensation. This document outlines some sections of VVC Draft 7 that may require improvement. Clause 7.3.2.3 of VVC Draft 7, version 14, is as follows:

[0076]

[0077] Clause 7.4.3.3 of VVC Draft Version 7, Version 14 contains the following:

[0078] `sps_ref_wraparound_enabled_flag` equal to 1 specifies that horizontal wraparound motion compensation is applied in inter-frame prediction. `sps_ref_wraparound_enabled_flag` equal to 0 specifies that horizontal wraparound motion compensation is not applied. When the value of (CtbSizeY / MinCbSizeY+1) is greater than or equal to (pic_width_in_luma_samples / MinCbSizeY-1), where pic_width_in_luma_samples is the value of pic_width_in_luma_samples in any PPS referencing SPS, the value of `sps_ref_wraparound_enabled_flag` should be equal to 0.

[0079] The increment of 1 in `sps_ref_wraparound_offset_minus1` specifies the offset used to calculate the horizontal wraparound position (in MinCbSizeY luminance samples). The value of `ref_wraparound_offset_minus1` should be in the range of (CtbSizeY / MinCbSizeY)+1 to (pic_width_in_luma_samples / MinCbSizeY)-1 (inclusive), where `pic_width_in_luma_samples` is the value of `pic_width_in_luma_samples` in any PPS referencing SPS.

[0080] Clause 7.3.2.4 of VVC Draft Version 7, Version 14 contains the following:

[0081]

[0082] Clause 7.4.3.4 of VVC Draft Version 7, Version 14 contains the following:

[0083] `pic_width_in_luma_samples` specifies the width (in luminance samples) of each decoded image referenced by PPS. `pic_width_in_luma_samples` should not be equal to 0, should be an integer multiple of `Max(8, MinCbSizeY)`, and should be less than or equal to `pic_width_max_in_luma_samples`.

[0084] When subpics_present_flag equals 1 or ref_pic_resampling_enabled_flag equals 0, the value of pic_width_in_luma_samples should be equal to pic_width_max_in_luma_samples.

[0085] `pic_height_in_luma_samples` specifies the height (in luminance samples) of each decoded image referenced by PPS. `pic_height_in_luma_samples` should not be equal to 0 and should be an integer multiple of `Max(8, MinCbSizeY)`, and should be less than or equal to `pic_height_max_in_luma_samples`.

[0086] When subpics_present_flag equals 1 or ref_pic_resampling_enabled_flag equals 0, the value of pic_height_in_luma_samples should be equal to pic_height_max_in_luma_samples.

[0087] Clause 8.5.3.2.2 of VVC Draft Version 7, Version 14 contains the following:

[0088] xInt i =Clip3(0,picW-1,sps_ref_wraparound_enabled_flag?

[0089] ClipH((sps_ref_wraparound_offset_minus1+1)*MinCbSizeY,picW,(xInt L +i))xInt L +i) (644)

[0090] yInt i =Clip3(0,picH-1,yInt) L +i) (645)

[0091] Statement 8.5.6.3.2 of VVC Draft Version 7, Section 14:

[0092] xInt i =Clip3(0,picW-1,sps_ref_wraparound_enabled_flag?

[0093] ClipH((sps_ref_wraparound_offset_minus1+1)*MinCbSizeY,picW,xInt L +i-3)xInt L +i-3) (957)

[0094] yInt i =Clip3(0,picH-1,yInt) L +i-3) (958)

[0095] Statement 8.5.6.3.3 of VVC Draft Version 7, Version 14:

[0096] xInt=Clip3(0,picW-1,sps_ref_wraparound_enabled_flag? (968)

[0097] ClipH((sps_ref_wraparound_offset_minus1+1)*MinCbSizeY,picW,xInt L ):xInt L yInt = Clip3(0, picH-1, yInt) L (969)

[0098] Statement 8.5.6.3.4 of VVC Draft Version 7, Section 14:

[0099] xInt i =Clip3(0,picW C -1,sps_ref_wraparound_enabled_flag? ClipH(xOffset,picW C ,xInt C +i-1):xInt C +i-1)

[0100] yInt i =Clip3(0,picH C -1,yInt C +i-1) (974)

[0101] This disclosure addresses one or more issues in VVC Draft 7. It employs a horizontal wrap-around offset in motion compensation within VVC to reduce subjective visual artifacts for 360-degree use cases using a filled equal rectangular projection (PERP) format, where the left and right image boundaries are padded identically. In other words, in a PEP format image, the sample value at the left image boundary can be the same as the sample value at the right image boundary.

[0102] Figure 3 An example of wrap-around offset is shown in a reference image with PEP format. Figure 3 In the example, for instance, video encoder 200 or video decoder 300 can predict the current PU 136 of the current image 138. To predict the current PU 136, video encoder 200 or video decoder 300 uses motion vector 140 to locate the top-left corner of prediction block 142 relative to the top-left corner of co-located PU 144. Prediction block 142 and co-located PU 144 are located in reference image 146. Figure 3 As shown, reference pixels extending beyond the boundary of reference image 146 are surrounded by a position shifted by the width of the ERP. That is, reference samples are found in the right-hand portion that would otherwise be located to the left of the left boundary of reference image 146. These reference samples in the right-hand portion of reference image 146 are shown as... Figure 3 Reference sample 148 for wrapping. The ERP width is signaled as the wrap offset. In the SPS of VVC draft 7, the wrap enable flag and wrap offset are signaled as follows:

[0103] A value of 1 for `sps_ref_wraparound_enabled_flag` specifies that horizontal wraparound motion compensation is applied in inter-frame prediction. A value of 0 for `sps_ref_wraparound_enabled_flag` specifies that horizontal wraparound motion compensation is not applied. When the value of `(CtbSizeY / MinCbSizeY+1)` is less than or equal to `(pic_width_in_luma_samples / MinCbSizeY-1)`, where `pic_width_in_luma_samples` is the value of `pic_width_in_luma_samples` in any PPS referencing SPS, the value of `sps_ref_wraparound_enabled_flag` should be 0.

[0104] The increment of 1 in `sps_ref_wraparound_offset_minus1` specifies the offset used to calculate the horizontal wraparound position (in MinCbSizeY luminance samples). The value of `ref_wraparound_offset_minus1` should be in the range of (CtbSizeY / MinCbSizeY)+1 to (pic_width_in_luma_samples / MinCbSizeY)-1 (inclusive), where `pic_width_in_luma_samples` is the value of `pic_width_in_luma_samples` in any PPS referencing SPS.

[0105] Figure 4 This illustrates how the position of a reference pixel could be incorrectly moved to an unknown location, i.e., outside the reference image, due to the wrap-around offset of the current image. Figure 4 In this context, `wraparound_offset_curr` represents the wraparound offset derived from the PPS of the current image 150. Since the reference image 152 has a different height and width than the current image 150, `wraparound_offset_curr` points outside the reference image 152.

[0106] The wrap offset can be less than or equal to the image width. Because RPR design may cause updates to the image width and height within a PPS, the wrap offset may differ for different images belonging to different PPSs. That is, the current image and reference image referencing two different PPSs can have different widths and heights, such as... Figure 4 As shown. VVC draft 7 fixes the wrap-around offset in the decoded layer video sequence (CLVS). However, as... Figure 4 As shown, VVC draft 7 may cause video decoders to incorrectly move the position of pixels outside the reference image by using the wrap offset derived from the PPS referenced by the current image, where the width of the reference image and the width of the current image are different due to RPR being enabled.

[0107] This disclosure describes techniques that can address this deficiency in VVC Draft 7. This disclosure describes several modifications to VVC Draft 7 that address the deficiency described above. According to one technique of this disclosure, the signaling for the wrap-around offset can be moved from SPS to PPS. That is, the video encoder 200 can be configured to include the wrap-around offset as part of the PPS syntax structure, and the video decoder 300 can be configured to receive the wrap-around offset as part of the PPS syntax structure.

[0108] The wraparound offset can be less than or equal to the image width. Since the RPR design can update the image width and height in the PPS, this disclosure proposes that the wraparound offset be signaled in the PPS instead of the SPS to ensure design consistency in the RPR. In a first example of such a constraint, the value of `ref_wraparound_enabled_flag` may be the same for all PPSs referenced by the decoded image in the CLVS. In a second example of such a constraint, the value of `ref_wraparound_enabled_flag` can be changed in the CLVS. Both examples can be applied to any of the changes proposed in this disclosure.

[0109] Examples of changes made to VVC Draft 7 to implement the techniques of this disclosure are shown below. Throughout this disclosure, the symbol "" is used. <add> "This indicates the start of the addition proposed through this disclosure, and is marked with "< / add> "This indicates the end of these additions. The marker " <del> "This indicates the start of the deletion (i.e., text removal) proposed through this disclosure, and is marked with "< / del> This indicates the end of these deletions.

[0110] In some examples, clause 7.3.2.3 of VVC draft version 7.14 can be modified as follows:

[0111]

[0112] In some examples, clause 7.4.3.3 in VVC draft version 7.14 can be modified as follows:

[0113] <del>`sps_ref_wraparound_enabled_flag` equal to 1 specifies that horizontal wraparound motion compensation is applied in inter-frame prediction. `sps_ref_wraparound_enabled_flag` equal to 0 specifies that horizontal wraparound motion compensation is not applied. When the value of (CtbSizeY / MinCbSizeY+1) is greater than or equal to (pic_width_in_luma_samples / MinCbSizeY-1), where pic_width_in_luma_samples is the value of pic_width_in_luma_samples in any PPS referencing SPS, the value of `sps_ref_wraparound_enabled_flag` should be equal to 0.

[0114] The increment of 1 in `sps_ref_wraparound_offset_minus1` specifies the offset used to calculate the horizontal wraparound position (in MinCbSizeY luminance samples). The value of `ref_wraparound_offset_minus1` should be in the range of (CtbSizeY / MinCbSizeY)+1 to (pic_width_in_luma_samples / MinCbSizeY)-1 (inclusive), where `pic_width_in_luma_samples` is the value of `pic_width_in_luma_samples` in any PPS referencing SPS.< / del>

[0115] In some examples, clause 7.3.2.4 of VVC draft version 7.14 can be modified as follows:

[0116]

[0117]

[0118] In some examples, clause 7.4.3.4 of VVC draft version 7.14 can be modified as follows:

[0119] <add>A value of 1 for `ref_wraparound_enabled_flag` specifies that horizontal wraparound motion compensation is applied in inter-frame prediction. A value of 0 for `ref_wraparound_enabled_flag` specifies that horizontal wraparound motion compensation is not applied. When the value of (CtbSizeY / MinCbSizeY+1) is greater than or equal to (pic_width_in_luma_samples / MinCbSizeY-1), where pic_width_in_luma_samples is the value of pic_width_in_luma_samples in any PPS referencing SPS, the value of `ref_wraparound_enabled_flag` should be 0.

[0120] The requirement for bitstream consistency is that the value of ref_wraparound_enabled_flag should be the same for all PPS referenced by decoded images in CLVS.

[0121] The increment of 1 in `ref_wraparound_offset_minus1` specifies the offset used to calculate the horizontal wraparound position (in units of `MinCbSizeY` luminance samples). The value of `ref_wraparound_offset_minus1` should be in the range of `(CtbSizeY / MinCbSizeY)+1` to `(pic_width_in_luma_samples / MinCbSizeY)-1` (inclusive).< / add>

[0122] The technology disclosed herein can also address issues related to motion compensation. In VVC Draft 7, the positions of pixels outside the reference image may be incorrectly offset by the wrap-around offset of the current image, such as... Figure 4 As shown, the widths of the reference image and the current image differ due to RPR being enabled. This disclosure describes a technique to address this issue by replacing the wrap offset of the current image with a wrap offset derived in the PPS referenced by the reference image.

[0123] Examples of changes to VVC Draft 7 made to implement the technology of this disclosure are shown below.

[0124] In some examples, clause 8.5.3.2.2 in VVC draft version 7, version 14 can be modified as follows: <add> ref_wraparound_offset_minus1 is the ref_wraparound_offset_minus1 parsed in the PPS referenced by the reference image.< / add>

[0125] xInt i =Clip3(0,picW-1, <del> sps_< / del> ref_wraparound_enabled_flag? ClipH(( <del> sps_< / del>ref_wraparound_offset_minus1+1)*MinCbSizeY,picW,(xInt L +i)): (644)

[0126] xInt L +i)

[0127] yInt i =Clip3(0,picH-1,yInt) L +i) (645)

[0128] In some examples, clause 8.5.6.3.2 in VVC draft version 7, version 14 can be modified as follows:

[0129] <add>

[0130] ref_wraparound_offset_minus1 is the ref_wraparound_offset_minus1 parsed in the PPS referenced by the reference image.

[0131] < / add>

[0132] xInt i =Clip3(0,picW-1, <del> sps_< / del> ref_wraparound_enabled_flag? ClipH(( <del> sps_< / del> ref_wraparound_offset_minus1+1)*MinCbSizeY,picW,xInt L +i-3): (957)

[0133] xInt L +i-3)

[0134] yInt i =Clip3(0,picH-1,yInt) L +i-3) (958)

[0135] In some examples, clause 8.5.6.3.3 in VVC draft version 7, version 14 can be modified as follows:

[0136] <add>

[0137] ref_wraparound_offset_minus1 is the ref_wraparound_offset_minus1 parsed in the PPS referenced by the reference image.

[0138] < / add>

[0139] xInt = Clip3(0, picW-1, ... <del> sps_< / del> ref_wraparound_enabled_flag? (968)ClipH(( <del> sps_< / del> ref_wraparound_offset_minus1+1)*MinCbSizeY,picW,xInt L ):xInt L )

[0140] yInt = Clip3(0, picH-1, yInt)L (969)

[0141] In some examples, clause 8.5.6.3.4 of VVC draft version 7, version 14 can be modified as follows:

[0142] <add>

[0143] ref_wraparound_offset_minus1 is the ref_wraparound_offset_minus1 parsed in the PPS referenced by the reference image.

[0144] < / add>

[0145] The variable xOffset is set to equal to ( <del> sps_< / del> ref_wraparound_offset_minus1+1)*MinCbSizeY) / SubWidthC.

[0146] xInt i =Clip3(0,picW C -1, <del> sps_< / del> ref_wraparound_enabled_flag? ClipH(xOffset,picW C ,xInt C +i-1):xInt C +i-1)

[0147] yInt i =Clip3(0,picH C -1,yInt C +i-1) (974)

[0148] In some examples, the techniques of this disclosure include not only replacing the wrap offset of the current image with the wrap offset derived in the PPS referenced by the reference image, but also replacing the wrap flag of the current image with the wrap flag derived in the PPS referenced by the reference image. Examples of changes to VVC Draft 7 made to implement the techniques of this disclosure are shown below.

[0149] In some examples, clause 8.5.3.2.2 in VVC draft version 7, version 14 can be modified as follows: <add>The ref_wraparound_enabled_flag is the ref_wraparound_enabled_flag parsed in the PPS referenced via the image.

[0150] ref_wraparound_offset_minus1 is the ref_wraparound_offset_minus1 parsed in the PPS referenced by the reference image.

[0151] < / add>

[0152] xInt i =Clip3(0,picW-1, <del> sps_< / del> ref_wraparound_enabled_flag? ClipH(( <del> sps_< / del> ref_wraparound_offset_minus1+1)*MinCbSizeY,picW,(xInt L +i)): (644)

[0153] xInt L +i)

[0154] yInt i =Clip3(0,picH-1,yInt) L +i) (645)

[0155] In some examples, clause 8.5.6.3.2 in VVC draft version 7, version 14 can be modified as follows:

[0156] <add>

[0157] The ref_wraparound_enabled_flag is the ref_wraparound_enabled_flag parsed in the PPS referenced via the image.

[0158] ref_wraparound_offset_minus1 is the ref_wraparound_offset_minus1 parsed in the PPS referenced by the reference image.

[0159] < / add>

[0160] xInt i =Clip3(0,picW-1, <del> sps_< / del> ref_wraparound_enabled_flag?

[0161] ClipH(( <del> sps_< / del> ref_wraparound_offset_minus1+1)*MinCbSizeY,picW,xInt L +i-3): (957)

[0162] xInt L +i-3)

[0163] yInt i =Clip3(0,picH-1,yInt) L +i-3) (958)

[0164] In some examples, clause 8.5.6.3.3 in VVC draft version 7, version 14 can be modified as follows:

[0165] <add>

[0166] The ref_wraparound_enabled_flag is the ref_wraparound_enabled_flag parsed in the PPS referenced via the image.

[0167] ref_wraparound_offset_minus1 is the ref_wraparound_offset_minus1 parsed in the PPS referenced by the reference image.

[0168] < / add>

[0169] xInt = Clip3(0, picW-1, ... <del> sps_< / del> ref_wraparound_enabled_flag? (968)

[0170] ClipH(( <del> sps_< / del> ref_wraparound_offset_minus1+1)*MinCbSizeY,picW,xInt L ):xInt L )

[0171] yInt = Clip3(0, picH-1, yInt) L (969)

[0172] In some examples, clause 8.5.6.3.4 of VVC draft version 7, version 14 can be modified as follows:<add>The ref_wraparound_enabled_flag is the ref_wraparound_enabled_flag parsed in the PPS referenced via the image.

[0173] ref_wraparound_offset_minus1 is the ref_wraparound_offset_minus1 parsed in the PPS referenced by the reference image.

[0174] < / add>

[0175] The variable xOffset is set to equal to

[0176] ( <del> sps_< / del> ref_wraparound_offset_minus1+1)*MinCbSizeY) / SubWidthC.

[0177] xInt i =Clip3(0,picW C -1, <del> sps_< / del> ref_wraparound_enabled_flag? ClipH(xOffset,picW C ,xInt C +i-1):xInt C +i-1)

[0178] yInt i =Clip3(0,picH C -1,yInt C +i-1) (974)

[0179] In some examples, the techniques of this disclosure include using a scaling ratio between the reference image and the current image for motion compensation to scale the wraparound offset derived from the PPS referenced by the current image. A general formula for deriving the wraparound offset of the reference image (defined as ref_wraparound_offset) can be shown below:

[0180] ref_wraparound_offset=curr_wraparound_offset*ref_pic_width÷curr_pic_width,

[0181] Where curr_wraparound_offset is the wraparound offset of the current image, ref_pic_width is defined as pic_width_max_in_luma_samples of the reference image, and curr_pic_width is defined as pic_width_max_in_luma_samples of the current image.

[0182] Considering the accuracy of fixed-point calculations, the formula can be calculated with higher precision. One example is using scalingRatio[0] as defined in VVC Draft 7, which calculates the value of ref_pic_width ÷ curr_pic_width by shifting ref_pic_width to the left by 14. The wrap-around offset of the reference image can then be derived as follows:

[0183] ref_wraparound_offset=((curr_wraparound_offset*scalingRatio[0])+8192)>>14

[0184] In some examples, the techniques of this disclosure include disallowing the simultaneous enabling of RPR and wraparound features. For example, in some examples, if sps_ref_wraparound_enabled_flag equals 1, the video decoder 300 can be configured to infer ref_pic_resampling_enabled_flag as 0. In some examples, if ref_pic_resampling_enabled_flag equals 1, the video decoder 300 can be configured to infer sps_ref_wraparound_enabled_flag as 0.

[0185] The video encoder 200 and video decoder 300 can be configured to apply surround offset when subpictures are enabled. The video encoder 200 and video decoder 300 can be configured to add subpicture indices to the signaling for surround offset during the parsing process in PPS and the decoding process for motion compensation. Examples of changes to VVC draft 7 made to implement the techniques of this disclosure are shown below.

[0186] In some examples, clause 7.3.2.4 of VVC draft version 7.14 can be modified as follows:

[0187]

[0188] In some examples, clause 7.4.3.4 of VVC draft version 7.14 can be modified as follows:

[0189] <add>

[0190] `ref_wraparound_enabled_flag[i]` equal to 1 specifies that horizontal wraparound motion compensation is applied to the i-th sub-image during inter-frame prediction. `ref_wraparound_enabled_flag` equal to 0 specifies that horizontal wraparound motion compensation is not applied to the i-th sub-image. When the value of (CtbSizeY / MinCbSizeY+1) is greater than or equal to (subpic_width_in_luma_samples / MinCbSizeY-1), where subpic_width_in_luma_samples is the sub-image width of the i-th sub-image, the value of `ref_wraparound_enabled_flag` should be equal to 0.

[0191] The increment of 1 in `ref_wraparound_offset_minus1[i]` specifies the offset (in units of `MinCbSizeY` luminance samples) used to calculate the horizontal wraparound position of the `i`-th subpic. The value of `ref_wraparound_offset_minus1[i]` should be in the range of `(CtbSizeY / MinCbSizeY)+1` to `(subpic_width_in_luma_samples / MinCbSizeY)-1` (inclusive).

[0192] < / add>

[0193] In some examples, clause 8.5.3.2.2 in VVC draft version 7, version 14 can be modified as follows:

[0194] <add>

[0195] RefSubPicIdx is deduced to make SubpicIdList[RefSubPicIdx] of the reference image equal to slice_subpic_id of the current slice. ref_wraparound_offset_minus1[RefSubPicIdx] is the ref_wraparound_offset_minus1[RefSubPicIdx] parsed in the PPS referenced by the reference image.

[0196] < / add>

[0197] xInt i=Clip3(0,picW-1, <del> sps_< / del> ref_wraparound_enabled_flag <add> [RefSubPicIdx]< / add> ?

[0198] ClipH(( <del> sps_< / del> ref_wraparound_offset_minus1 <add> [RefSubPicIdx]< / add> +1)*MinCbSizeY,picW,(xInt L +i)): (644)

[0199] xInt L +i)

[0200] yInt i =Clip3(0,picH-1,yInt) L +i) (645)

[0201] In some examples, clause 8.5.6.3.2 in VVC draft version 7, version 14 can be modified as follows:

[0202] <add>RefSubPicIdx is deduced to make SubpicIdList[RefSubPicIdx] of the reference image equal to slice_subpic_id of the current slice.

[0203] ref_wraparound_offset_minus1[RefSubPicIdx] is the ref_wraparound_offset_minus1[RefSubPicIdx] parsed in the PPS referenced by the reference image.< / add>

[0204] xInt i =Clip3(0,picW-1, <del> sps_< / del> ref_wraparound_enabled_flag <add> [RefSubPicIdx]< / add> ClipH(( <del> sps_< / del> ref_wraparound_offset_minus1 <add> [RefSubPicIdx]< / add> +1)*MinCbSizeY,picW,xInt L +i-3): (957)

[0205] xInt L +i-3)

[0206] yInt i =Clip3(0,picH-1,yInt) L +i-3) (958)

[0207] In some examples, clause 8.5.6.3.3 in VVC draft version 7, version 14 can be modified as follows:

[0208] <add>RefSubPicIdx is deduced to make SubpicIdList[RefSubPicIdx] of the reference image equal to slice_subpic_id of the current slice.

[0209] ref_wraparound_offset_minus1[RefSubPicIdx] is the ref_wraparound_offset_minus1[RefSubPicIdx] parsed in the PPS referenced by the reference image.< / add>

[0210] xInt = Clip3(0, picW-1, ... <del> sps_< / del> ref_wraparound_enabled_flag <add> [RefSubPicIdx]< / add>ClipH(( <del> sps_< / del> ref_wraparound_offset_minus1 <add> [RefSubPicIdx]< / add> +1)*MinCbSizeY,picW,xInt L ):xInt L (968)

[0211] yInt = Clip3(0, picH-1, yInt) L (969)

[0212] In some examples, clause 8.5.6.3.4 of VVC draft version 7, version 14 can be modified as follows:

[0213] <add>RefSubPicIdx is deduced to make SubpicIdList[RefSubPicIdx] of the reference image equal to slice_subpic_id of the current slice.

[0214] ref_wraparound_offset_minus1[RefSubPicIdx] is the ref_wraparound_offset_minus1[RefSubPicIdx] parsed in the PPS referenced by the reference image.< / add>

[0215] The variable xOffset is set to equal to ( <del> sps_< / del> ref_wraparound_offset_minus1 <add> [RefSubPicIdx]< / add> +1)*MinCbSizeY) / SubWidthC.

[0216] xInt i =Clip3(0,picW C -1, <del> sps_< / del> ref_wraparound_enabled_flag <add> [RefSubPicIdx]< / add> ClipH(xOffset, picW) C ,xInt C +i-1):

[0217] xInt C +i-1)

[0218] yInt i =Clip3(0,picH C -1,yInt C +i-1) (974)

[0219] Figure 5 This is a block diagram illustrating an example video encoder 200 that can perform the techniques described in this disclosure. Figure 5 This disclosure is provided for illustrative purposes and should not be construed as limiting the techniques illustrated and described in this disclosure in a general manner. For illustrative purposes, this disclosure describes a video encoder 200 based on the techniques of JEM, VVC (ITU-T H.266 under development), and HEVC (ITU-T H.265). However, the techniques of this disclosure can be implemented by video encoding devices configured for other video decoding standards.

[0220] exist Figure 5 In the example, the video encoder 200 includes a video data memory 230, a mode selection unit 202, a residual generation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, a reconstruction unit 214, a filter unit 216, a decoded picture buffer (DPB) 218, and an entropy coding unit 220. Any or all of the video data memory 230, mode selection unit 202, residual generation unit 204, transform processing unit 206, quantization unit 208, inverse quantization unit 210, inverse transform processing unit 212, reconstruction unit 214, filter unit 216, DPB 218, and entropy coding unit 220 can be implemented in one or more processors or in processing circuitry. For example, the units of the video encoder 200 can be implemented as one or more circuit or logic elements as part of hardware circuitry, or as part of a processor, ASIC, or FPGA. Furthermore, the video encoder 200 may include additional or alternative processors or processing circuitry to perform these and other functions.

[0221] The video data storage device 230 can store video data to be encoded by the components of the video encoder 200. The video encoder 200 can obtain data from, for example, a video source 104 (…). Figure 1 The video data memory 230 receives video data stored in the video data memory 230. The DPB 218 can act as a reference picture memory, storing reference video data for use when the video encoder 200 predicts subsequent video data. The video data memory 230 and DPB 218 can be formed from any of a variety of memory devices, such as dynamic random access memory (DRAM) (including synchronous DRAM (SDRAM)), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. The video data memory 230 and DPB 218 can be provided by the same memory device or separate memory devices. In various examples, the video data memory 230 can be on-chip (as shown) with other components of the video encoder 200, or off-chip relative to those components.

[0222] In this disclosure, references to video data memory 230 should not be construed as limited to memory within video encoder 200 (unless so specifically described) or to memory outside video encoder 200 (unless so specifically described). Rather, references to video data memory 230 should be understood as reference memory storing video data received by video encoder 200 for encoding (e.g., video data for the current block to be encoded). Figure 1The memory 106 can also provide temporary storage for the outputs from the various units of the video encoder 200.

[0223] It shows Figure 5 The various units help to understand the operations performed by the video encoder 200. These units can be implemented as fixed-function circuits, programmable circuits, or a combination thereof. Fixed-function circuits refer to circuits that provide a specific function and are pre-configured regarding the operations that can be performed. Programmable circuits refer to circuits that can be programmed to perform various tasks and provide flexible functionality in terms of the operations that can be performed. For example, a programmable circuit can execute software or firmware that causes the programmable circuit to operate in a manner defined by the instructions of the software or firmware. Fixed-function circuits can execute software instructions (e.g., to receive or output parameters), but the type of operation performed by a fixed-function circuit is generally immutable. In some examples, one or more of these units can be different circuit blocks (fixed-function or programmable), and in some examples, one or more of these units can be integrated circuits.

[0224] The video encoder 200 may include an arithmetic logic unit (ALU), an essential function unit (EFU), digital circuitry, analog circuitry, and / or a programmable core, all formed by programmable circuitry. In an example where the operation of the video encoder 200 is performed using software executed by programmable circuitry, memory 106 ( Figure 1 The video encoder 200 may store instructions (e.g., object code) of the software received and executed by the video encoder 200, or another memory (not shown) within the video encoder 200 may store such instructions.

[0225] The video data storage unit 230 is configured to store the received video data. The video encoder 200 can retrieve images of the video data from the video data storage unit 230 and provide the video data to the residual generation unit 204 and the mode selection unit 202. The video data in the video data storage unit 230 can be the raw video data to be encoded.

[0226] The mode selection unit 202 includes a motion estimation unit 222, a motion compensation unit 224, and an intra-frame prediction unit 226. The mode selection unit 202 may include additional functional units that perform video prediction based on other prediction modes. As an example, the mode selection unit 202 may include a palette unit, an intra-block copy unit (which may be part of the motion estimation unit 222 and / or the motion compensation unit 224), an affine unit, a linear model (LM) unit, etc.

[0227] Mode selection unit 202 typically coordinates multiple coding passes to test combinations of coding parameters and the rate-distortion values ​​obtained for such combinations. Coding parameters may include segmenting the CTU into CUs, the prediction mode for the CUs, the transformation type of the residual data for the CUs, and the quantization parameters of the residual data for the CUs. Mode selection unit 202 can ultimately select a combination of coding parameters that yields a better rate-distortion value than other tested combinations.

[0228] The video encoder 200 can segment images retrieved from the video data storage 230 into a series of CTUs and encapsulate one or more CTUs within a slice. The mode selection unit 202 can segment the CTUs of the image according to a tree structure (such as the QTBT structure or quadtree structure of HEVC described above). As mentioned above, the video encoder 200 can form one or more CUs by segmenting CTUs according to a tree structure. Such CUs can also be referred to as "video blocks" or "blocks".

[0229] Typically, mode selection unit 202 also controls its components (e.g., motion estimation unit 222, motion compensation unit 224, and intra-prediction unit 226) to generate prediction blocks for the current block (e.g., the current CU, or the overlapping portion of PU and TU in HEVC). To perform inter-frame prediction for the current block, motion estimation unit 222 may perform a motion search to identify one or more closely matching reference blocks in one or more reference pictures (e.g., one or more previously decoded pictures stored in DPB 218). Specifically, motion estimation unit 222 may calculate values ​​representing the similarity between a potential reference block and the current block, for example, based on sum of absolute differences (SAD), sum of squared differences (SSD), mean absolute difference (MAD), mean squared difference (MSD), etc. Motion estimation unit 222 may typically perform these calculations using sample-by-sample differences between the current block and the considered reference blocks. Motion estimation unit 222 may identify the reference block with the lowest value obtained from these calculations, indicating the reference block that most closely matches the current block.

[0230] Motion estimation unit 222 can generate one or more motion vectors (MVs) that define the position of a reference block in a reference image relative to the position of the current block in the current image. Motion estimation unit 222 can then provide the motion vectors to motion compensation unit 224. For example, for unidirectional inter-frame prediction, motion estimation unit 222 can provide a single motion vector, while for bidirectional inter-frame prediction, motion estimation unit 222 can provide two motion vectors. Motion compensation unit 224 can then use the motion vectors to generate prediction blocks. For example, motion compensation unit 224 can use the motion vectors to retrieve data for the reference blocks. As another example, if the motion vectors have fractional-sample precision, motion compensation unit 224 can interpolate the values ​​used for the prediction blocks according to one or more interpolation filters. Furthermore, for bidirectional inter-frame prediction, motion compensation unit 224 can retrieve data for two reference blocks identified by corresponding motion vectors and combine the retrieved data, for example, by per-sample averaging or weighted averaging.

[0231] As another example, for intra-prediction or intra-prediction decoding, intra-prediction unit 226 can generate a prediction block based on samples adjacent to the current block. For example, in directional mode, intra-prediction unit 226 can typically mathematically combine the values ​​of adjacent samples and fill these calculated values ​​across the current block in a defined direction to generate a prediction block. As another example, in DC mode, intra-prediction unit 226 can calculate the average of the adjacent samples of the current block and generate a prediction block to include the obtained average for each sample of the prediction block.

[0232] Mode selection unit 202 provides a prediction block to residual generation unit 204. Residual generation unit 204 receives the original, uncoded version of the current block from video data memory 230 and the prediction block from mode selection unit 202. Residual generation unit 204 calculates the sample-by-sample difference between the current block and the prediction block. The resulting sample-by-sample difference defines the residual block for the current block. In some examples, residual generation unit 204 may determine the difference between sample values ​​in the residual block to generate the residual block using residual differential pulse decode-modulation (RDPCM). In some examples, one or more subtractor circuits performing binary subtraction may be used to form residual generation unit 204.

[0233] In the example where mode selection unit 202 divides a CU into PUs, each PU can be associated with a luma prediction unit and a corresponding chroma prediction unit. Video encoder 200 and video decoder 300 can support PUs of various sizes. As noted above, the size of a CU can refer to the size of the luma decoding block of the CU, while the size of a PU can refer to the size of the luma prediction unit of the PU. Assuming a particular CU size is 2Nx2N, video encoder 200 can support PU sizes of 2Nx2N or NxN for intra-frame prediction, and symmetrical PU sizes of 2Nx2N, 2NxN, Nx2N, NxN, or similar sizes for inter-frame prediction. Video encoder 200 and video decoder 300 can also support asymmetric segmentation for PU sizes of 2NxnU, 2NxnD, nLx2N, and nRx2N for inter-frame prediction.

[0234] In the example where mode selection unit 202 does not further divide the CU into PUs, each CU can be associated with a luminance decoding block and a corresponding chrominance decoding block. As mentioned above, the size of the CU can refer to the size of the luminance decoding block of the CU. The video encoder 200 and the video decoder 300 can support CU sizes of 2Nx2N, 2NxN, or Nx2N.

[0235] For other video decoding techniques (such as block-based copy mode decoding, affine mode decoding, and linear model (LM) mode decoding, mode selection unit 202 generates a prediction block for the current block being encoded via a corresponding unit associated with the decoding technique. In some examples (such as palette mode decoding), mode selection unit 202 may not generate a prediction block, but instead generate syntax elements indicating how the block should be reconstructed based on the selected palette. In such a mode, mode selection unit 202 can provide these syntax elements to entropy coding unit 220 for encoding.

[0236] As described above, the residual generation unit 204 receives video data for the current block and the corresponding prediction block. Then, the residual generation unit 204 generates a residual block for the current block. To generate the residual block, the residual generation unit 204 calculates the sample-by-sample difference between the prediction block and the current block.

[0237] Transform processing unit 206 applies one or more transformations to the residual block to generate a block of transform coefficients (referred to herein as a "transform coefficient block"). Transform processing unit 206 can apply various transformations to the residual block to form the transform coefficient block. For example, transform processing unit 206 can apply a discrete cosine transform (DCT), direction transform, Karhunen-Loeve transform (KLT), or conceptually similar transformations to the residual block. In some examples, transform processing unit 206 can perform multiple transformations on the residual block, such as primary and secondary transformations (e.g., rotation transformations). In some examples, transform processing unit 206 does not apply any transformations to the residual block.

[0238] Quantization unit 208 can quantize the transform coefficients in the transform coefficient block to produce a quantized transform coefficient block. Quantization unit 208 can quantize the transform coefficients of the transform coefficient block based on the quantization parameter (QP) value associated with the current block. Video encoder 200 (e.g., via mode selection unit 202) can adjust the degree of quantization applied to the transform coefficient block associated with the current block by adjusting the QP value associated with the CU. Quantization may cause information loss, and therefore, the quantized transform coefficients may have lower accuracy compared to the original transform coefficients produced by transform processing unit 206.

[0239] The inverse quantization unit 210 and the inverse transform processing unit 212 can apply inverse quantization and inverse transform, respectively, to the quantized transform coefficient block to reconstruct the residual block from the transform coefficient block. The reconstruction unit 214 can generate a reconstructed block corresponding to the current block based on the reconstructed residual block and the prediction block generated by the mode selection unit 202 (although potentially with some degree of distortion). For example, the reconstruction unit 214 can add samples of the reconstructed residual block to corresponding samples of the prediction block generated by the mode selection unit 202 to generate the reconstructed block.

[0240] Filter unit 216 can perform one or more filtering operations on the reconstructed block. For example, filter unit 216 can perform deblocking to reduce block artifacts along the edges of the CU. In some examples, the operations of filter unit 216 can be skipped.

[0241] The video encoder 200 stores the reconstructed blocks in the DPB 218. For example, in an example where the operation of the filter unit 216 is not required, the reconstruction unit 214 can store the reconstructed blocks in the DPB 218. In an example where the operation of the filter unit 216 is required, the filter unit 216 can store the filtered reconstructed blocks in the DPB 218. The motion estimation unit 222 and the motion compensation unit 224 can retrieve a reference picture formed by the reconstructed (and potentially filtered) blocks from the DPB 218 to perform inter-frame prediction of blocks in subsequently encoded pictures. Additionally, the intra-frame prediction unit 226 can use the reconstructed blocks of the current picture in the DPB 218 to perform intra-frame prediction of other blocks in the current picture.

[0242] Typically, entropy coding unit 220 can entropy-encode syntax elements received from other functional components of video encoder 200. For example, entropy coding unit 220 can entropy-encode quantized transform coefficient blocks from quantization unit 208. As another example, entropy coding unit 220 can entropy-encode predictive syntax elements (e.g., motion information for inter-frame prediction or intra-frame mode information for intra-frame prediction) from mode selection unit 202. Entropy coding unit 220 can perform one or more entropy coding operations on syntax elements, another example of video data, to generate entropy-encoded data. For example, entropy coding unit 220 can perform context-adaptive variable-length decoding (CAVLC), CABAC, variable-to-variable (V2V) length decoding, syntax-based context-adaptive binary arithmetic decoding (SBAC), probabilistic interval partitioned entropy (PIPE) decoding, exponential Golomb coding, or another type of entropy coding operation on the data. In some examples, entropy coding unit 220 can operate in a bypass mode where syntax elements are not entropy-encoded.

[0243] The video encoder 200 can output a bitstream that includes entropy-encoded syntax elements required for reconstructing slices or blocks of images. Specifically, the entropy coding unit 220 can output a bitstream.

[0244] The above operations are described in relation to the blocks. Such a description should be understood as referring to the operations used for the luma decoding block and / or the chroma decoding block. As mentioned above, in some examples, the luma decoding block and the chroma decoding block are the luma and chroma components of the CU. In some examples, the luma decoding block and the chroma decoding block are the luma and chroma components of the PU.

[0245] In some examples, it is not necessary to repeat the operations performed for the luma-coded block for the chroma-decoded block. As an example, it is not necessary to repeat the operations used to identify the motion vector (MV) and reference image for the luma-decoded block to identify the MV and reference image for the chroma-decoded block. Specifically, the MV for the luma-decoded block can be scaled to determine the MV for the chroma-decoded block, and the reference image can be the same. As another example, the intra-frame prediction process can be the same for both the luma-decoded block and the chroma-decoded block.

[0246] Video encoder 200 represents an example of a device configured to encode video data, the device including: a memory configured to store video data; and one or more processing units implemented in circuitry and configured to perform the techniques of this disclosure, including the techniques in the claims portion.

[0247] Figure 6 This is a block diagram illustrating an example video decoder 300 capable of performing the techniques described herein. Figure 6 This disclosure is provided for illustrative purposes and does not limit the techniques illustrated and described in this disclosure in a general manner. For illustrative purposes, this disclosure describes a video decoder 300 based on the techniques of JEM, VVC (ITU-T H.266 under development), and HEVC (ITU-TH.265). However, the techniques of this disclosure can be implemented by video decoding devices configured for other video decoding standards.

[0248] exist Figure 6 In the example, the video decoder 300 includes a decoded picture buffer (CPB) memory 320, an entropy decoding unit 302, a prediction processing unit 304, an inverse quantization unit 306, an inverse transform processing unit 308, a reconstruction unit 310, a filter unit 312, and a decoded picture buffer (DPB) 134. Any or all of the CPB memory 320, entropy decoding unit 302, prediction processing unit 304, inverse quantization unit 306, inverse transform processing unit 308, reconstruction unit 310, filter unit 312, and DPB 134 can be implemented in one or more processors or in processing circuitry. For example, units of the video decoder 300 can be implemented as one or more circuit or logic elements as part of hardware circuitry, or as part of a processor, ASIC, or FPGA. Furthermore, the video decoder 300 may include additional or alternative processors or processing circuitry to perform these and other functions.

[0249] The prediction processing unit 304 includes a motion compensation unit 316 and an intra-frame prediction unit 318. The prediction processing unit 304 may include an addition unit that performs predictions based on other prediction modes. As an example, the prediction processing unit 304 may include a palette unit, an intra-block copy unit (which may form part of the motion compensation unit 316), an affine unit, a linear model (LM) unit, etc. In other examples, the video decoder 300 may include more, fewer, or different functional components.

[0250] CPB memory 320 can store video data to be decoded by components of video decoder 300, such as encoded video bitstreams. For example, it can be stored from computer-readable medium 110 ( Figure 1 The video data stored in the CPB memory 320 is obtained. The CPB memory 320 may include a CPB that stores encoded video data (e.g., syntax elements) from the encoded video bitstream. Additionally, the CPB memory 320 may store video data other than the syntax elements of the decoded picture, such as temporary data representing the output from various units of the video decoder 300. The DPB 314 typically stores decoded pictures that the video decoder 300 may output and / or use as reference video data when decoding subsequent data or pictures of the encoded video bitstream. The CPB memory 320 and DPB 314 may be formed of any of a variety of memory devices, such as DRAM, including SDRAM, MRAM, RRAM, or other types of memory devices. The CPB memory 320 and DPB 314 may be provided by the same memory device or separate memory devices. In various examples, the CPB memory 320 may be on-chip with other components of the video decoder 300, or off-chip relative to those components.

[0251] Alternatively or concurrently, in some examples, the video decoder 300 can be derived from the memory 120 ( Figure 1 The decoded video data is retrieved. In other words, memory 120 can utilize CPB memory 320 to store data, as discussed above. Similarly, when some or all of the functions of video decoder 300 are implemented using software to be executed by the processing circuitry of video decoder 300, memory 120 can store instructions to be executed by video decoder 300.

[0252] It shows Figure 6 The various units shown help to understand the operations performed by the video decoder 300. These units can be implemented as fixed-function circuits, programmable circuits, or a combination thereof. Similar to... Figure 5Fixed-function circuits refer to circuits that provide a specific function and are pre-configured regarding the operations that can be performed. Programmable circuits refer to circuits that can be programmed to perform various tasks and provide flexible functionality in terms of the operations that can be performed. For example, a programmable circuit can execute software or firmware that causes the programmable circuit to operate in a manner defined by the instructions of the software or firmware. Fixed-function circuits can execute software instructions (e.g., to receive or output parameters), but the type of operation performed by a fixed-function circuit is typically immutable. In some examples, one or more of these units may be different circuit blocks (fixed-function or programmable), and in some examples, one or more of these units may be integrated circuits.

[0253] The video decoder 300 may include an ALU, EFU, digital circuitry, analog circuitry, and / or a programmable core formed by programmable circuitry. In an example where the operation of the video decoder 300 is performed by software executing on the programmable circuitry, on-chip or off-chip memory may store instructions (e.g., object code) of the software received and executed by the video decoder 300.

[0254] Entropy decoding unit 302 can receive encoded video data from the CPB and perform entropy decoding on the video data to regenerate syntax elements. Prediction processing unit 304, inverse quantization unit 306, inverse transform processing unit 308, reconstruction unit 310, and filter unit 312 can generate decoded video data based on syntax elements extracted from the bitstream.

[0255] Typically, the video decoder 300 reconstructs the image block by block. The video decoder 300 can perform the reconstruction operation on each block individually (where the block currently being reconstructed (i.e., decoded) can be referred to as the "current block").

[0256] Entropy decoding unit 302 can entropy decode the syntax elements of the quantized transform coefficients that define the quantized transform coefficient block, as well as transform information such as quantization parameters (QP) and / or transform mode indications. Inverse quantization unit 306 can use the QP associated with the quantized transform coefficient block to determine the degree of quantization, and similarly, determine the degree of inverse quantization to be applied by inverse quantization unit 306. Inverse quantization unit 306 can, for example, perform a bitwise left shift operation to inverse quantize the quantized transform coefficients. Inverse quantization unit 306 can thus form a transform coefficient block including the transform coefficients.

[0257] After the inverse quantization unit 306 forms the transform coefficient block, the inverse transform processing unit 308 can apply one or more inverse transforms to the transform coefficient block to generate a residual block associated with the current block. For example, the inverse transform processing unit 308 can apply the inverse DCT, inverse integer transform, inverse Karhunen-Loeve transform (KLT), inverse rotation transform, inverse direction transform, or another inverse transform to the transform coefficient block.

[0258] Furthermore, the prediction processing unit 304 generates a prediction block based on the prediction information syntax elements entropy-decoded by the entropy decoding unit 302. For example, if the prediction information syntax elements indicate that the current block is inter-frame predicted, the motion compensation unit 316 can generate the prediction block. In this case, the prediction information syntax elements may indicate the reference picture from which the reference block is to be retrieved in the DPB 314, and a motion vector identifying the position of the reference block in the reference picture relative to the position of the current block in the current picture. The motion compensation unit 316 can typically be configured with respect to the motion compensation unit 224 ( Figure 5 The method described is basically similar to the way the inter-frame prediction process is performed.

[0259] As another example, if the prediction information syntax element indicates that the current block is intra-predicted, then intra-prediction unit 318 can generate a prediction block according to the intra-prediction mode indicated by the prediction information syntax element. Again, intra-prediction unit 318 can typically be configured with respect to intra-prediction unit 226 ( Figure 5 The intra-prediction process is performed in a manner substantially similar to that described above. The intra-prediction unit 318 can retrieve data from neighboring samples of the current block from the DPB 314.

[0260] Reconstruction unit 310 can reconstruct the current block using the prediction block and the residual block. For example, reconstruction unit 310 can reconstruct the current block by adding the samples of the residual block to the corresponding samples of the prediction block.

[0261] Filter unit 312 can perform one or more filter operations on the reconstructed block. For example, filter unit 312 can perform a deblocking operation to reduce block artifacts along the edges of the reconstructed block. The operation of filter unit 312 is not necessarily performed in all examples.

[0262] The video decoder 300 can store the reconstructed blocks in the DPB 314. For example, in an example where the filter unit 312 is not operated, the reconstruction unit 310 can store the reconstructed blocks in the DPB 314. In an example where the filter unit 312 is operated, the filter unit 312 can store the filtered reconstructed blocks in the DPB 314. As discussed above, the DPB 314 can provide reference information (such as samples of the current image for intra-frame prediction and samples of previously decoded images for subsequent motion compensation) to the prediction processing unit 304. Furthermore, the video decoder 300 can output decoded images (e.g., decoded video) from the DPB 314 for use in applications such as... Figure 1 The subsequent presentation on the display device 118.

[0263] In this way, video decoder 300 represents an example of a video decoding device, which includes: a memory configured to store video data; and one or more processing units implemented in circuitry and configured to perform the techniques of this disclosure, including the techniques in the claims portion.

[0264] Figure 7 This is a flowchart illustrating an example method for encoding the current block. The current block may include the current CU. Although regarding video encoder 200 ( Figure 1 and Figure 5 The description is provided, but it should be understood that other devices can be configured to perform the same actions. Figure 7 Similar to the method.

[0265] In this example, the video encoder 200 initially predicts the current block (350). For example, the video encoder 200 may form a prediction block for the current block. Then, the video encoder 200 may compute a residual block for the current block (532). To compute the residual block, the video encoder 200 may compute the difference between the original unencoded block and the prediction block for the current block. Then, the video encoder 200 may transform the residual block and quantize the transform coefficients of the residual block (354). Next, the video encoder 200 may scan the quantized transform coefficients of the residual block (356). During or after the scan, the video encoder 200 may entropy encode the transform coefficients (358). For example, the video encoder 200 may use CAVLC or CABAC to encode the transform coefficients. Then, the video encoder 200 may output the entropy-encoded data of the block (360).

[0266] Figure 8 This is a flowchart illustrating an example method for decoding the current block of video data. The current block may include the current CU. Although regarding video decoder 300 ( Figure 1 and Figure 6 The description is provided, but it should be understood that other devices can be configured to perform the same actions. Figure 8 Similar to the method.

[0267] The video decoder 300 can receive entropy-coded data for the current block (e.g., entropy-coded prediction information and entropy-coded data for the coefficients of the residual block corresponding to the current block) (370). The video decoder 300 can entropy decode the entropy-coded data to determine the prediction information for the current block and regenerate the coefficients of the residual block (372). The video decoder 300 can predict the current block (374), for example, by calculating a prediction block for the current block using an intra-frame or inter-frame prediction mode indicated by the prediction information of the current block. The video decoder 300 can then inverse scan the regenerated coefficients (376) to create a block of quantized transform coefficients. The video decoder 300 can then inverse quantize and inverse transform the transform coefficients to produce the residual block (378). Finally, the video decoder 300 can decode the current block by combining the prediction block and the residual block (380).

[0268] Figure 9 This is a flowchart illustrating an example method for encoding the current block of video data. The video data can be, for example, 360-degree video data. The current block may include the current CU. Although regarding video encoder 200 ( Figure 1 and Figure 5 The description is provided, but it should be understood that other devices can be configured to perform the same actions. Figure 9 Similar to the method.

[0269] The video encoder 200 determines a value for the surround offset (400). In response to determining that the current frame of the video data is encoded in inter-frame prediction mode, the video encoder 200 determines a motion vector for the frame and a reference image for the frame (402). The reference image may be, for example, a PERP format image. The left and right portions of the reference image may be padded with the same amount of material.

[0270] The video encoder 200 determines the horizontal wrap position (404) in the reference image based on the wrap offset value. The horizontal wrap position may, for example, correspond to the top left corner of the prediction block.

[0271] The video encoder 200 determines the prediction block (406) for the block based on the horizontal wrap position. The video encoder 200 may further determine the horizontal wrap position in the reference image based on the image width and image height relative to the reference image, and includes information identifying the image width and image height relative to the reference image in the PPS data structure.

[0272] The video encoder 200 includes a wrap offset value (408) in the PPS data structure for the current image. The wrap offset may be, for example, a first wrap offset for a first sub-image, and the video encoder 200 may be configured to include a second wrap offset value in the PPS data structure for a second sub-image, which is a portion of the same image as the first sub-image. The video encoder 200 may, for example, determine that horizontal wrap motion compensation is enabled for a block, and in response to determining that horizontal wrap motion compensation is enabled for a block, include a syntax element in the PPS data structure to indicate that horizontal wrap motion compensation is enabled for the block, and include the wrap offset value in the PPS data structure.

[0273] The video encoder 200 outputs a bitstream of encoded video data including a PPS data structure (410). The video encoder 200 may also include SPSs with SPS ID values ​​in the bitstream of encoded video data, and the PPS data structure may include syntax elements identifying the SPS ID values. For example, the video encoder 200 may output a bitstream of encoded video data either by outputting a bitstream for decoding and display or by storing the bitstream for later decoding.

[0274] Figure 10 This is a flowchart illustrating an example method for decoding the current block of video data. The video data can be, for example, 360-degree video data. The current block may include the current CU. Although regarding video decoder 300 ( Figure 1 and Figure 6 The description is provided, but it should be understood that other devices can be configured to perform the same actions. Figure 10 Similar to the method.

[0275] The video decoder 300 obtains a value for the wrap-around offset (420) from the PPS data structure of the current frame of the video data. For example, the video decoder 300 may receive a syntax element in the PPS data structure, determine whether horizontal wrap-around motion compensation is enabled for the block based on the value of the syntax element, and receive a value for the wrap-around offset in response to determining that horizontal wrap-around motion compensation is enabled for the block. The video decoder 300 may also receive an SPS with an SPS ID value, and the PPS data structure may include a syntax element identifying the SPS ID value. The wrap-around offset may be, for example, a first wrap-around offset for a first sub-frame, and the video decoder 300 may also be configured to obtain a value for a second wrap-around offset for a second sub-frame from the PPS data structure, wherein the first and second sub-frames are sub-frames of the same frame.

[0276] In response to determining that a block of the current image is encoded in inter-frame prediction mode, the video decoder 300 determines the motion vectors for the block and a reference image (422) for the block. The reference image may be, for example, a PRP format image. The left and right portions of the reference image may be padded with the same values. The reference image may have an associated PPS data structure separate from the PPS data structure of the current image.

[0277] The video decoder 300 determines the horizontal wrap position (424) in the reference image based on the value of the wrap offset. The horizontal wrap position may, for example, correspond to the top left corner of the prediction block. The video decoder 300 may also receive information in the PPS data structure identifying the image width and image height for the reference image, and further determine the horizontal wrap position in the reference image based on the image width and image height for the reference image.

[0278] The video decoder 300 determines the prediction block (426) for the block based on the horizontal wrap position. The video decoder 300 can also be configured to determine residual blocks, add residual blocks to the prediction blocks to determine reconstructed blocks of video data, apply one or more filters to the reconstructed blocks to determine the final decoded blocks, and output a decoded picture of the video data including the final decoded blocks.

[0279] The following clauses describe the characteristics of the aforementioned equipment and technology.

[0280] Clause 1: A method for decoding video data includes: receiving one or more surround offsets in a picture parameter set (PPS); and processing the one or more surround offsets according to any technique or combination thereof described in this disclosure.

[0281] Clause 2: The method according to Clause 1 further includes: receiving an SPS having a Sequence Parameter Set (SPS) ID value, wherein the PPS includes a syntax element identifying the SPS ID value.

[0282] Clause 3: An apparatus for decoding video data, the apparatus comprising one or more units for performing the method according to Clause 1 or 2.

[0283] Clause 4: The device according to Clause 3, wherein the one or more units include one or more processors implemented in a circuit.

[0284] Clause 5: The device according to Clause 3 or 4 further includes: a memory for storing the video data.

[0285] Clause 6: The device according to any one of Clauses 3-5 further includes: a display configured to display decoded video data.

[0286] Clause 7: The device according to any one of Clauses 3-6, wherein the device includes one or more of a camera, a computer, a mobile device, a broadcast receiver set-top box or a set-top box.

[0287] Clause 8: The device according to any one of Clauses 3-7, wherein the device includes a video decoder.

[0288] Clause 9: A method for encoding video data includes: determining one or more wrap-around offsets according to any technique or combination thereof described in this disclosure; and generating a set of picture parameters including the one or more wrap-around offsets.

[0289] Clause 10: The method according to Clause 1 further includes: generating an SPS having a Sequence Parameter Set (SPS) ID value, wherein the PPS includes a syntax element identifying the SPS ID value.

[0290] Clause 11: An apparatus for decoding video data, the apparatus comprising one or more units for performing the method according to Clause 9 or 10.

[0291] Clause 12: The device according to Clause 11, wherein the one or more units include one or more processors implemented in a circuit.

[0292] Clause 13: The device according to Clause 11 or 12 further includes: a memory for storing the video data.

[0293] Clause 14: The device according to any one of Clauses 11-13 further includes: a camera configured to capture the video data.

[0294] Clause 15: The device according to any one of Clauses 11-14, wherein the device includes one or more of a camera, a computer, or a mobile device.

[0295] Clause 16: The device according to any one of Clauses 11-15, wherein the device includes a video encoder.

[0296] Clause 17: A computer-readable storage medium having instructions stored thereon, which, when executed, cause one or more processors to perform the method according to any one of Clauses 1, 2, 9 or 10.

[0297] Clause 18: An apparatus for decoding video data includes a unit for performing the method according to any one of Clauses 1, 2, 9 or 10.

[0298] It should be recognized that, based on the examples, certain actions or events of any technique described herein may be performed in a different order, and may be added, combined, or omitted entirely (e.g., not all described actions or events are necessary for implementing the technique). Furthermore, in some examples, actions or events may be performed concurrently rather than sequentially, for example, through multithreaded processing, interrupt handling, or multiple processors.

[0299] In one or more examples, the described functionality can be implemented using hardware, software, firmware, or any combination thereof. If implemented in software, the functionality can be stored or transmitted as one or more instructions or code on or through a computer-readable medium and executed by a hardware-based processing unit. A computer-readable medium can include a computer-readable storage medium, which corresponds to a tangible medium such as a data storage medium or a communication medium, including, for example, any medium that facilitates the transfer of a computer program from one place to another according to a communication protocol. In this way, a computer-readable medium can generally correspond to (1) a non-transitory tangible computer-readable storage medium or (2) a communication medium such as a signal or carrier wave. A data storage medium can be any available medium that can be accessed by one or more computers or one or more processors to obtain instructions, code, and / or data structures for implementing the techniques described in this disclosure. Computer program products can include computer-readable media.

[0300] For example, rather than limiting, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage devices, flash memory, or any other medium capable of storing desired program code in the form of instructions or data structures and accessible by a computer. Furthermore, any connection is appropriately referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technology (e.g., infrared, radio, and microwave), then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technology (e.g., infrared, radio, and microwave) is included in the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but rather refer instead to non-transient tangible storage media. As used herein, disks and optical discs include compact optical discs (CDs), laser optical discs, optical discs, digital versatile optical discs (DVDs), floppy disks, and Blu-ray discs, wherein disks typically magnetically copy data, while optical discs utilize lasers to optically copy data. The combination of the above items should also be included within the scope of computer-readable media.

[0301] Instructions can be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, the terms "processor" and "processing circuitry" as used herein can refer to any of the foregoing structures or any other structure suitable for implementing the techniques described herein. Additionally, in some aspects, the functionality described herein can be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into combined codecs. Furthermore, the techniques can be implemented entirely within one or more circuit or logic elements.

[0302] The technologies disclosed herein can be implemented in a wide variety of devices or apparatuses, including wireless mobile phones, integrated circuits (ICs), or a set of ICs (e.g., chipsets). Various components, modules, or units are described in this disclosure to emphasize functional aspects of a device configured to perform the disclosed technologies, but they do not necessarily need to be implemented through different hardware units. Specifically, as described above, the various units can be combined in a codec hardware unit, or provided by a collection of interoperable hardware units (including one or more processors as described above) combined with appropriate software and / or firmware.

[0303] Examples have been described. These and other examples are within the scope of the appended claims.

Claims

1. A method for decoding video data, the method comprising: Obtain a value indicating the number of sub-images of the current image from the Picture Parameter Set (PPS) data structure used for the current image; From the PPS data structure, obtain the value of a flag indicating whether horizontal surround motion compensation for each sub-image of the specified number of sub-images is applied in inter-frame prediction; For each of the number of sub-images, if the value of the obtained flag indicates that horizontal surround motion compensation for the respective sub-image is applied in the inter-frame prediction, then the syntax elements for the respective sub-image are decoded from the PPS data structure to obtain the value for the surround offset for the respective sub-image. In response to determining that a block of the first sub-image of the current image is encoded in an inter-frame prediction mode, a motion vector for the block and a reference image for the block are determined, wherein information identifying the image width and image height for the reference image is included in the PPS data structure; Based on the wrap-around offset value obtained for the first sub-image, and based on the image width and image height for the reference image, the horizontal wrap-around position in the reference image is determined; and The predicted block for the block is determined based on the horizontal surrounding position.

2. The method according to claim 1, further comprising: Receive the syntax elements in the PPS data structure; The value of the syntax element determines whether to enable horizontal wrap motion compensation for the block. as well as In response to determining that the horizontal surround motion compensation is enabled for the block, the value for the surround offset is received.

3. The method according to claim 1, wherein, The video data includes 360-degree video data.

4. The method according to claim 1, wherein, The reference images include images in the format of filled equal rectangular projection (PERP).

5. The method according to claim 1, wherein, The left and right portions of the reference image are filled with the same material.

6. The method according to claim 1, further comprising: Receive an SPS with a Sequence Parameter Set (SPS) ID value, wherein the PPS data structure includes a syntax element that identifies the SPS ID value, and wherein the block belongs to a slice that references the PPS data structure.

7. The method according to claim 1, wherein, The reference image has an associated PPS data structure that is separate from the PPS data structure used for the current image.

8. The method according to claim 1, wherein, The horizontal wrapping position corresponds to the top left corner of the prediction block.

9. The method according to claim 1, wherein, The wrap-around offset includes a first wrap-around offset for the first sub-image, and the method further includes: The value of the second wrap-around offset for the second sub-image is obtained from the PPS data structure, wherein the first sub-image and the second sub-image are sub-images of the same image.

10. The method according to claim 1, further comprising: Determine the residual block; The residual block is added to the prediction block to determine the reconstructed block of the video data; One or more filters are applied to the reconstructed block to determine the final decoded block; as well as The output video data includes the decoded image of the final decoded block.

11. The method according to claim 1, wherein, The motion vector points to a sample outside the boundary of the reference image.

12. The method according to claim 1, wherein, The motion vector points to an interpolated sample determined by an interpolation operation that uses samples outside the boundaries of the reference image.

13. An apparatus for decoding video data, the apparatus comprising: A memory configured to store the video data; as well as One or more processors, which are implemented in a circuit and configured to: Obtain a value indicating the number of sub-images of the current image from the Picture Parameter Set (PPS) data structure used for the current image; From the PPS data structure, obtain the value of a flag indicating whether horizontal surround motion compensation for each sub-image of the specified number of sub-images is applied in inter-frame prediction; For each of the number of sub-images, if the value of the obtained flag indicates that horizontal surround motion compensation for the respective sub-image is applied in the inter-frame prediction, then the syntax elements for the respective sub-image are decoded from the PPS data structure to obtain the value for the surround offset for the respective sub-image. In response to determining that a block of the first sub-image of the current image is encoded in an inter-frame prediction mode, motion vectors for the block and a reference image for the block are determined, wherein information identifying the image width and image height for the reference image is included in the PPS data structure, and wherein the block belongs to a slice referencing the PPS data structure. Based on the wrap-around offset value obtained for the first sub-image, and based on the image width and image height for the reference image, the horizontal wrap-around position in the reference image is determined; and The predicted block for the block is determined based on the horizontal surrounding position.

14. The device according to claim 13, wherein, The one or more processors are further configured to: Receive the syntax elements in the PPS data structure; The value of the syntax element is used to determine whether horizontal wrap motion compensation is enabled for the block; and In response to determining that the horizontal surround motion compensation is enabled for the block, the value for the surround offset is received.

15. The device according to claim 13, wherein, The video data includes 360-degree video data.

16. The device according to claim 13, wherein, The reference images include images in the format of filled equal rectangular projection (PERP).

17. The device according to claim 13, wherein, The left and right portions of the reference image are filled with the same material.

18. The device according to claim 13, wherein, The one or more processors are further configured to: Receive an SPS with a Sequence Parameter Set (SPS) ID value, wherein the PPS data structure includes a syntax element that identifies the SPS ID value, and wherein the block belongs to a slice that references the PPS data structure.

19. The device according to claim 13, wherein, The reference image has an associated PPS data structure that is separate from the PPS data structure used for the current image.

20. The device according to claim 13, wherein, The horizontal wrapping position corresponds to the top left corner of the prediction block.

21. The device according to claim 13, wherein, The wrap-around offset includes a first wrap-around offset for the first sub-image, and wherein the one or more processors are further configured to: The value of the second wrap-around offset for the second sub-image is obtained from the PPS data structure, wherein the first sub-image and the second sub-image are sub-images of the same image.

22. The device according to claim 13, wherein, The one or more processors are further configured to: Determine the residual block; The residual block is added to the prediction block to determine the reconstructed block of the video data; One or more filters are applied to the reconstructed block to determine the final decoded block; as well as The output video data includes the decoded image of the final decoded block.

23. The device according to claim 13, wherein, The device includes a wireless communication device, which further includes a receiver configured to receive encoded video data.

24. The device according to claim 23, wherein, The wireless communication device includes a telephone, and the receiver is configured to demodulate a signal including the encoded video data according to a wireless communication standard.

25. The apparatus of claim 13, further comprising: A display configured to show decoded video data.

26. The device according to claim 13, wherein, The device includes one or more of a camera, a computer, a mobile device, a broadcast receiver, and a set-top box.

27. The device according to claim 13, wherein, The motion vector points to a sample outside the boundary of the reference image.

28. The device according to claim 13, wherein, The motion vector points to an interpolated sample determined by an interpolation operation that uses samples outside the boundaries of the reference image.

29. A computer-readable storage medium storing instructions, which, when executed by one or more processors, cause the one or more processors to perform the following operations: Obtain a value indicating the number of sub-images of the current image from the Picture Parameter Set (PPS) data structure used for the current image; From the PPS data structure, obtain the value of a flag indicating whether horizontal surround motion compensation for each sub-image of the specified number of sub-images is applied in inter-frame prediction; For each of the number of sub-images, if the value of the obtained flag indicates that horizontal surround motion compensation for the respective sub-image is applied in the inter-frame prediction, then the syntax elements for the respective sub-image are decoded from the PPS data structure to obtain the value for the surround offset for the respective sub-image. In response to determining that a block of the first sub-image of the current image is encoded in an inter-frame prediction mode, motion vectors for the block and a reference image for the block are determined, wherein... Information identifying the image width and image height for the reference image is included in the PPS data structure, and wherein the block belongs to a slice that references the PPS data structure; Based on the wrap-around offset value obtained for the first sub-image, and based on the image width and image height for the reference image, the horizontal wrap-around position in the reference image is determined; and The predicted block for the block is determined based on the horizontal surrounding position.

30. An apparatus for encoding video data, the apparatus comprising: A memory configured to store the video data; as well as One or more processors, which are implemented in a circuit and configured to: Determine the wrap-around offset value for each of a number of sub-images in the current image; In response to determining that a block of the first sub-image of the current image is encoded in an inter-frame prediction mode, a motion vector for the block and a reference image for the block are determined, wherein information identifying the image width and image height for the reference image is included in a Picture Parameter Set (PPS) data structure. The horizontal wrap position in the reference image is determined based on the wrap offset value obtained for the first sub-image, and based on the image width and image height for the reference image. The predicted block for the block is determined based on the horizontal surrounding position; The PPS data structure for the current image in the video data includes a value indicating the number of sub-images of the current image, a value for the surround offset for each sub-image of the current image, and a value for a flag indicating whether horizontal surround motion compensation for each sub-image is applied in inter-frame prediction; and The output includes a bitstream of encoded video data from the PPS data structure.

31. The device according to claim 30, wherein, The one or more processors are further configured to: Determine that horizontal orbital motion compensation is enabled for the block; The PPS data structure includes syntax elements, wherein the syntax elements are set to indicate a value that enables the horizontal surround motion compensation for the block; and In response to determining that the horizontal surround motion compensation is enabled for the block, the value for the surround offset is included in the PPS data structure.

32. The device according to claim 30, wherein, The video data includes 360-degree video data.

33. The device according to claim 30, wherein, The reference images include images in the format of filled equal rectangular projection (PERP).

34. The device according to claim 30, wherein, The left and right portions of the reference image are filled with the same material.

35. The device according to claim 30, wherein, The one or more processors are further configured to: The bitstream of the encoded video data includes an SPS with a Sequence Parameter Set (SPS) ID value, wherein the PPS data structure includes a syntax element that identifies the SPS ID value.

36. The device according to claim 30, wherein, The horizontal wrapping position corresponds to the top left corner of the prediction block.

37. The device according to claim 30, wherein, The wrap-around offset includes a first wrap-around offset for the first sub-image, and wherein the one or more processors are further configured to: The PPS data structure includes a value for a second wrap-around offset for the second sub-image, wherein the first sub-image and the second sub-image are sub-images of the same image.

38. The device according to claim 30, wherein, The device includes a wireless communication device, which further includes a transmitter configured to transmit a bitstream of the encoded video data.

39. The device according to claim 38, wherein, The wireless communication device includes a telephone, and the transmitter is configured to modulate a signal comprising a bitstream of the encoded video data according to a wireless communication standard.

40. The device according to claim 30, wherein, The device includes one or more of a camera, a computer, a mobile device, a broadcast receiver, and a set-top box.

Images