Method of signaling video encoding data

Abstract

The present disclosure provides systems and methods for wraparound motion compensation. An example method includes receiving a wraparound motion compensation flag, determining whether wraparound motion compensation is enabled based on the wraparound motion compensation flag, in response to determining that the wraparound motion compensation is enabled, receiving data indicative of a difference between a width of a picture and an offset used to determine a horizontal wraparound position, and performing motion compensation according to the wraparound motion compensation flag and the difference.

Description

[0001] Cross-references to related applications

[0002] This disclosure claims priority to and the benefits of priority to U.S. Provisional Patent Application No. 63 / 000,443, filed March 26, 2020. The provisional application is incorporated herein by reference in its entirety. Technical Field

[0003] This disclosure generally relates to video data processing, and more specifically, to methods and apparatus for signaling information regarding surround motion compensation, strip layout, and strip address. Background Technology

[0004] Video is a set of still images (or "frames") that capture visual information. To reduce storage memory and transmission bandwidth, video can be compressed before storage or transmission and then decompressed before display. The compression process is usually called encoding, and the decompression process is usually called decoding. There are various video coding formats that use standardized video coding techniques, the most common being prediction, transform, quantization, entropy coding, and in-loop filtering. Standardization organizations have developed video coding standards, such as the High Efficiency Video Coding (HEVC / H.265) standard, the Universal Video Coding (VVC / H.266) standard, and the AVS standard, specifying particular video coding formats. As more and more advanced video coding techniques are adopted in video standards, the coding efficiency of new video coding standards is becoming increasingly higher. Summary of the Invention

[0005] This disclosure provides a method for signaling video encoded data, the method comprising: receiving a surround motion compensation flag; determining whether to enable surround motion compensation based on the surround motion compensation flag; in response to determining that surround motion compensation is enabled, receiving data of the difference between the width of an indication image and an offset used to determine a horizontal surround position; and performing motion compensation based on the surround motion compensation flag and the difference.

[0006] This disclosure also provides a method for signaling video encoded data, the method comprising: receiving an image for encoding, wherein the image includes one or more stripes; and signaling a variable indicating the number of stripes in a video frame minus 2, within an image parameter set of the image.

[0007] This disclosure also provides a method for signaling video encoded data, the method comprising: receiving an image for encoding, wherein the image includes one or more stripes and one or more sub-images; and, in the image parameter set of the image, signaling a variable indicating the number of stripes in the image minus the number of sub-images in the image minus 1.

[0008] This disclosure also provides a method for signaling video encoded data, the method comprising: receiving an image for encoding, wherein the image includes one or more stripes; signaling a variable indicating whether an image header syntax structure of the image exists in the strip header of the one or more stripes; and signaling a strip address based on the variable.

[0009] Embodiments of this disclosure also provide a system for performing video data processing, the system comprising: a memory storing an instruction set; and a processor configured to execute the instruction set to cause the system to: receive a surround motion compensation flag; determine whether surround motion compensation is enabled based on the surround motion compensation flag; in response to determining that surround motion compensation is enabled, receive data indicating a difference between the width of an image and an offset used to determine a horizontal surround position; and perform motion compensation based on the surround motion compensation flag and the difference.

[0010] Embodiments of this disclosure also provide a system for performing video data processing, the system comprising: a memory storing a set of instructions; and a processor configured to execute the set of instructions to cause the system to: receive an image for encoding, said image comprising one or more stripes; and signal a variable in the image parameter set of said image indicating the number of stripes in a video frame minus 2.

[0011] Embodiments of this disclosure also provide a system for performing video data processing, the system comprising: a memory storing an instruction set; and a processor configured to execute the instruction set to cause the system to: receive an image for encoding, wherein the image includes one or more stripes and one or more sub-images; and, in an image parameter set of the image, signal a variable indicating the number of stripes in the image minus the number of sub-images in the image minus 1.

[0012] Embodiments of this disclosure also provide a system for performing video data processing, the system comprising: a memory storing an instruction set; and a processor configured to execute the instruction set to cause the system to: receive an image for encoding, wherein the image comprises one or more stripes; signal a variable indicating the presence of an image header syntax structure of the image in the strip header of the one or more stripes; and signal a strip address based on the variable.

[0013] Embodiments of this disclosure also provide a non-transitory computer-readable medium storing a set of instructions executable by one or more processors of the apparatus to cause the apparatus to initiate a method for performing video data processing, the method comprising: receiving a surround motion compensation flag; determining, based on the surround motion compensation flag, whether surround motion compensation is enabled; in response to determining that surround motion compensation is enabled, receiving data indicating a difference between the width of an image and an offset for determining a horizontal surround position; and performing motion compensation based on the surround motion compensation flag and the difference.

[0014] Embodiments of this disclosure also provide a non-transitory computer-readable medium storing a set of instructions executable by one or more processors of the apparatus to cause the apparatus to initiate a method for performing video data processing, the method comprising: receiving an image for encoding, wherein the image includes one or more stripes; and signaling a variable indicating the number of stripes in a video frame minus 2 in a set of image parameters of the image.

[0015] Embodiments of this disclosure also provide a non-transitory computer-readable medium storing an instruction set executable by one or more processors of a device to cause the device to initiate a method for performing video data processing, the method comprising: receiving an image for encoding, wherein the image includes one or more stripes and one or more sub-images; and signaling a variable in the image parameter set of the image indicating the number of stripes in the image minus the number of sub-images in the image minus 1.

[0016] Embodiments of this disclosure also provide a non-transitory computer-readable medium storing an instruction set executable by one or more processors of a device to cause the device to initiate a method for performing video data processing, the method comprising: receiving an image for encoding, wherein the image includes one or more stripes; signaling a variable indicating the presence of an image header syntax structure of the image in the strip header of the one or more stripes; and signaling a stripe address based on the variable. Attached Figure Description

[0017] Embodiments and aspects of this disclosure are illustrated in the following detailed description and accompanying drawings. The various features shown in the figures are not drawn to scale.

[0018] Figure 1 The structure of an example video sequence according to some embodiments of this disclosure is shown.

[0019] Figure 2A A schematic diagram of an exemplary encoding process according to some embodiments of the present disclosure is shown.

[0020] Figure 2BA schematic diagram of another example encoding process according to some embodiments of the present disclosure is shown.

[0021] Figure 3A A schematic diagram of an exemplary decoding process according to some embodiments of the present disclosure is shown.

[0022] Figure 3B A schematic diagram of another example decoding process according to some embodiments of the present disclosure is shown.

[0023] Figure 4 A block diagram of an example apparatus for encoding or decoding video according to some embodiments of the present disclosure is shown.

[0024] Figure 5A A schematic diagram of an example blending operation for generating a reconstructed rectangular projection is shown according to some embodiments of the present disclosure.

[0025] Figure 5B A schematic diagram of an example clipping operation for generating a reconstructed rectangular projection is shown according to some embodiments of the present disclosure.

[0026] Figure 6A A schematic diagram of an exemplary horizontal surround motion compensation process for a rectangular projection according to some embodiments of the present disclosure is shown.

[0027] Figure 6B A schematic diagram of an exemplary horizontal surround motion compensation process for filling a rectangular projection according to some embodiments of the present disclosure is shown.

[0028] Figure 7 Example syntax for advanced wrap offsets according to some embodiments of this disclosure is shown.

[0029] Figure 8 The semantics of example high-level wrap-around offsets are illustrated according to some embodiments of this disclosure.

[0030] Figure 9 A schematic diagram illustrating example stripes and sub-image divisions of an image according to some embodiments of the present disclosure is shown.

[0031] Figure 10 A schematic diagram illustrating example strip and sub-image segmentation of an image with different stripes and sub-images according to some embodiments of the present disclosure is shown.

[0032] Figure 11 The syntax of example image parameter sets for block mapping and strip layout according to some embodiments of this disclosure is shown.

[0033] Figure 12 The semantics of example image parameter sets for patch mapping and strip layout are illustrated according to some embodiments of this disclosure.

[0034] Figure 13 The syntax of example strip headers according to some embodiments of this disclosure is shown.

[0035] Figure 14 The semantics of example strip headers according to some embodiments of this disclosure are shown.

[0036] Figure 15 The syntax of an example improved set of image parameters according to some embodiments of this disclosure is shown.

[0037] Figure 16 The semantics of an example improved set of image parameters according to some embodiments of this disclosure are illustrated.

[0038] Figure 17 The syntax of an example set of image parameters with the variable wraparound_offset_type is shown according to some embodiments of this disclosure.

[0039] Figure 18 The semantics of an example improved set of image parameters with the variable wraparound_offset_type are shown according to some embodiments of this disclosure.

[0040] Figure 19 The syntax of an example set of image parameters with the variable num_slices_in_pic_minus2 is shown according to some embodiments of this disclosure.

[0041] Figure 20 The semantics of an example improved set of image parameters with the variable num_slices_in_pic_minus2 are shown according to some embodiments of this disclosure.

[0042] Figure 21 The syntax of an example set of image parameters with the variable num_slices_in_pic_minus_subpic_num_minus1 is shown according to some embodiments of this disclosure.

[0043] Figure 22 The semantics of an example improved set of image parameters with the variable num_slices_in_pic_minus_subpic_num_minus1 are shown according to some embodiments of this disclosure.

[0044] Figure 23 The syntax of an example updated stripe header is shown according to some embodiments of this disclosure.

[0045] Figure 24A flowchart of an example video encoding method according to some embodiments of the present disclosure is shown, the method having a variable that signals the difference between the width of a video frame and an offset used to calculate the horizontal wrap position.

[0046] Figure 25 A flowchart of an example video encoding method according to some embodiments of the present disclosure is shown, which has a variable that signals the number of stripes in a video frame minus 2.

[0047] Figure 26 A flowchart of an example video encoding method with a variable according to some embodiments of the present disclosure is shown, wherein the variable is signaled to indicate a variable representing the number of stripes in a video frame minus the number of sub-images in the video frame minus 1.

[0048] Figure 27 A flowchart of an example video encoding method with a variable indicating the presence of an image header syntax structure in the strip header of a video frame is shown, according to some embodiments of the present disclosure. Detailed Implementation

[0049] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. The following description refers to the accompanying drawings, wherein, unless otherwise stated, the same numerals in different drawings denote the same or similar elements. The embodiments set forth in the following description of the exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with aspects of this disclosure as described in the appended claims. Specific aspects of this disclosure are described below in more detail. In the event of any conflict with terms and / or definitions incorporated by reference, the terms and definitions provided herein shall prevail.

[0050] The Joint Video Experts Group (JVET) of the ITU-T Video Coding Experts Group (ITU-T VCEG) and the ISO / IEC Moving Picture Experts Group (ISO / IEC MPEG) is currently developing the Universal Video Coding (VVC / H.266) standard. The VVC standard aims to double the compression efficiency of its predecessor, the High Efficiency Video Coding (HEVC / H.265) standard. In other words, VVC aims to achieve the same subjective quality as HEVC / H.265 using half the bandwidth.

[0051] To achieve the same subjective quality as HEVC / H.265 using half the bandwidth, JVET has been developing techniques other than HEVC using the Joint Exploratory Model (JEM) reference software. With the incorporation of coding techniques into JEM, JEM achieves higher coding performance than HEVC.

[0052] The VVC standard is a recent development and continues to include more coding techniques that provide better compression performance. VVC is based on a hybrid video coding system that has been used in modern video compression standards such as HEVC, H.264 / AVC, MPEG2, and H.263.

[0053] Video is a set of still images (or “frames”) arranged in chronological order to store visual information. These images can be captured and stored in chronological order using video capture devices (e.g., cameras), and displayed in a time-series using video playback devices (e.g., televisions, computers, smartphones, tablets, video players, or any end-user terminal with a display capability). Furthermore, in some applications, video capture devices can transmit captured video in real time to video playback devices (e.g., computers with monitors), such as for surveillance, conferencing, or live broadcasting.

[0054] To reduce the storage space and transmission bandwidth required for such applications, video can be compressed before storage and transmission, and decompressed before display. Compression and decompression can be implemented by software executed by a processor (e.g., a processor in a general-purpose computer) or dedicated hardware. The module used for compression is typically called an "encoder," and the module used for decompression is typically called a "decoder." Encoders and decoders can be collectively referred to as a "codec." Encoders and decoders can be implemented as any of a variety of suitable hardware, software, or combinations thereof. For example, hardware implementations of encoders and decoders can include circuits such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, or any combination thereof. Software implementations of encoders and decoders can include program code embedded in a computer-readable medium, computer-executable instructions, firmware, or any suitable computer-implemented algorithm or process. Video compression and decompression can be implemented using various algorithms or standards, such as MPEG-1, MPEG-2, MPEG-4, the H.26x series, etc. In some applications, a codec can decompress video from a first encoding standard and recompress the decompressed video using a second encoding standard; in this case, the codec can be referred to as a "transcoder."

[0055] Video encoding processes identify and retain useful information that can be used to reconstruct the image, while ignoring unimportant reconstruction information. If ignoring unimportant information prevents complete reconstruction, such an encoding process can be called "lossy." Otherwise, it can be called "lossless." Most encoding processes are lossy, a trade-off to reduce required storage space and transmission bandwidth.

[0056] In many cases, useful information about an encoded image (referred to as the "current image") includes changes relative to a reference image (e.g., a previously encoded and reconstructed image). Such changes can include variations in pixel position, brightness, or color, with positional changes being the most important. The positional changes of a set of pixels representing an object can reflect the object's movement between the reference and current images.

[0057] An image encoded without referencing another image (i.e., it is its own reference image) is called an "I-image". An image encoded using a previous image as a reference image is called a "P-image", and an image encoded using both a previous image and a future image as reference images is called a "B-image" (the reference is "bidirectional").

[0058] Figure 1 The structure of an example video sequence 100 according to some embodiments of the present disclosure is shown. The video sequence 100 may be live video or video that has been captured and archived. The video 100 may be real-life video, computer-generated video (e.g., computer game video), or a combination of both (e.g., real video with augmented reality effects). The video sequence 100 may be input from a video capture device (e.g., a camera), a video archive containing previously captured video (e.g., a video file stored on a storage device), or a video feed interface (e.g., a video broadcast transceiver) receiving video from a video content provider.

[0059] like Figure 1 As shown, video sequence 100 may include a series of images arranged temporally along a timeline, including images 102, 104, 106, and 108. Images 102-106 are consecutive, with more images between images 106 and 108. Figure 1 In this diagram, image 102 is an I-image, and its reference image is image 102 itself. Image 104 is a P-image, and its reference image is image 102, as indicated by the arrow. Image 106 is a B-image, and its reference images are images 104 and 108, as indicated by the arrow. In some embodiments, the reference image of an image (e.g., image 104) may not immediately precede or follow the image. For example, the reference image of image 104 may be an image preceding image 102. It should be noted that the reference images of images 102-106 are merely examples, and this disclosure does not limit the scope to such cases. Figure 1 An example of the reference image shown.

[0060] Typically, due to the computational complexity of encoding and decoding tasks, video codecs do not encode or decode the entire image at once. Instead, they can segment the image into basic segments and encode or decode each segment sequentially. In this disclosure, such basic segments are referred to as basic processing units (“BPUs”). For example, Figure 1Structure 110 illustrates an example structure of an image (e.g., any of images 102-108) from video sequence 100. In structure 110, the image is divided into 4×4 basic processing units, whose boundaries are shown as dashed lines. In some embodiments, the basic processing unit may be referred to as a “macroblock” in some video coding standards (e.g., MPEG family, H.261, H.263, or H.264 / AVC), or as a “coding tree unit” (“CTU”) in some other video coding standards (e.g., H.265 / HEVC or H.266 / VVC). The basic processing unit may have a variable size in the image, such as 128×128, 64×64, 32×32, 16×16, 4×8, 16×32, or pixels of any shape and size. The size and shape of the basic processing units for the image can be chosen based on a balance between coding efficiency and the level of detail to be maintained within the basic processing units.

[0061] A basic processing unit can be a logical unit that may include a set of different types of video data stored in computer memory (e.g., in a video frame buffer). For example, a basic processing unit for a color image may include a luminance component (Y) representing achromatic luminance information, one or more chrominance components (e.g., Cb and Cr) representing color information, and associated syntax elements, where the luminance and chrominance components may have the same size as the basic processing unit. In some video coding standards (e.g., H.265 / HEVC or H.266 / VVC), the luminance and chrominance components may be referred to as “code tree blocks” (“CTBs”). Any operation performed on a basic processing unit may be performed repeatedly on each of its luminance and chrominance components.

[0062] Video encoding involves multiple operational stages, examples of which are as follows: Figure 2A-2B and Figures 3A-3BAs shown. For each stage, the size of the basic processing unit may still be too large for the processing, and therefore can be further divided into segments referred to herein as "basic processing subunits". In some embodiments, the basic processing subunit may be referred to as a "block" in some video coding standards (e.g., MPEG family, H.261, H.263, or H.264 / AVC), or as a "coding unit" ("CU") in some other video coding standards (e.g., H.265 / HEVC or H.266 / VVC). The basic processing subunit may have the same size as the basic processing unit or a smaller size. Similar to the basic processing unit, the basic processing subunit is also a logical unit that may include a set of different types of video data (e.g., Y, Cb, Cr, and associated syntax elements) stored in computer memory (e.g., in a video frame buffer). Any operation performed on the basic processing subunit may be repeated for each of its luminance and chrominance components. It should be noted that this division can be performed to further levels as needed for processing. It should also be noted that different schemes can be used to divide the basic processing units for different stages.

[0063] For example, in the pattern decision-making stage (examples of which are in...) Figure 2B As shown, the encoder can decide which prediction mode (e.g., intra-frame prediction or inter-frame prediction) to use for a basic processing unit, which may be too large to make such a decision. The encoder can divide the basic processing unit into multiple basic processing subunits (e.g., CUs in H.265 / HEVC or H.266 / VVC) and determine the prediction type for each individual basic processing subunit.

[0064] For another example, in the prediction phase (the example is in...) Figure 2A-2B As shown in the diagram, the encoder can perform prediction operations at the level of a basic processing subunit (e.g., a CU). However, in some cases, the basic processing subunit may still be too large to handle. The encoder can further divide the basic processing subunit into smaller segments (e.g., referred to as "prediction blocks" or "PBs" in H.265 / HEVC or H.266 / VVC) at which prediction operations can be performed.

[0065] For another example, in the transformation phase (the example of which is in...) Figure 2A-2BAs shown in the diagram, the encoder can perform transformation operations on residual basic processing subunits (e.g., CUs). However, in some cases, the basic processing subunits may still be too large to process. The encoder can further divide the basic processing subunits into smaller segments (e.g., referred to as "transform blocks" or "TBs" in H.265 / HEVC or H.266 / VVC), at which level transformation operations can be performed. It is important to note that the partitioning scheme of the same basic processing subunit can differ between the prediction and transformation phases. For example, in H.265 / HEVC or H.266 / VVC, the prediction blocks and transform blocks of the same CU can have different sizes and numbers.

[0066] exist Figure 1 In structure 110, the basic processing unit 112 is further divided into 3×3 basic processing sub-units, the boundaries of which are shown by dashed lines. Different basic processing units of the same image can be divided into basic processing sub-units in different schemes.

[0067] In some implementations, to provide parallel processing capabilities and fault tolerance for video encoding and decoding, an image can be divided into regions for processing, such that the encoding or decoding process for a given region of the image can be independent of information from any other region of the image. In other words, each region of the image can be processed independently. By doing so, the codec can process different regions of the image in parallel, thereby improving encoding efficiency. Furthermore, when data in one region is corrupted during processing or lost during network transmission, the codec can correctly encode or decode other regions of the same image without relying on the corrupted or lost data, thus providing fault tolerance. In some video coding standards, images can be divided into different types of regions. For example, H.265 / HEVC and H.266 / VVC provide two types of regions: “slices” and “tiles.” It should also be noted that different images in the video sequence 100 can have different partitioning schemes for dividing the image into regions.

[0068] For example, in Figure 1 In the diagram, structure 110 is divided into three regions 114, 116, and 118, whose boundaries are shown as solid lines within structure 110. Region 114 comprises four basic processing units. Regions 116 and 118 each comprise six basic processing units. It should be noted that... Figure 1 The basic processing unit, basic processing subunit, and structural region of 110 are merely examples, and this disclosure does not limit its embodiments.

[0069] Figure 2A A schematic diagram of an exemplary encoding process 200A according to an embodiment of the present disclosure is shown. For example, the encoding process 200A may be performed by an encoder. Figure 2AAs shown, the encoder can encode the video sequence 202 into a video bitstream 228 according to process 200A. Similar to... Figure 1 Video sequence 100 and video sequence 202 may include a set of images arranged in chronological order (referred to as "original images"). Similar to... Figure 1 In structure 110, each raw image of video sequence 202 can be divided into basic processing units, basic processing subunits, or regions by an encoder for processing. In some embodiments, the encoder can perform process 200A at the level of basic processing units for each raw image of video sequence 202. For example, the encoder can perform process 200A iteratively, wherein the encoder can encode basic processing units in one iteration of process 200A. In some embodiments, the encoder can perform process 200A in parallel for regions (e.g., regions 114-118) of each raw image of video sequence 202.

[0070] refer to Figure 2A The encoder feeds the basic processing unit (referred to as the "raw BPU") of the original image of video sequence 202 to prediction stage 204 to generate prediction data 206 and prediction BPU 208. The encoder can subtract the predicted BPU 208 from the raw BPU to generate residual BPU 210. The encoder can feed residual BPU 210 to transform stage 212 and quantization stage 214 to generate quantization transform coefficients 216. The encoder can feed prediction data 206 and quantization transform coefficients 216 to binary encoding stage 226 to generate video bitstream 228. Components 202, 204, 206, 208, 210, 212, 214, 216, 226, and 228 can be referred to as the "forward path". During process 200A, after quantization stage 214, the encoder can feed quantization transform coefficients 216 to inverse quantization stage 218 and inverse transform stage 220 to generate reconstructed residual BPU 222. The encoder can add the reconstructed residual BPU 222 to the predicted BPU 208 to generate a prediction reference 224, which is used in the prediction stage 204 of the next iteration of process 200A. Components 218, 220, 222, and 224 of process 200A can be referred to as the "reconstruction path". The reconstruction path can be used to ensure that both the encoder and decoder use the same reference data for prediction.

[0071] The encoder can iteratively execute process 200A to encode each raw BPU (in the forward path) of the original image and generate a prediction reference 224 for encoding the next raw BPU (in the reconstruction path) of the original image. After encoding all raw BPUs of the original image, the encoder can continue to encode the next image in the video sequence 202.

[0072] Referring to process 200A, the encoder may receive a video sequence 202 generated by a video acquisition device (e.g., a camera). As used herein, the term "receive" can refer to any action that receives, inputs, acquires, retrieves, obtains, reads, accesses, or is used for inputting data in any manner.

[0073] In prediction phase 204, during the current iteration, the encoder can receive the original BPU and prediction reference 224, and perform prediction operations to generate prediction data 206 and prediction BPU 208. Prediction reference 224 can be generated from the reconstruction path of previous iterations of process 200A. The purpose of prediction phase 204 is to reduce information redundancy by extracting prediction data 206 from prediction data 206 and prediction reference 224 that can be used to reconstruct the original BPU into prediction BPU 208.

[0074] Ideally, the predicted BPU 208 should be identical to the original BPU. However, due to non-ideal prediction and reconstruction operations, the predicted BPU 208 is typically slightly different from the original BPU. To record these differences, the encoder can subtract the predicted BPU 208 from the original BPU to generate the residual BPU 210. For example, the encoder can subtract the value of the corresponding pixel in the predicted BPU 208 (e.g., grayscale or RGB value) from the pixel value of the original BPU. Each pixel in the residual BPU 210 can have a residual value as the result of this subtraction between the corresponding pixel in the original BPU and the predicted BPU 208. Compared to the original BPU, the predicted data 206 and the residual BPU 210 can have fewer bits, but they can be used to reconstruct the original BPU without a significant quality degradation. Thus, the original BPU is compressed.

[0075] To further compress the residual BPU 210, in the transform stage 212, the encoder can reduce its spatial redundancy by decomposing the residual BPU 210 into a set of two-dimensional “base patterns”. Each base pattern is associated with “transform coefficients”. The base patterns can have the same size (e.g., the size of the residual BPU 210), and each base pattern can represent the frequency (e.g., the frequency of brightness variation) component of the residual BPU 210. None of the base patterns can be reproduced from any combination (e.g., a linear combination) of any other base patterns. In other words, the decomposition can decompose the variation of the residual BPU 210 into the frequency domain. This decomposition is analogous to the discrete Fourier transform of a function, where the base patterns are analogous to the base functions of the discrete Fourier transform (e.g., trigonometric functions), and the transform coefficients are analogous to the coefficients associated with the base functions.

[0076] Different transform algorithms can use different base patterns. Various transform algorithms, such as discrete cosine transform, discrete sine transform, etc., can be used in transform stage 212. The transform at transform stage 212 is reversible. That is, the encoder can recover the residual BPU 210 through the inverse operation of the transform (called the "inverse transform"). For example, to recover the pixels of the residual BPU 210, the inverse transform can be to multiply the values ​​of the corresponding pixels of the base pattern by the corresponding correlation coefficients and sum the products to produce a weighted sum. For video coding standards, both the encoder and decoder can use the same transform algorithm (and therefore have the same base pattern). Therefore, the encoder can only record the transform coefficients, from which the decoder can reconstruct the residual BPU 210 without receiving the base pattern from the encoder. Compared to the residual BPU 210, the transform coefficients can have fewer bits, but they can be used to reconstruct the residual BPU 210 without significant quality degradation. Therefore, the residual BPU 210 is further compressed.

[0077] The encoder can further compress the transform coefficients during the quantization stage 214. During the transform process, different fundamental patterns can represent different frequencies of change (e.g., brightness change frequencies). Because the human eye is generally better at recognizing low-frequency changes, the encoder can ignore information about high-frequency changes without causing significant quality degradation in decoding. For example, in the quantization stage 214, the encoder can generate quantized transform coefficients 216 by dividing each transform coefficient by an integer value (called the “quantization parameter”) and rounding the quotient to its nearest integer. After such an operation, some transform coefficients of the high-frequency fundamental patterns can be converted to zero, and the transform coefficients of the low-frequency fundamental patterns can be converted to smaller integers. The encoder can ignore the zero-valued quantized transform coefficients 216, thus further compressing the transform coefficients. This quantization process is also reversible, where the quantized transform coefficients 216 can be reconstructed into transform coefficients in the inverse operation of quantization (called “inverse quantization”).

[0078] Because the encoder ignores the remainder of the division during rounding, quantization stage 214 can be lossy. Typically, quantization stage 214 contributes the most to information loss in process 200A. The greater the information loss, the fewer bits are required for the quantization transform coefficients 216. To obtain different levels of information loss, the encoder can use different quantization parameter values ​​or any other parameter of the quantization process.

[0079] In the binary encoding stage 226, the encoder can encode the prediction data 206 and the quantization transform coefficients 216 using binary encoding techniques, such as entropy coding, variable-length coding, arithmetic coding, Huffman coding, context-adaptive binary arithmetic coding, or any other lossless or lossy compression algorithm. In some embodiments, in addition to the prediction data 206 and the quantization transform coefficients 216, the encoder can encode other information in the binary encoding stage 226, such as the prediction mode used in the prediction stage 204, the parameters of the prediction operation, the transform type at the transform stage 212, the parameters of the quantization process (e.g., quantization parameters), encoder control parameters (e.g., bit rate control parameters), etc. The encoder can use the output data of the binary encoding stage 226 to generate a video bitstream 228. In some embodiments, the video bitstream 228 can be further packaged for network transmission.

[0080] Following the reconstruction path of process 200A, in the inverse quantization stage 218, the encoder can perform inverse quantization on the quantized transform coefficients 216 to generate reconstructed transform coefficients. In the inverse transform stage 220, the encoder can generate a reconstruction residual BPU 222 based on the reconstructed transform coefficients. The encoder can add the reconstruction residual BPU 222 to the prediction BPU 208 to generate a prediction reference 224 that will be used in the next iteration of process 200A.

[0081] It should be noted that other variations of process 200A can be used to encode video sequence 202. In some embodiments, the stages of process 200A may be performed by the encoder in different orders. In some embodiments, one or more stages of process 200A may be combined into a single stage. In some embodiments, a single stage of process 200A may be divided into multiple stages. For example, transform stage 212 and quantization stage 214 may be combined into a single stage. In some embodiments, process 200A may include additional stages. In some embodiments, process 200A may be omitted. Figure 2A One or more stages in the process.

[0082] Figure 2B A schematic diagram of another example encoding process 200B according to an embodiment of the present disclosure is shown. Process 200B can be modified from process 200A. For example, process 200B can be used by an encoder conforming to a hybrid video coding standard (e.g., H.26x series). Compared to process 200A, the forward path of process 200B further includes a mode decision stage 230, and divides the prediction stage 204 into a spatial prediction stage 2042 and a temporal prediction stage 2044. The reconstruction path of process 200B also additionally includes a loop filtering stage 232 and a buffer 234.

[0083] Generally, prediction techniques can be categorized into two types: spatial prediction and temporal prediction. Spatial prediction (e.g., intra-frame image prediction or "intra-prediction") uses pixels from one or more already encoded neighboring BPUs in the same image to predict the current BPU. That is, the prediction reference 224 in spatial prediction can include neighboring BPUs. Spatial prediction can reduce the inherent spatial redundancy of images. Temporal prediction (e.g., inter-image prediction or "inter-frame prediction") uses regions from one or more already encoded images to predict the current BPU. That is, the prediction reference 224 in temporal prediction can include encoded images. Temporal prediction can reduce the inherent temporal redundancy of images.

[0084] In reference process 200B, during the forward path, the encoder performs prediction operations in spatial prediction phase 2042 and temporal prediction phase 2044. For example, in spatial prediction phase 2042, the encoder may perform intra-frame prediction. For the original BPU of the encoded image, prediction reference 224 may include one or more adjacent BPUs that have been encoded (in the forward path) and reconstructed (in the reconstruction path) in the same image. The encoder can generate the predicted BPU 208 by interpolating adjacent BPUs. Interpolation techniques may include, for example, linear interpolation or interpolation, polynomial interpolation or interpolation, etc. In some embodiments, the encoder may perform interpolation at the pixel level, for example, by interpolating to predict the value of the corresponding pixel for each pixel of BPU 208. The adjacent BPUs used for interpolation may be located in various directions relative to the original BPU, such as in the vertical direction (e.g., at the top of the original BPU), the horizontal direction (e.g., to the left of the original BPU), the diagonal direction (e.g., at the lower left, lower right, upper left, or upper right of the original BPU), or any direction defined in the video coding standard used. For intra-frame prediction, prediction data 206 may include, for example, the location (e.g., coordinates) of the neighboring BPUs used, the size of the neighboring BPUs used, the interpolation parameters, the orientation of the neighboring BPUs used relative to the original BPU, etc.

[0085] In another example, during the temporal prediction phase 2044, the encoder can perform inter-frame prediction. For the original BPU of the current image, the prediction reference 224 can include one or more images (referred to as "reference images") that have been encoded (in the forward path) and reconstructed (in the reconstruction path). In some embodiments, the reference images can be encoded and reconstructed on a BPU-by-BPU basis. For example, the encoder can add the reconstructed residual BPU 222 to the prediction BPU 208 to generate a reconstructed BPU. When all reconstructed BPUs for the same image have been generated, the encoder can generate a reconstructed image as the reference image. The encoder can perform a "motion estimation" operation to search for matching regions within the range of the reference image (referred to as a "search window"). The position of the search window in the reference image can be determined based on the position of the original BPU in the current image. For example, the search window can be centered at a location in the reference image that has the same coordinates as the original BPU in the current image and can extend outward by a predetermined distance. When the encoder identifies a region in the search window that resembles the original BPU (e.g., by using a PEL recursive algorithm, a block matching algorithm, etc.), the encoder can determine such a region as a matching region. The matching region can have a different size than the original BPU (e.g., less than, equal to, greater than, or with a different shape). This is because the reference image and the current image are temporally separated on the timeline (e.g., as...). Figure 1 As shown in the image, the matching region can be considered to have "moved" to the original BPU's location over time. The encoder can record the direction and distance of this movement as a "motion vector." When using multiple reference images (e.g., such as...), Figure 1 In image 106, the encoder can search for matching regions and determine the associated motion vector for each reference image. In some embodiments, the encoder can assign weights to the pixel values ​​of the matching regions of each matching reference image.

[0086] Motion estimation can be used to identify various types of motion, such as translation, rotation, scaling, etc. For inter-frame prediction, prediction data 206 may include, for example, the location (e.g., coordinates) of the matching region, the motion vector associated with the matching region, the number of reference images, the weights associated with the reference images, etc.

[0087] To generate the predicted BPU 208, the encoder can perform a "motion compensation" operation. Motion compensation can be used to reconstruct the predicted BPU 208 based on the predicted data 206 (e.g., motion vectors) and the predicted reference 224. For example, the encoder can move a matching region of the reference image according to the motion vectors, where the encoder can predict the original BPU of the current image. When using multiple reference images (e.g., such as...), Figure 1In image 106), the encoder can move the matching region of the reference image based on the individual motion vectors and average pixel values ​​of the matching region. In some embodiments, if the encoder has already assigned weights to the pixel values ​​of the matching regions of the respective matching reference images, the encoder can add the weighted sums of the pixel values ​​of the moved matching regions.

[0088] In some embodiments, inter-frame prediction can be unidirectional or bidirectional. Unidirectional inter-frame prediction can use one or more reference images in the same temporal direction relative to the current image. For example, Figure 1 Image 104 in the diagram is a one-way inter-frame prediction image, where the reference image (i.e., image 102) precedes image 104. Two-way inter-frame prediction can use one or more reference images in two temporal directions relative to the current image. For example, Figure 1 Image 106 in the image is a bidirectional inter-frame prediction image, in which the reference image (i.e., images 104 and 08) is relative to image 104 in two temporal directions.

[0089] Referring again to the forward path of process 200B, after spatial prediction 2042 and temporal prediction stages 2044, in the mode decision stage 230, the encoder can select a prediction mode (e.g., one of intra-frame prediction or inter-frame prediction) for the current iteration of process 200B. For example, the encoder can perform a rate distortion optimization technique, whereby the encoder selects a prediction mode based on the bit rate of the candidate prediction modes and the distortion of the reconstructed reference image under the candidate prediction modes to minimize the value of the cost function. Based on the selected prediction mode, the encoder can generate the corresponding prediction BPU 208 and prediction data 206.

[0090] In the reconstruction path of process 200B, if intra-frame prediction mode has been selected in the forward path, the encoder can directly feed prediction reference 224 (e.g., the current BPU that has been encoded and reconstructed in the current image) to spatial prediction stage 2042 for later use (e.g., for interpolating the next BPU of the current image) after generating prediction reference 224. If inter-frame prediction mode has been selected in the forward path, the encoder can feed prediction reference 224 (e.g., the current image where all BPUs have been encoded and reconstructed) to loop filter stage 232 after generating prediction reference 224. In this stage, the encoder can apply loop filters to prediction reference 224 to reduce or eliminate distortions introduced by inter-frame prediction (e.g., block artifacts). The encoder can apply various loop filter techniques at loop filter stage 232, such as deblocking, adaptive sampling compensation, adaptive loop filtering, etc. The loop-filtered reference image can be stored in buffer 234 (or "decoded image buffer") for later use (e.g., as an inter-frame prediction reference image for future images of video sequence 202). The encoder can store one or more reference images in buffer 234 for use at temporal prediction stage 2044. In some embodiments, the encoder can encode parameters of the loop filter (e.g., loop filter strength) as well as quantization transform coefficients 216, prediction data 206, and other information at binary encoding stage 226.

[0091] Figure 3A A schematic diagram of an exemplary decoding process 300A according to an embodiment of the present disclosure is shown. Process 300A may be corresponding to Figure 2A The compression process 200A in the video stream is followed by the decompression process. In some embodiments, process 300A can be similar to the reconstruction path of process 200A. The decoder can decode the video bitstream 228 into video stream 304 according to process 300A. Video stream 304 can be very similar to video sequence 202. However, due to information loss during compression and decompression (e.g., Figure 2A-2B In the quantization stage 214), the video stream 304 is typically different from the video sequence 202. Similar to... Figure 2A-2B In processes 200A and 200B, the decoder can perform process 300A at the basic processing unit (BPU) level for each image encoded in the video bitstream 228. For example, the decoder can perform process 300A iteratively, where the decoder can decode the basic processing unit in one iteration of process 300A. In some embodiments, the decoder can perform process 300A in parallel for a region (e.g., region 114-118) of each image encoded in the video bitstream 228.

[0092] like Figure 3AAs shown, the decoder can feed a portion of the video bitstream 228 associated with a basic processing unit (referred to as the "encoded BPU") of the encoded image to a binary decoding stage 302, where the decoder can decode this portion into prediction data 206 and quantization transform coefficients 216. The decoder can feed the quantization transform coefficients 216 to an inverse quantization stage 218 and an inverse transform stage 220 to generate a reconstruction residual BPU 222. The decoder can feed the prediction data 206 to a prediction stage 204 to generate a prediction BPU 208. The decoder can add the reconstruction residual BPU 222 to the prediction BPU 208 to generate a prediction reference 224. In some embodiments, the prediction reference 224 can be stored in a buffer (e.g., a decoded image buffer in computer memory). The decoder can feed the prediction reference 224 to the prediction stage 204 for performing a prediction operation in the next iteration of process 300A.

[0093] The decoder can iteratively execute process 300A to decode each encoded BPU of the encoded image and generate a prediction reference 224 for the next encoded BPU of the encoded image. After decoding all encoded BPUs of the encoded image, the decoder can output the image to video stream 304 for display and continue decoding the next encoded image in video bit stream 228.

[0094] In the binary decoding stage 302, the decoder can perform the inverse operation of the binary encoding technique used by the encoder (e.g., entropy coding, variable-length coding, arithmetic coding, Huffman coding, context-adaptive binary arithmetic coding, or any other lossless compression algorithm). In some embodiments, in addition to the prediction data 206 and the quantized transform coefficients 216, the decoder can decode other information in the binary decoding stage 302, such as the prediction mode, parameters of the prediction operation, transform type, parameters of the quantization process (e.g., quantization parameters), encoder control parameters (e.g., bit rate control parameters), etc. In some embodiments, if the video bitstream 228 is transmitted over a network in packets, the decoder can unpack it before feeding the video bitstream 228 to the binary decoding stage 302.

[0095] Figure 3B A schematic diagram of another example decoding process 300B according to an embodiment of the present disclosure is shown. Process 300B can be modified from process 300A. For example, process 300B can be used by a decoder conforming to a hybrid video coding standard (e.g., H.26x series). Compared to process 300A, process 300B additionally divides the prediction stage 204 into a spatial prediction stage 2042 and a temporal prediction stage 2044, and additionally includes a loop filtering stage 232 and a buffer 234.

[0096] In process 300B, for the encoding basic processing unit (referred to as the "current BPU") of the decoded encoded image (referred to as the "current image"), the prediction data 206 decoded by the decoder from the binary decoding stage 302 can include various types of data depending on the prediction mode used by the encoder to encode the current BPU. For example, if the encoder uses intra-frame prediction to encode the current BPU, the prediction data 206 can include prediction mode indicators (e.g., flag values) that indicate intra-frame prediction, parameters of the intra-frame prediction operation, etc. Parameters of the intra-frame prediction operation can include, for example, the positions (e.g., coordinates) of one or more neighboring BPUs used as references, the sizes of neighboring BPUs, interpolation parameters, the orientation of neighboring BPUs relative to the original BPU, etc. For another example, if the current BPU is encoded by inter-frame prediction used by the encoder, the prediction data 206 can include prediction mode indicators (e.g., flag values) that indicate inter-frame prediction, parameters of the inter-frame prediction operation, etc. The parameters of the inter-frame prediction operation may include, for example, the number of reference images associated with the current BPU, the weights associated with the reference images respectively, the positions (e.g., coordinates) of one or more matching regions in the corresponding reference images, and one or more motion vectors associated with the matching regions respectively.

[0097] Based on the prediction mode indicator, the decoder can decide whether to perform spatial prediction (e.g., intra-frame prediction) in the spatial prediction phase 2042 or temporal prediction (e.g., inter-frame prediction) in the temporal prediction phase 2044. The details of performing this spatial or temporal prediction are detailed in [the relevant section]. Figure 2B As described herein, it will not be repeated below. After performing such spatial or temporal prediction, the decoder can generate a predicted BPU 208, which can be added to the predicted BPU 208 and the reconstructed residual BPU 222 to generate a prediction reference 224, as shown below. Figure 3A As described in [the text].

[0098] In process 300B, the decoder can feed prediction reference 224 to either spatial prediction stage 2042 or temporal prediction stage 2044 for performing prediction operations in the next iteration of process 300B. For example, if intra-frame prediction is used to decode the current BPU in spatial prediction stage 2042, the decoder can feed prediction reference 224 directly to spatial prediction stage 2042 for later use (e.g., for interpolating the next BPU of the current image) after generating prediction reference 224 (e.g., the decoded current BPU). If inter-frame prediction is used to decode the current BPU in temporal prediction stage 2044, the encoder can feed prediction reference 224 to loop filter stage 232 to reduce or eliminate distortion (e.g., block artifacts) after generating prediction reference 224 (e.g., a reference image where all BPUs are decoded). The decoder can, as follows: Figure 2BThe illustrated method applies a loop filter to prediction reference 224. The loop-filtered reference image can be stored in buffer 234 (e.g., a decoded image buffer in computer memory) for later use (e.g., as an inter-prediction reference image for future encoded images of video bitstream 228). The decoder can store one or more reference images in buffer 234 for use at temporal prediction stage 2044. In some embodiments, the prediction data can further include parameters of the loop filter (e.g., loop filter strength) when the prediction mode indicator of prediction data 206 indicates that inter-frame prediction is used to encode the current BPU. The reconstructed image from buffer 234 can also be sent to a display, such as a TV, PC, smartphone, or tablet, for viewing by an end user.

[0099] There can be four types of loop filters. For example, loop filters can include deblocking filters, sample adaptive offset (“SAO”) filters, luminance mapping with chroma scaling (“LMCS”) filters, and adaptive loop filters (“ALF”). The order in which the four types of loop filters are applied can be LMCS filter, deblocking filter, SAO filter, and ALF. An LMCS filter can include two main components. The first component is an in-loop mapping of the luminance component based on an adaptive piecewise linear model. The second component can be used for the chroma component, and luminance-related chroma residual scaling can be applied.

[0100] Figure 4 This is a block diagram of an example apparatus 400 for encoding or decoding video according to embodiments of the present disclosure. Figure 4 As shown, device 400 may include processor 402. When processor 402 executes the instructions described herein, device 400 may become a dedicated machine for video encoding or decoding. Processor 402 may be any type of circuit capable of manipulating or processing information. For example, processor 402 may include any number of central processing units (or “CPU”), graphics processing units (or “GPU”), neural processing units (“NPU”), microcontroller units (“MCU”), optical processors, programmable logic controllers, microprocessors, digital signal processors, intellectual property (IP) cores, programmable logic arrays (PLAs), programmable array logic (PALs), general-purpose array logic (GALs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), system-on-a-chip (SoCs), application-specific integrated circuits (ASICs), and any combination thereof. In some embodiments, processor 402 may also be a group of processors grouped into individual logic components. For example, such as Figure 4 As shown, processor 402 may include multiple processors, including processor 402a, processor 402b and processor 402n.

[0101] The device 400 may also include a memory 404 configured to store data (e.g., instruction sets, computer code, intermediate data, etc.). For example, such as Figure 4 As shown, the stored data may include program instructions (e.g., for implementing stages in processes 200A, 200B, 300A, or 300B) and data for processing (e.g., video sequence 202, video bitstream 228, or video stream 304). Processor 402 can access the program instructions and data for processing (e.g., via bus 410) and execute the program instructions to perform operations or manipulations on the data for processing. Memory 404 may include a high-speed random access memory device or a non-volatile memory device. In some embodiments, memory 404 may include any combination of any number of random access memories (RAM), read-only memories (ROM), optical discs, magnetic disks, hard disks, solid-state drives, flash drives, secure digital cards (SD cards), memory sticks, compact flash memory (CF cards), etc. Memory 404 may also be a group of memories grouped into single logical components. Figure 4 (Not shown in the image).

[0102] Bus 410 may be a communication device for transmitting data between components within device 400, such as an internal bus (e.g., CPU-memory bus), an external bus (e.g., a Universal Serial Bus port, a Peripheral Component Interconnect Fast Port), or the like.

[0103] For ease of explanation and to avoid ambiguity, the processor 402 and other data processing circuitry are collectively referred to as "data processing circuitry" in this disclosure. The data processing circuitry may be implemented entirely in hardware, or as a combination of software, hardware, or firmware. Furthermore, the data processing circuitry may be a single, separate module, or may be wholly or partially integrated into any other component of the device 400.

[0104] The device 400 may also include a network interface 406 to provide wired or wireless communication with a network (e.g., the Internet, intranet, local area network, mobile communication network, etc.). In some embodiments, the network interface 406 may include any combination of any number of network interface controllers (NICs), radio frequency (RF) modules, transceivers, transceivers, modems, routers, gateways, wired network adapters, wireless network adapters, Bluetooth adapters, infrared adapters, near field communication (“NFC”) adapters, cellular network chips, etc.

[0105] In some embodiments, optionally, the device 400 may further include a peripheral interface 408 to provide connectivity to one or more peripheral devices. Figure 4As shown, peripheral devices may include, but are not limited to, cursor control devices (e.g., a mouse, touchpad, or touch screen), keyboards, displays (e.g., cathode ray tube displays, liquid crystal displays, or light-emitting diode displays), video input devices (e.g., a camera or an input interface coupled to a video file), and the like.

[0106] It should be noted that a video codec (e.g., a codec that performs process 200A, 200B, 300A, or 300B) may be implemented as any combination of any software or hardware modules in apparatus 400. For example, some or all of the stages of process 200A, 200B, 300A, or 300B may be implemented as one or more software modules of apparatus 400, such as program instances that may be loaded into memory 404. For another example, some or all of the stages of process 200A, 200B, 300A, or 300B may be implemented as one or more hardware modules of apparatus 400, such as dedicated data processing circuits (e.g., FPGA, ASIC, NPU, etc.).

[0107] In the quantization and inverse quantization functional blocks (e.g., Figure 2A or Figure 2B quantization 214 and inverse quantization 218 of, Figure 3A or Figure 3B inverse quantization 218 of), a quantization parameter (QP) is used to determine the amount of quantization (and inverse quantization) applied to the prediction residual. The initial QP value used to encode an image or slice may be signaled at a higher level, for example, using the init_qp_minus26 syntax element in the Picture Parameter Set (PPS) and the slice_qp_delta syntax element in the slice header. In addition, the QP value can be adapted at the local level of each CU using a delta QP value sent at the granularity of the quantization group.

[0108] The equirectangular projection ("ERP") format is a commonly used projection format for representing 360-degree videos and images. The projection maps meridians to equally spaced vertical straight lines, and maps circles of latitude to equally spaced horizontal straight lines. Because the relationship between the position of an image pixel on the map and its corresponding geographic position on the sphere is particularly simple, ERP is one of the most common projections for 360-degree videos and images.

[0109] The algorithm description of projection format conversion and video quality metrics output by JVET gives the coordinate conversion and introduction between ERP and sphere. For 2D to 3D coordinate conversion, given a sampling position (m, n), (u, v) can be calculated based on the following formulas (1) and (2).

[0110] u=(m+0.5) / W, 0≤m<W Formula (1)

[0111] v=(n+0.5) / H, 0≤n<H Equation (2)

[0112] Then, the longitude and latitude (φ, θ) in the sphere can be calculated from (u, v) based on the following Equations (3) and (4).

[0113] φ=(u-0.5)×(2×π) Equation (3)

[0114] θ=(0.5-v)×π Equation (4)

[0115] The 3D coordinates (X, Y, Z) can be calculated based on the following Equations (5)-(7).

[0116] X=cos(θ)cos(φ) Equation (5)

[0117] Y=sin(θ) Equation (6)

[0118] Z=-cos(θ)sin(φ) Equation (7)

[0119] For 3D to 2D coordinate transformation starting from (X, Y, Z), (φ, θ) can be calculated based on the following Equations (8) and (9). Then, (u, v) is calculated based on Equations (3) and (4). Finally, the 2D coordinates (m, n) can be calculated according to Equations (1) and (2).

[0120] φ=tan⁻¹(-Z / X) Equation (8)

[0121] θ=sin⁻¹(Y / (X²+Y²+Z²)^(1 / 2)) Equation (9)

[0122] To reduce seam artifacts in the reconstructed viewport that includes the left and right boundaries of the ERP image, a new format called Padded Equirectangular Projection ("PERP") is provided by padding samples on each of the left and right sides of the ERP image.

[0123] When PERP is used to represent 360-degree video, the PERP image is encoded. After decoding, the reconstructed PERP is converted back to the reconstructed ERP by blending the copied samples or cropping the padded regions.

[0124] Figure 5A A schematic diagram illustrating an example blending operation for generating a reconstructed equirectangular projection according to some embodiments of the present disclosure is shown. Unless otherwise specified, "recPERP" is used to denote the reconstructed PERP before post-processing, and "recERP" is used to denote the reconstructed ERP after post-processing. In Figure 5AIn the image, A1 and B2 are boundary regions, and B1 and A2 are filled regions, where A2 is filled from A1, and B1 is filled from B2. Figure 5A As shown, replicas of recPERP can be blended by applying a distance-based weighted average operation. For example, region A can be generated by blending region A1 with region A2, and region B can be generated by blending region B1 with region B2.

[0125] In the following description, the width and height of the unpadded recERP are represented as "W" and "H" respectively. The left and right padded widths are represented as "P" respectively. L "and "P R The total fill width is represented as "P". w “, which can be P” L and P R The sum of . In some embodiments, recPERP can be converted to recERP through a blending operation. For example, for a sample recERP(j, i) in A, where (j, i) are coordinates in the ERP image, i is in [0, P]. R In [-1], j is in [0, H-1], recERP(j, i) can be determined according to the following formula.

[0126] A = w × A1 + (1 – w) × A2, where w is PL / Pw to 1. Formula (10)

[0127] recERP(j,i)=(recPERP(j,i+PL)×(i+PL) in A

[0128] +recPERP(j,i+PL+W)×(PR-i)+(PW>>1)) / PW Formula (11) where recPERP(y,x) is the sample on the reconstructed PEP image, and (y,x) is the coordinate of the sample in the PEP image.

[0129] In some embodiments, for a sample recERP(j, i) in B, where (j, i) are coordinates in the ERP image, and i is in [WP L If j is in [0, H-1] and i is in [W-1], then recERP(j, i) can be generated according to the following equation:

[0130] B = k × B1 + (1 – k) × B2, where k ranges from 0 to PL / Pw. Formula (12)

[0131] recERP(j,i)=(recPERP(j,i+PL)×(PR-i+W) in B

[0132] +recPERP(j,i+PL-w)×(i–W+PL)+(Pw>>1)) / PW Formula (13) where recPERP(y,x) is the sample on the reconstructed PEP image, and (y,x) is the coordinate of the sample in the PEP image.

[0133] Figure 5B A schematic diagram of an example clipping operation for generating a reconstructed isorectangular projection according to some embodiments of the present disclosure is shown. Figure 5B In the image, A1 and B2 are the boundary regions within the ERP image, and B1 and A2 are the filled regions, where A2 is filled from A1, and B1 is filled from B2. Figure 5B As shown, during the trimming process, the filled samples in recPERP can be directly discarded to obtain recERP. For example, filled samples B1 and A2 can be discarded.

[0134] In some embodiments, horizontal surround motion compensation can be used to improve the coding performance of ERP. For example, horizontal surround motion compensation can be used as a 360-specific coding tool in the VVC standard, designed to improve the visual quality of reconstructed 360-degree video in ERP or PEP formats. In conventional motion compensation, when a motion vector references samples that extend beyond the image boundary of a reference image, repeated padding is applied to derive the values ​​of the out-of-bounds samples by copying those nearest neighboring samples from the corresponding image boundary. For 360-degree video, this repeated padding approach is unsuitable and can result in visual artifacts known as “seam artifacts” in the reconstructed viewport video. Because 360-degree video is captured on a sphere and inherently has no “boundaries,” reference samples outside the boundaries of the reference image in the projection domain can be obtained from neighboring samples in the spherical domain. For general projection formats, it can be difficult to derive the corresponding neighboring samples in the spherical domain because it involves 2D-to-3D and 3D-to-2D coordinate transformations, as well as sample interpolation of fractional sample locations. For the left and right boundaries of ERP or PERP projection formats, this issue can be addressed because spherical neighbor samples outside the left image boundary can be obtained from samples inside the right image boundary, and vice versa. Given the widespread use and relative ease of implementation of ERP or PERP projection formats, VVC employs horizontal surround motion compensation to improve the visual quality of 360-degree video encoded in ERP or PERP projection formats.

[0135] Figure 6A A schematic diagram of an example horizontal surround motion compensation process for a uniform rectangular projection according to some embodiments of the present disclosure is shown. Figure 6AAs shown, when a portion of the reference block lies outside the left (or right) boundary of the reference image in the projection domain, it is not repeated padding; the "outside the boundary" portion can be obtained from the corresponding spherical neighbor portion located within the reference image facing the right (or left) boundary in the projection domain. In some embodiments, repeated padding can be used for the top and bottom image boundaries.

[0136] Figure 6B A schematic diagram of an example horizontal surround motion compensation process for filling a rectangular projection according to some embodiments of the present disclosure is shown. Figure 6B As shown, horizontal wrap motion compensation can be combined with decanted padding methods commonly used in 360-degree video coding. In some embodiments, this is achieved by signaling a high-level syntax element to indicate a wrap motion compensation offset, which can be set to the ERP image width before padding. This syntax can be used to adjust the position of the horizontal wrap accordingly. In some embodiments, this syntax is unaffected by a specific amount of padding on the left or right image boundaries. As a result, this syntax naturally supports asymmetric padding of ERP images. In asymmetric padding of ERP images, the left and right padding can be different. In some embodiments, the wrap motion compensation can be determined according to the following formula:

[0137]

[0138] The offset can be a motion compensation offset signaled in the bitstream, picW can be the image width including the padding region before encoding, and pos... x It can be a reference position determined by the current block position and the motion vector, and the formula pos x The output of `_wrap` can be the actual reference position from the reference block in the surround motion compensation. To save on the signal notification overhead of the surround motion compensation offset, it can be in units of the smallest luminance coded block; therefore, the offset can be replaced with `offset`. w ×MinCbSizeY, where offset w This is the surround motion compensation offset in units of the minimum luminance coded block, which is signaled in the bitstream, and MinCbSizeY is the size of the minimum luminance coded block. Conversely, in conventional motion compensation, the actual reference position from which the reference block originates can be determined by shearing pos within the range of 0 to picW-1. x Export directly.

[0139] When the reference sample is outside the left and right boundaries of the reference image, horizontal surround motion compensation can provide more meaningful information for motion compensation. Under common test conditions for 360-degree video, this tool can improve compression performance not only in terms of rate distortion, but also in terms of reducing seam artifacts and improving the subjective quality of the reconstructed 360-degree video. Horizontal surround motion compensation can also be used for other single-sided projection formats with a constant sampling density in the horizontal direction, such as adjusted equal-area projections.

[0140] In VVC (e.g., VVC draft 8), wraparound motion compensation can be implemented by signaling the high-level syntax variable `pps_ref_wraparound_offset` to indicate the wraparound offset. Sometimes, the wraparound offset should be set to the ERP image width before padding. This syntax can be used to adjust the position of the horizontal wraparound motion compensation accordingly. This syntax may be unaffected by specific padding amounts on the left or right image boundaries. As a result, this syntax can naturally support asymmetric padding of the ERP image (e.g., when the left and right paddings are different). Horizontal wraparound motion compensation can provide more meaningful information for motion compensation when the reference sample is outside the left and right boundaries of the reference image.

[0141] Figure 7 Example syntax for high-level wrap-around offsets according to some embodiments of this disclosure is shown. It should be understood that... Figure 7 The syntax shown can be used in VVC (e.g., VVC Draft 8). Figure 7 As shown, when surround motion compensation is enabled (e.g., pps_ref_wraparound_enabled_flag == 1), the surround offset pps_ref_wraparound_offset can be directly signaled. Figure 7 As shown, syntax elements or variables in the bitstream are displayed in bold.

[0142] Figure 8 The semantics of example high-level wrap-around offsets according to some embodiments of this disclosure are illustrated. It should be understood that... Figure 8 The semantics shown can correspond to Figure 7 The syntax is shown. In some embodiments, such as Figure 8 As shown, a value of 1 for pps_ref_wraparound_enabled_flag means that horizontal wraparound motion compensation is applied in inter-frame prediction. If the value of pps_ref_wraparound_enabled_flag is 0, horizontal wraparound motion compensation is not applied. When the value of CtbSizeY / MinCbSizeY+1 is greater than pic_width_in_luma_samples / MinCbSizeY-1,

[0143] The value of pps_ref_wraparound_enabled_flag should be equal to 0. When

[0144] When sps_ref_wraparound_enabled_flag equals 0, the value of pps_ref_wraparound_enabled_flag should also equal 0. CtbSizeY is the size of the luminance coding tree block.

[0145] In some embodiments, such as Figure 8 As shown, the value of pps_ref_wraparound_offset plus (CtbSizeY / MinCbSizeY) + 2 specifies the offset used to calculate the horizontal wraparound position in units of MinCbSizeY luminance samples. The value of pps_ref_wraparound_offset can range from 0 to (pic_width_in_luma_samples / MinCbSizeY) - (CtbSizeY / MinCbSizeY) - 2, including the extreme values. The variable PpsRefWraparoundOffset can be set to equal pps_ref_wraparound_offset + (CtbSizeY / MinCbSizeY) + 2. In some embodiments, the variable PpsRefWraparoundOffset can be used to determine the luminance position of a sub-block (e.g., in VVC draft 8).

[0146] In VVC (e.g., VVC draft 8), an image can be divided into one or more stripe rows or one or more stripe columns. A stripe can be a sequence of coding tree units (“CTUs”) covering a rectangular region of the image. A stripe can include an integer number of complete stripes or an integer number of consecutive complete CTU rows within a stripe of the image.

[0147] It supports two stripe modes: raster scan stripe mode and rectangular stripe mode. In raster scan stripe mode, a stripe can include a complete sequence of patches in a patch raster scan of an image. In rectangular stripe mode, a stripe can include multiple complete patches that together form a rectangular region of the image, or multiple consecutive complete CTU rows of a patch that together form a rectangular region of the image. Patches within a rectangular stripe are scanned in the patch raster scan sequence within the rectangular region corresponding to that stripe.

[0148] A sub-image may include one or more strips that collectively cover a rectangular region of the image. Figure 9 A schematic diagram illustrating example stripes and sub-image divisions of images according to some embodiments of the present disclosure is shown. Figure 9As shown, the image is divided into 20 patches, with 5 patch columns and 4 patch rows. There are 12 patches on the left, each covering a strip of 4×4 CTUs. There are 8 patches on the right, each covering a strip of 2 vertically stacked 2×2 CTUs. In total, there are 28 strips and 28 sub-images of different sizes (e.g., each strip can be a sub-image).

[0149] Figure 10 A schematic diagram illustrating example stripe and sub-image segmentation with different stripes and sub-images according to some embodiments of the present disclosure is shown. Figure 10 As shown, the image is divided into 20 patches, with 5 patch columns and 4 patch rows. There are 12 patches on the left, each covering a strip of 4×4 CTUs. There are 8 patches on the right, each covering two vertically stacked strips of 2×2 CTUs. There are a total of 28 strips. For the 12 strips on the left, each strip is a sub-image. For the 16 strips on the right, each set of 4 strips forms a sub-image. As a result, there are a total of 16 sub-images of the same size.

[0150] In VVC (e.g., VVC draft 8), information about the stripe layout can be signaled in the image parameter set (“PPS”). In some embodiments, the image parameter set is a syntax structure that includes syntax elements or variables applied to zero or more entire encoded images determined by syntax elements found in each image header. Figure 11 The syntax for example image parameter sets for patch mapping and stripe layout according to some embodiments of this disclosure is shown. It should be understood that... Figure 11 The syntax shown can be used in VVC (e.g., VVC Draft 8). Figure 11 As shown, syntax elements or variables in the bitstream are displayed in bold. For example... Figure 11As shown, if the number of slices in the current image is greater than 1 and a rectangular slice pattern is used (e.g., rect_slice_flag == 1), a flag called single_slice_per_subpic_flag can be signaled first to indicate that each sub-image includes only one slice. In this case (e.g., single_slice_per_subpic_flag == 1), there is no need to further signal the slice layout information, as it can be the same as the sub-image layout already signaled in the sequence parameter set (“SPS”). In some embodiments, SPS is a syntax structure that includes syntax elements applied to zero or more entire coding layer video sequences (“CLVS”), said syntax elements being determined by the content of syntax elements found in the image parameter set referenced by the syntax elements found in each image header. In some embodiments, the image header is a syntax structure that includes syntax elements applicable to all slices of the encoded image. In some embodiments, as Figure 11 As shown, if the value of single_slice_per_subpic_flag is 0, the number of slices in the image can be signaled first (e.g., num_slices_in_pic_minus1), followed by the slice position and size information for each slice.

[0151] In some embodiments, instead of directly signaling the number of slices, the number of slices can be signaled by subtracting 1 (e.g., num_slices_in_pic_minus1) because there is at least one slice in the image. Generally, sending smaller positive values ​​with signals consumes fewer bits and improves the overall efficiency of video processing.

[0152] Figure 12 The semantics of example image parameter sets for patch mapping and strip layout according to some embodiments of this disclosure are illustrated. It should be understood that... Figure 12 The semantic 2 shown can correspond to Figure 11 The syntax is shown. In some embodiments, such as Figure 12 The semantics shown correspond to VVC (e.g., VVC draft 8).

[0153] In some embodiments, such as Figure 12As shown, a value of 0 for the variable `rect_slice_flag` indicates that the slices within each slice are in the raster scan sequence and no slice information is signaled in the PPS. When `rect_slice_flag` equals 1, the slices within each slice can cover a rectangular area of ​​the image, and slice information can be signaled in the PPS. In some embodiments, when it does not exist, `rect_slice_flag` can be inferred to be equal to 1. In some embodiments, when `subpic_info_present_flag` equals 1, the value of `rect_slice_flag` should be equal to 1.

[0154] In some embodiments, such as Figure 12 As shown, a value of 1 for the variable `single_slice_per_subpic_flag` means that each subpicture can include one and only one rectangular stripe. When `single_slice_per_subpic_flag` is equal to 0, each subpicture can include one or more rectangular stripes. In some embodiments, when `single_slice_per_subpic_flag` is equal to 1, the variable `num_slices_in_pic_minus1` can be inferred to be equal to the variable `sps_num_subpics_minus1`. In some embodiments, when it does not exist, the value of `single_slice_per_subpic_flag` can be inferred to be equal to 0.

[0155] In some embodiments, such as Figure 12 As shown, the variable `num_slices_in_pic_minus1` plus 1 represents the number of rectangular strips representing the reference PPS in each image. In some embodiments, the value of `num_slices_in_pic_minus1` can range from 0 to `MaxSlicesPerPicture-1`, including the extreme values. In some embodiments, when the variable `no_pic_partition_flag` equals 1, it can be inferred that the value of `num_slices_in_pic_minus1` is equal to 0.

[0156] In some embodiments, such as Figure 12As shown, if the variable `tile_idx_delta_present_flag` equals 0, then the value of `tile_idx_delta` does not exist in the PPS, and all rectangular stripes in the image of the reference PPS are specified in raster order. In some embodiments, when the variable `tile_idx_delta_present_flag` equals 1, the value of `tile_idx_delta` may exist in the PPS, and all rectangular stripes in the image of the reference PPS are specified in the order indicated by the value of `tile_idx_delta`. In some embodiments, when it does not exist, it can be inferred that the value of `tile_idx_delta_present_flag` is equal to 0.

[0157] In some embodiments, such as Figure 12 As shown, incrementing the variable slice_width_in_tiles_minus1[i] by 1 specifies the width of the i-th rectangular strip in units of tiles. In some embodiments, the value of slice_width_in_tiles_minus1[i] should be in the range of 0 to NumTileColumns-1, including the end value. In some embodiments, if slice_width_in_tiles_minus1[i] does not exist, the following can be applied: if NumTileColumns equals 1, the value of slice_width_in_tiles_minus1[i] can be inferred to be equal to 0; otherwise, the value of slice_width_in_tiles_minus1[i] can be inferred to be the value specified in the clause in the VVC draft (e.g., VVC draft 8).

[0158] In some embodiments, such as Figure 12As shown, incrementing the variable slice_height_in_tiles_minus1 by 1 specifies the height of the i-th rectangular strip in units of slice rows. In some embodiments, the value of slice_height_in_tiles_minus1[i] should be in the range of 0 to NumTileRows-1, including the end value. In some embodiments, when the variable slice_height_in_tiles_minus1[i] does not exist, the following applies: if NumTileRows equals 1, or the variable tile_idx_delta_present_flag equals 0 and tileIdx%NumTileColumns is greater than 0, then the value of slice_height_in_tiles_minus1[i] is inferred to be equal to 0; otherwise (e.g., NumTileRows is not equal to 1, and tile_idx_delta_present_flag equals 1 or tileIdx%NumTileColumns equals 0), when tile_idx_delta_present_flag equals 1 or tileIdx%NumTileColumns equals 0, the value of slice_height_in_tiles_minus1[i] can be inferred to be equal to slice_height_in_tiles_minus1[i-1].

[0159] In some embodiments, such as Figure 12 As shown, the value of num_exp_slices_in_tile[i] specifies the number of explicitly provided stripe heights in the current slice comprising multiple rectangular stripes. In some embodiments, the value of num_exp_slices_in_tile[i] should be in the range of 0 to RowHeight[tileY]-1, inclusive, where tileY is the tile row index including the i-th stripe. In some embodiments, when it does not exist, the value of num_exp_slices_in_tile[i] can be inferred to be equal to 0. In some embodiments, when num_exp_slices_in_tile[i] is equal to 0, the value of the variable NumSliceInTile[i] is deduced to be equal to 1.

[0160] In some embodiments, such as Figure 12As shown, the value of exp_slice_height_in_ctus_minus1[j] plus 1 specifies the height of the j-th rectangular stripe in the current slice, in units of CTU rows. In some embodiments, the value of exp_slice_height_in_ctus_minus1[j] should be in the range of 0 to RowHeight[tileY]-1, including the end value, where tileY is the slice row index of the current slice.

[0161] In some embodiments, such as Figure 12 As shown, when the variable num_exp_slices_in_tile[i] is greater than 0, we can obtain the variables NumSlicesInTile[i] and SliceHeightInCtusMinus1[i+k], where k is in the range of 0 to NumSlicesInTile[i]-1. In some embodiments, such as Figure 8 As shown, when the value of num_exp_slices_in_tile[i] is greater than 0, the values ​​of NumSlicesInTile[i] and SliceHeightInCtusMinus1[i+k] can be obtained.

[0162] In some embodiments, such as Figure 12 As shown, the value of `tile_idx_delta[i]` specifies the difference between the tile index of the first tile in the i-th rectangular strip and the tile index of the first tile in the (i+1)-th rectangular strip. The value of `tile_idx_delta[i]` should be in the range of -NumTilesInPic+1 to NumTilesInPic-1, inclusive. In some embodiments, when it does not exist, it can be inferred that the value of `tile_idx_delta[i]` is equal to 0. In some embodiments, when it exists, it can be inferred that the value of `tile_idx_delta[i]` is not equal to 0.

[0163] In VVC (e.g., VVC draft 8), in order to locate each stripe in an image, one or more stripe addresses can be signaled in the stripe header. Figure 13 The syntax of example strip headers according to some embodiments of this disclosure is shown. It should be understood that... Figure 13 Syntax 3 shown can be used in VVC (e.g., VVC draft 8). Figure 11 As shown, syntax elements or variables in the bitstream are displayed in bold. For example... Figure 13 As shown, if the variable picture_header_in_slice_header_flag is equal to 1, then the image header syntax structure exists in the slice header.

[0164] Figure 14 The semantics of example strip headers according to some embodiments of this disclosure are illustrated. It should be understood that... Figure 14 The semantics shown can correspond to Figure 13 The syntax is shown. In some embodiments, Figure 14 The semantics shown correspond to VVC (e.g., VVC draft 8).

[0165] In some embodiments, such as Figure 14 As shown, the requirement for bitstream consistency is that the value of picture_header_in_slice_header_flag should be the same in all encoded slices in CLVS.

[0166] In some embodiments, such as Figure 14 As shown, when the variable picture_header_in_slice_header_flag is equal to 1 for the coding slice, the requirement for bitstream consistency is that there should be no video coding layer (“VCL”) network abstraction layer (“NAL”) unit with nal_unit_type equal to PH_NUT in CLVS.

[0167] In some embodiments, such as Figure 14 As shown, when picture_header_in_slice_header_flag equals 0, all coded stripes in the current image should have picture_header_in_slice_header_flag equal to 0, and the current PU should have PH NAL units.

[0168] In some embodiments, such as Figure 14 As shown, the variable `slice_address` can specify the slice address. In some embodiments, the value of `slice_address` can be inferred to be 0 when it does not exist. The value of `slice_address` is inferred to be 0 when the variable `rect_slice_flag` is equal to 1 and `NumSliceInSubpic[CurrSubpicIdx]` is equal to 1.

[0169] In some embodiments, such as Figure 14 As shown, if the variable rect_slice_flag equals 0, the following can be applied: the slice address can be the raster scan block index; the length of slice_address can be ceil(Log2(NumTilesInPic)) bits; and the value of slice_address should be in the range of 0 to NumTilesInPic-1, including the end value.

[0170] In some embodiments, such as Figure 14 As shown, the bitstream consistency requirement is subject to the following constraints: if the variable rect_slice_flag is equal to 0 or the variable subpic_info_present_flag is equal to 0, then the value of slice_address should not be equal to the value of slice_address of any other coded slice NAL unit in the same coded image; otherwise, the slice_subpic_id and slice_address value pair should not be equal to the slice_subpic_id and slice_address value pair of any other coded slice NAL unit in the same coded image.

[0171] In some embodiments, such as Figure 14 As shown, the shape of the image stripes should be such that each CTU, when decoded, should have its entire left boundary and its entire top boundary, which includes the image boundary or the boundary of the previously decoded CTU.

[0172] In some embodiments, such as Figure 14 As shown, the variable `slice_address` can be signaled only if one of the following two conditions is met: a rectangular stripe pattern is used and the number of stripes in the current sub-image is greater than 1; or a rectangular stripe pattern is not used and the number of patches in the current image is greater than 1. In some embodiments, if neither of the above conditions is met, then only one stripe exists in the current sub-image or the current image. In this case, since the entire sub-image or the entire image is a single stripe, it is not necessary to signal the stripe address.

[0173] The current VVC design has several problems. First, the wraparound offset pps_ref_wraparound_offset is signaled in the bitstream and should be set to the ERP image width before padding. To save bits, a minimum value of the wraparound offset (e.g., CtbSizeY / MinCbSizeY) + 2 is subtracted from the wraparound offset before signaling. However, the width of the padding region is much smaller than the width of the original ERP image. This is especially true for encoded ERP images where the padding region width can be 0. Assuming the total width of the image to be encoded or decoded is known, signaling the width of the original ERP portion may take more bits than signaling the width of the padding region. As a result, the current signaling of the wraparound offset, which is the original ERP width in MinCbSizeY units, is inefficient.

[0174] Furthermore, signal notification for stripe layout can be improved. For example, subtract 1 from the number of stripes before signaling the number of stripes, since the number of stripes in an image is always greater than or equal to 1, and signaling smaller positive values ​​requires fewer bits. However, in the current VVC (e.g., VVC Draft 8), subpicks comprise an integer number of complete stripes. As a result, the number of stripes in an image is greater than or equal to the number of subpicks in the image. In the current VVC (e.g., VVC Draft 8), num_slices_in_pic_minus1 is signaled only when the variable single_slice_per_subpic_flag is 0, and single_slice_per_subpic_flag equal to 0 indicates that at least one subpick contains multiple stripes. Therefore, in this case, the number of stripes must be greater than the number of subpicks with a minimum value of 1. As a result, the minimum value of num_slices_in_pic_minus1 is greater than zero. Signaling non-negative values ​​whose range does not start from zero is inefficient.

[0175] Furthermore, there are other issues with stripe address signaling. When there is only one stripe in the current image, it may not be necessary to signal the stripe address because the entire sub-image or the entire image is a single stripe. The stripe address can be inferred to be 0. However, the two conditions for skipping stripe address signaling are not complete. For example, in raster scan stripe mode, even if the number of patches is greater than 1, there may be only one stripe that includes all patches in the image, in which case stripe address signaling can also be avoided.

[0176] Embodiments of this disclosure provide a method for solving the above-mentioned problems. In some embodiments, since the width of the fill region is typically smaller than the width of the original ERP image, and the width of the original ERP image may be the same as the surround offset in surround motion compensation, it is suggested to signal the difference between the width of the encoded image in the bitstream and the width of the original ERP image. For example, it is suggested to signal the difference between the width of the encoded image and the surround motion compensation offset in the bitstream, and to perform derivation after parsing the difference signaled to obtain the surround offset on the decoder side. Since the difference between the width of the encoded image and the surround offset is typically smaller than the surround offset itself, this method can save bits of signal transmission.

[0177] Figure 15 The syntax of an example improved image parameter set according to some embodiments of this disclosure is shown. For example... Figure 15 As shown, syntax elements or variables in the bitstream are indicated in bold, and changes made to previous VVCs are indicated in italics (e.g., Figure 7 The syntax shown is illustrated, with the suggested deletion syntax further indicated by strikethrough. (See the example syntax.) Figure 15As shown, a new variable `pps_pic_width_minus_wraparound_offset` can be created.

[0178] The value of pps_pic_width_minus_wraparound_offset can be determined based on...

[0179] The value of pps_ref_wraparound_enabled_flag is signaled. In some embodiments, the new variable pps_pic_width_minus_wraparound_offset can replace the variable pps_ref_wraparound_offset in the original VVC (e.g., VVC draft 8).

[0180] Figure 16 The semantics of an example improved set of image parameters according to some embodiments of this disclosure are illustrated. For example... Figure 16 As shown, changes to the previous VVC (e.g., Figure 8 The semantics shown are in italics, and the proposed deletion syntax is further shown with strikethrough. It should be understood that... Figure 16 The semantics shown can correspond to Figure 15 The syntax is shown. In some embodiments, Figure 16 The semantics shown correspond to VVC (e.g., VVC draft 8).

[0181] like Figure 16 As shown, the semantics of the new variable pps_pic_width_minus_wraparound_offset are different from those of the variable pps_ref_wraparound_offset (for example, as...). Figure 8 (As shown). The variable pps_pic_width_minus_wraparound_offset can specify the difference between the image width and the offset used to calculate the horizontal wraparound position in units of MinCbSizeY luma samples. In some embodiments, such as Figure 16 As shown, the value of pps_pic_width_minus_wraparound_offset should be less than or equal to (pic_width_in_luma_samples / MinCbSizeY) - (CtbSizeY / MinCbSizeY) - 2. In some embodiments, the variable PpsRefWraparoundOffset can be set to equal to pic_width_in_luma_samples / MinCbSizeY – pps_pic_width_minus_wraparound_offset.

[0182] In some embodiments, the signal notification flag `wraparound_offset_type` can be used to indicate whether the wraparound offset used for signal notification is the original ERP image width or the difference between the encoded image width and the original ERP image width. The encoder can choose the smaller of these two values ​​and signal it in the bitstream, thereby further reducing the overhead of signal notification. Figure 17 The syntax of an example image parameter set with the variable wraparound_offset_type is shown according to some embodiments of this disclosure. For example... Figure 17 As shown, syntax elements or variables in the bitstream are shown in bold, and changes made to previous VVCs (e.g., Figure 7 The syntax shown is presented in italics. Figure 17 As shown, a new variable pps_ref_wraparound_offset can be added to specify the value used to determine PpsRefWraparoundOffset.

[0183] Figure 18 The semantics of an example improved set of image parameters with the variable wraparound_offset_type are illustrated according to some embodiments of this disclosure. For example... Figure 18 As shown, changes to the previous VVC (e.g., Figure 8 The semantics shown are in italics, and the proposed deletion syntax is further shown with strikethrough. It should be understood that... Figure 18 The semantics shown can correspond to Figure 17 The syntax is shown. In some embodiments, Figure 18 The semantics shown correspond to VVC (e.g., VVC draft 8).

[0184] In some embodiments, such as Figure 18 As shown, a new variable `wraparound_offset_type` can be added to specify the type of the variable `pps_ref_wraparound_offset`. The value of `pps_ref_wraparound_offset` should be in the range of 0 to `((pps_pic_width_in_luma_samples / MinCbSizeY)–(CtbSizeY / MinCbSizeY)–2) / 2`.

[0185] In some embodiments, such as Figure 18As shown, the value of the variable PpsRefWraparoundOffset can be derived. For example, when the variable wraparound_offset_type equals 0, the wrapound offset signaled by the signal is the original ERP image width. As a result, the variable PpsRefWraparoundOffset equals pps_ref_wraparound_offset + (CrbSizeY / MinCbSizeY) + 2. When the variable wraparound_offset_type is not equal to 0, the wrapound offset signaled by the signal is the difference between the encoded image width and the original ERP image width. As a result, the variable PpsRefWraparoundOffset equals...

[0186] pps_pic_width_in_luma_samples / MinCbSizeY–pps_ref_wraparound_offset.

[0187] In previous VVC implementations (e.g., VVC draft 8), when the variable `single_slice_per_subpic_flag` was 0, the number of slices was signaled in the PPS, indicating the existence of at least one sub-image containing more than one slice. In this case, considering that each sub-image should contain one or more complete slices, the number of slices needs to be greater than the number of sub-images, thus resulting in a minimum number of slices of 2. This is because an image must contain at least one sub-image.

[0188] Embodiments of this disclosure provide a method for signaling notification that improves the number of stripes. Figure 19 The syntax of an example image parameter set with the variable num_slices_in_pic_minus2 is shown according to some embodiments of this disclosure. For example... Figure 19 As shown, syntax elements or variables in the bitstream are shown in bold, and changes made to previous VVCs (e.g., Figure 11 The syntax shown is in italics, with suggested deletion syntax further indicated by strikethrough. Figure 19 As shown, a new variable num_slices_in_pic_minus2 can be added to specify the value used to determine PpsRefWraparoundOffset.

[0189] Figure 20 The semantics of an example improved set of image parameters with the variable num_slices_in_pic_minus2 are illustrated according to some embodiments of this disclosure. For example... Figure 20 As shown, changes to the previous VVC (e.g., Figure 12 The semantics shown are in italics, and the proposed deletion syntax is further shown with strikethrough. It should be understood that... Figure 20 The semantics shown can correspond to Figure 19 The syntax is shown. In some embodiments, Figure 20 The semantics shown correspond to VVC (e.g., VVC draft 8).

[0190] In some embodiments, such as Figure 20 As shown, adding 2 to the value of the variable num_slices_in_pic_minus2 specifies the number of rectangular stripes in each image of the reference PPS. In some embodiments, such as Figure 20 As shown, the variable `num_slices_in_pic_minus2` can replace the variable `num_slices_in_pic_minus_1`. In some embodiments, the value of `num_slices_in_pic_minus2` plus 2 should be in the range of 0 to `MaxSlicesPerPicture - 2`, inclusive, where `MaxSlicesPerPicture` is specified in the VVC (e.g., VVC draft 8). When the variable `no_pic_partition_flag` is equal to 1, the value of `num_slices_in_pic_minus2` is inferred to be equal to -1.

[0191] In some embodiments, the number of slices can be signaled by subtracting the number of subpictures from the number of slices and then subtracting 1 (e.g., num_slices_in_pic_minus_subpic_num_minus1). Figure 21 The syntax of an example image parameter set with the variable num_slices_in_pic_minus_subpic_num_minus1 is shown according to some embodiments of this disclosure. For example... Figure 21 As shown, syntax elements or variables in the bitstream are shown in bold, and changes made to previous VVCs (e.g., Figure 11 The syntax shown is in italics, with the suggested deletion syntax further shown with strikethrough.

[0192] Figure 22 The semantics of an example improved set of image parameters with the variable num_slices_in_pic_minus_subpic_num_minus1 are illustrated according to some embodiments of this disclosure. For example... Figure 22 As shown, changes to the previous VVC (e.g., Figure 12 The semantics shown are in italics, and the proposed deletion syntax is further shown with strikethrough. It should be understood that... Figure 22The semantics shown can correspond to Figure 21 The syntax is shown. In some embodiments, Figure 22 The semantics shown correspond to VVC (e.g., VVC draft 8).

[0193] In VVC (e.g., VVC draft 8), the number of slices is signaled in the PPS only when the variable `single_slice_per_subpic_flag` equals 0. This is equivalent to indicating the existence of at least one subpicture containing more than one slice. Therefore, the number of slices should be greater than the number of subpictures, since a subpicture should contain one or more complete slices. Consequently, the minimum number of slices is equal to the number of subpictures plus 1. Figure 21 and Figure 22 As shown, the signal indicates that the number of stripes is subtracted from the number of sub-images, and then 1 is subtracted (e.g.,

[0194] Instead of subtracting 1 from the number of slices (e.g., num_slices_in_pic_minus_subpic_num_minus1), reduce the number of bits signaled.

[0195] In some embodiments, such as Figure 22 As shown, the value of num_slices_in_pic_minus_subpic_num_minus1 plus the number of subpictures plus 1 can specify the number of rectangular strips in each image of the reference PPS. In some embodiments, the value of num_slices_in_pic_minus_subpic_num_minus1 should be in the range of 0 to MaxSlicesPerPicture minus sps_num_subpics_minus1 minus 2, inclusive, where MaxSlicesPerPicture can be specified in VVC (e.g., VVC draft 8). In some embodiments, when the variable no_pic_partition_flag is equal to 1, the value of num_slices_in_pic_minus_subpic_num_minus1 can be inferred to be equal to (sps_num_subpics_minus1 + 1). In some embodiments, the variable SliceNumInPic can be derived as SliceNumInPic = num_slices_in_pic_minus_subpic_num_minus1 + sps_num_subpics_minus1 + 2.

[0196] Embodiments of this disclosure also provide a novel method for signaling striped addresses. Figure 23 The syntax of an example updated stripe header is shown according to some embodiments of this disclosure. For example... Figure 23 As shown, syntax elements or variables in the bitstream are shown in bold, and changes to previous VVCs (e.g., Figure 13 The syntax shown is in italics, and the suggested deletion syntax is further shown with strikethrough.

[0197] In some embodiments, such as Figure 23 As shown, the variable `picture_header_in_slice_header_flag` can be signaled in the slice header to indicate the presence of a PH syntax structure within the slice header. VVC (e.g., VVC draft 8) has constraints on the presence or absence of a PH syntax structure in the slice header and the number of slices in an image. When a PH syntax structure exists in the slice header, the image should have only one slice. Therefore, it is not necessary to signal the slice address. Thus, signaling the slice address can be conditional on `picture_header_in_slice_header_flag`. Figure 23 As shown, the value of `picture_header_in_slice_header_flag` can be used as another condition to determine whether to signal the variable `slice_address`. When `picture_header_in_slice_header_flag` equals 1, signaling for the variable `slice_address` is skipped. In some embodiments, when the variable `slice_address` is not signaled, it can be inferred that it is 0.

[0198] Embodiments of this disclosure also provide methods for performing video encoding. Figure 24 A flowchart of an example video encoding method according to some embodiments of the present disclosure is shown, the method having a variable that signals the difference between the width of a video frame and an offset used to calculate the horizontal wrap-around position. In some embodiments, Figure 24 The method 24000 shown can be derived from Figure 4 The illustrated device 400 performs this function. In some embodiments, Figure 24 The method shown 24000 can be based on Figure 15 The syntax shown is or Figure 16 The semantic execution is shown. In some embodiments, Figure 24 The method 24000 shown includes a surround motion compensation process performed according to the VVC standard. In some embodiments, Figure 24 The method 24000 shown can be executed with a 360-degree video sequence as input.

[0199] In step S24010, a surround motion compensation flag is received, wherein the surround motion compensation flag is associated with the image. For example, such as Figure 15 or Figure 16 As shown, the wraparound motion compensation flag can be the variable pps_ref_wraparound_enabled_flag. In some embodiments, the image is in a bitstream. In some embodiments, the image is part of a 360-degree video.

[0200] In step S24020, it is determined whether the orbital motion compensation flag is enabled. For example, as... Figure 15 As shown, determine whether the variable pps_ref_wraparound_enabled_flag is equal to 1.

[0201] In step S24030, in response to determining that the surround motion compensation flag is enabled, the difference between the width of the received image and the offset used to determine the horizontal surround position is received. For example, as Figure 15 As shown, when the variable pps_ref_wraparound_enabled_flag is determined to be equal to 1, the variable pps_pic_width_minus_wraparound_offset can be received or signaled. In some embodiments, the difference is less than or equal to the width of the image divided by the size of the minimum luminance coding block minus the size of the luminance coding tree block divided by the size of the minimum luminance coding block minus 2.

[0202] In some embodiments, in S24030, receiving the difference includes receiving a wraparound offset type flag. For example, as... Figure 17 and Figure 18 As shown, the signal notification flag `wraparound_offset_type` can be used to indicate whether the notified wraparound offset value is the original ERP image width or the difference between the encoded image width and the original ERP image width. In some embodiments, such as Figure 17 and Figure 18 As shown, the value of the flag wraparound_offset_type can be 0 or 1.

[0203] In step S24040, motion compensation is performed on the image based on the surrounding motion compensation flag and the difference. For example, it can be based on... Figure 15 and Figure 16The variables pps_ref_wraparound_enabled_flag and pps_pic_width_minus_wraparound_offset shown in the diagram perform motion compensation on the image. In some embodiments, performing the motion compensation further includes: determining a wraparound motion compensation offset based on the width of the image and the difference, and performing the motion compensation on the image based on the wraparound motion compensation offset. In some embodiments, the wraparound motion compensation offset may be determined as the width of the image divided by the size of the minimum luminance coded block minus the difference.

[0204] Figure 25 A flowchart of an example video encoding method according to some embodiments of the present disclosure is shown, which includes signaling a variable representing the number of stripes in a video frame minus 2. In some embodiments, Figure 25 The method 25000 shown can be derived from Figure 4 The illustrated device 400 performs this function. In some embodiments, Figure 25 The method shown 25000 can be based on Figure 19 The syntax shown is or Figure 20 To execute according to the semantics shown. In some embodiments, Figure 25 The method 25000 shown can be executed according to the VVC standard.

[0205] In step S25010, an image for encoding is received. In some embodiments, the image may include one or more stripes. In some embodiments, the image is in a bitstream. In some embodiments, the one or more stripes are rectangular stripes.

[0206] In step S25020, a signal is used in the image parameter set of the image to indicate a variable indicating the number of bands in the image minus 2. For example, this variable could be... Figure 19 or Figure 20 The variable num_slices_in_pic_minus2 is shown. In some embodiments, adding 2 to the value of the variable can specify the number of rectangular stripes in each image. In some embodiments, the variable is part of PPS. In some embodiments, it is similar to... Figure 20 As shown in the semantics, the variable num_slices_in_pic_minus2 can replace the variable num_slices_in_pic_minus_1.

[0207] Figure 26 A flowchart of an example video encoding method with a variable according to some embodiments of the present disclosure is shown, wherein the variable signals a variable indicating the number of stripes in a video frame minus the number of sub-images in the video frame minus 1. In some embodiments, Figure 26 The method 26000 shown can be derived from Figure 4 The illustrated device 400 performs this function. In some embodiments, Figure 26 The method shown 26000 can be based on Figure 21 The syntax shown is or Figure 22 The semantic execution is shown. In some embodiments, Figure 26 The method 26000 shown can be executed according to the VVC standard.

[0208] In step S26010, an image for encoding is received. In some embodiments, the image may include one or more stripes and one or more sub-images. In some embodiments, the video frame is in a bitstream. In some embodiments, the one or more stripes are rectangular stripes.

[0209] In step S26020, a signal is used in the image parameter set of the image to indicate a variable representing the number of stripes in the image minus the number of sub-images in the image minus 1. For example, the variable could be as follows: Figure 21 and Figure 22 The example shows `num_slices_in_pic_minus_subpic_num_minus1`. In some embodiments, the minimum number of slices can be equal to the number of subpicks plus 1. In some embodiments, such as... Figure 21 and Figure 22 As shown, instead of subtracting the number of subpictures from the number of signal notification slices (e.g., num_slices_in_pic_minus_subpic_num_minus1), the number of bits notified by the signal can be reduced by subtracting 1 from the number of slices (e.g., num_slices_in_pic_minus1). In some embodiments, the variable is part of the PPS. In some embodiments, as... Figure 22 As shown, the value of num_slices_in_pic_minus_subpic_num_minus1 plus the number of subpicks plus 1 specifies the number of rectangular stripes in each image of the reference PPS. In some embodiments, similar to Figure 20 As shown in the semantics, the variable num_slices_in_pic_minus2 can replace the variable num_slices_in_pic_minus_1.

[0210] In some embodiments, as shown in step S26020, the variable indicating the number of stripes in an image can be determined based on the number of stripes in the indicating video frame minus the number of sub-images in the video frame minus 1. For example, as... Figure 21 and Figure 22As shown, the flags or the variable SliceNumInPic can be derived from the variable num_slices_in_pic_minus_subpic_num_minus1.

[0211] Figure 27 A flowchart illustrating an example video encoding method with a variable indicating the presence of an image header syntax structure within the stripe header of a video frame is shown, according to some embodiments of the present disclosure. In some embodiments, Figure 27 The method 27000 shown can be used by Figure 4 The illustrated device 400 performs this function. In some embodiments, Figure 27 The method shown 27000 can be based on Figure 23 The syntax shown is executed. In some embodiments, Figure 27 The method 27000 shown can be performed according to the VVC standard.

[0212] In step S27010, an image for encoding is received. The image comprises one or more stripes. In some embodiments, the image is in a bitstream. In some embodiments, the one or more stripes are rectangular stripes.

[0213] In step S27020, a signal is used to indicate whether a variable for the image header syntax structure of the image exists in the strip header of one or more stripes. For example, the variable could be as follows: Figure 23 The image shown is `picture_header_in_slice_header_flag`. In some embodiments, such as... Figure 23 As shown, when the PH syntax structure exists in the slice header, the image may have only one slice. Therefore, there is no need to signal the slice address. Thus, signaling the slice address can be conditional on the variable picture_header_in_slice_header_flag. In some embodiments, the variable is part of the PPS.

[0214] In some embodiments, a non-transitory computer-readable storage medium including instructions is also provided, and these instructions can be executed by a device (such as the disclosed encoder and decoder) to perform the methods described above. Common forms of non-transitory media include, for example, floppy disks, hard disks, solid-state drives, magnetic tape or any other magnetic data storage media, CD-ROMs, any other optical data storage media, any physical media with a perforated pattern, RAM, PROMs and EPROMs, FLASH-EPROMs or any other flash memory, NVRAM, caches, registers, any other memory chips or cassettes, and their networking versions. The device may include one or more processors (CPUs), input / output interfaces, network interfaces, and / or memory.

[0215] It should be noted that relational terms in this document, such as “first” and “second”, are used only to distinguish an entity or operation from another entity or operation and do not require or imply any actual relationship or order between these entities or operations. Furthermore, the words “including,” “having,” “containing,” and “comprising,” and other similar forms, are semantically equivalent and open-ended, because one or more items following any of these words do not imply an exhaustive list of such items or that the list is limited to only one or more items.

[0216] As used herein, unless otherwise specified, the term "or" includes all possible combinations except where impractical. For example, if a database is declared to include A or B, then unless otherwise expressly stated or impractical, the database may include A, or B, or A and B. As a second example, if a database is declared to include A, B, or C, then unless otherwise expressly stated or impractical, the database may include A, or B, or C, or A and B, or A and C, or B and C, or A, B, and C.

[0217] It should be understood that the above embodiments can be implemented by hardware, software (program code), or a combination of hardware and software. If implemented by software, it can be stored in the above-described computer-readable medium. The software, when executed by a processor, can perform the disclosed methods. The computing units and other functional units described in this disclosure can be implemented by hardware, software, or a combination of hardware and software. Those skilled in the art will also understand that the above-described multiple modules / units can be combined into one module / unit, and each of the above-described modules / units can be further divided into multiple sub-modules / sub-units.

[0218] In the foregoing description, numerous specific details have been described with reference to embodiments, which may vary as implementation progresses. Certain modifications and changes may be made to the described embodiments. Other embodiments will be apparent to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This specification and embodiments are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the appended claims. The sequence of steps shown in the accompanying drawings is for illustrative purposes only and is not intended to limit one to any particular sequence of steps. Therefore, those skilled in the art will understand that these steps may be performed in a different order while implementing the same method.

[0219] The embodiments may be further described using the following terms:

[0220] 1. A video decoding method, comprising:

[0221] Receive orbital motion compensation markers;

[0222] Whether to enable orbital motion compensation is determined based on the orbital motion compensation flag.

[0223] In response to determining that the surround motion compensation is enabled, data is received indicating the difference between the width of the image and the offset used to determine the horizontal surround position; and

[0224] Motion compensation is performed based on the surrounding motion compensation flag and the difference.

[0225] 2. The method according to Clause 1, wherein the difference is in units of the size of the minimum luminance coded block.

[0226] 3. The method according to Clause 2, wherein the difference is less than or equal to (pps_pic_width_in_luma_samples / MinCbSizeY)-(CtbSizeY / MinCbSizeY)–2, where pps_pic_width_in_luma_samples is the width of the image in units of luminance samples, MinCbSizeY is the size of the minimum luminance coding block, and CtbSizeY is the size of the luminance coding tree block.

[0227] 4. The method according to Clause 1, wherein performing the motion compensation further includes:

[0228] The orbital motion compensation offset is determined based on the width of the image and the difference; and

[0229] The motion compensation is performed based on the surrounding motion compensation offset.

[0230] 5. The method according to Clause 4, wherein determining the orbital motion compensation offset based on the width of the image and the difference further comprises:

[0231] Divide the width of the image, measured in units of luminance samples, by the size of the minimum luminance coding block to generate the first value; and

[0232] The orbital motion compensation offset is determined to be equal to the first value minus the difference.

[0233] 6. The method according to Clause 1, wherein receiving the data indicating the difference further comprises:

[0234] Receive wrap-around offset type flag;

[0235] Determine whether the wrap-around offset type flag is equal to a first value or a second value;

[0236] In response to determining that the wrap-around offset type flag is equal to the first value, data indicating the difference between the width of the image and the offset used to calculate the horizontal wrap-around position is received; and

[0237] In response to determining that the surround offset type flag is equal to the second value, data indicating the offset for calculating the horizontal surround position is received.

[0238] 7. The method according to Clause 6, wherein each of the first value and the second value is 0 or 1.

[0239] 8. The method according to Clause 1, wherein the motion compensation is performed in accordance with the General Video Coding Standard.

[0240] 9. The method according to Clause 1, wherein the image is part of a 360-degree video sequence.

[0241] 10. The method according to Clause 1, wherein the orbital motion compensation flag and the difference are signaled in the Image Parameter Set (PPS).

[0242] 11. A video decoding method, comprising:

[0243] A signal is used to notify a surround motion compensation flag, which indicates whether surround motion compensation is enabled.

[0244] In response to the surround motion compensation flag indicating that surround motion compensation is enabled, data indicating the difference between the width of the image and the offset used to determine the horizontal surround position is signaled; and

[0245] Motion compensation is performed based on the surrounding motion compensation flag and the difference.

[0246] 12. The method according to Clause 11, wherein the difference is in units of the size of the minimum luminance coded block.

[0247] 13. The method according to Clause 12, wherein the difference is less than or equal to (pps_pic_width_in_luma_samples / MinCbSizeY)-(CtbSizeY / MinCbSizeY)–2, where pps_pic_width_in_luma_samples is the width of the image in units of luminance samples, MinCbSizeY is the size of the minimum luminance coding block, and CtbSizeY is the size of the luminance coding tree block.

[0248] 14. The method according to Clause 11, wherein performing the motion compensation further comprises:

[0249] The orbital motion compensation offset is determined based on the width and difference of the image; and

[0250] The motion compensation is performed based on the surrounding motion compensation offset.

[0251] 15. The method according to Clause 14, wherein determining the orbital motion compensation offset based on the width of the image and the difference further comprises:

[0252] Divide the width of the image, measured in units of luminance samples, by the size of the minimum luminance coding block to generate a first value; and

[0253] The orbital motion compensation offset is determined to be equal to the first value minus the difference.

[0254] 16. The method according to Clause 11, wherein signaling the data indicating the difference further comprises:

[0255] The signal notifies the wrap-around offset type flag, wherein the value of the wrap-around offset type flag is a first value or a second value;

[0256] In response to the value of the wrap-around offset type flag being equal to the first value, data indicating the difference between the width of the image and the offset used to calculate the horizontal wrap-around position is signaled; and

[0257] In response to the value of the surround offset type flag being equal to the second value, the data used to calculate the offset of the horizontal surround position is signaled.

[0258] 17. The method according to Clause 16, wherein each of the first value and the second value is 0 or 1.

[0259] 18. The method according to Clause 11, wherein the motion compensation is performed in accordance with a general video coding standard.

[0260] 19. The method according to Clause 11, wherein the image is part of a 360-degree video sequence.

[0261] 20. The method according to Clause 11, wherein the orbital motion compensation flag and the difference are signaled in the Image Parameter Set (PPS).

[0262] 21. A video encoding method, comprising:

[0263] Receive an image for encoding, wherein the image comprises one or more stripes; and

[0264] In the image parameter set of the image, a variable indicating the number of stripes in the video frame minus 2 is used.

[0265] 22. The method according to Clause 21, wherein the image is in a bitstream.

[0266] 23. The method according to Clause 21, wherein the image is encoded according to a common video coding standard.

[0267] 24. The method according to Clause 21, wherein the one or more strips are rectangular strips.

[0268] 25. A video encoding method, comprising:

[0269] Receive an image for encoding, wherein the image includes one or more stripes and one or more sub-images; and

[0270] In the image parameter set of the image, a signal is used to indicate the variable of the number of stripes in the image minus the number of sub-images in the image minus 1.

[0271] 26. The method according to Clause 25, wherein the image is in a bitstream.

[0272] 27. The method described under Clause 25 further includes:

[0273] The variable indicating the number of bands in the image is determined by subtracting the number of sub-images in the image from the number of bands in the image.

[0274] 28. The method according to Clause 25, wherein the image is encoded according to a common video coding standard.

[0275] 29. The method described in Clause 25, wherein the one or more strips are rectangular strips.

[0276] 30. A video encoding method, comprising:

[0277] Receive an image for encoding, wherein the image comprises one or more stripes;

[0278] The variable indicating whether the image header syntax structure of the image exists in the strip header of one or more strips is signaled; and

[0279] The stripe address is notified using a signal based on the variables.

[0280] 31. The method according to Clause 30, wherein the image is in a bitstream.

[0281] 32. The method according to Clause 30, wherein the image is encoded according to a common video coding standard.

[0282] 33. The method according to Clause 30, wherein the one or more strips are rectangular.

[0283] 34. A system for performing video data processing, the system comprising:

[0284] Memory for storing instruction sets; and

[0285] A processor configured to execute the instruction set to cause the system to perform the following operations:

[0286] Receive orbital motion compensation markers;

[0287] Whether to enable orbital motion compensation is determined based on the orbital motion compensation flag.

[0288] In response to determining that the surround motion compensation is enabled, data is received indicating the difference between the width of the image and the offset used to determine the horizontal surround position; and

[0289] Motion compensation is performed based on the surrounding motion compensation flag and the difference.

[0290] 35. The system according to Clause 34, wherein the difference is in units of the size of the minimum luminance coded block.

[0291] 36. The system according to Clause 35, wherein the difference is less than or equal to (pps_pic_width_in_luma_samples / MinCbSizeY)-(CtbSizeY / MinCbSizeY)–2, where pps_pic_width_in_luma_samples is the width of the image in units of luminance samples, MinCbSizeY is the size of the minimum luminance coding block, and CtbSizeY is the size of the luminance coding tree block.

[0292] 37. The system according to Clause 34, wherein, when performing the motion compensation, the processor is configured to execute the instruction set to cause the system to perform:

[0293] The orbital motion compensation offset is determined based on the width of the image and the difference; and

[0294] The motion compensation is performed based on the surrounding motion compensation offset.

[0295] 38. The system according to clause 37, wherein, when determining the surround motion compensation offset based on the width and the difference of the image, the processor is configured to execute the instruction set to cause the system to perform:

[0296] Divide the width of the image, measured in units of luminance samples, by the size of the minimum luminance coding block to generate the first value; and

[0297] The orbital motion compensation offset is determined to be equal to the first value minus the difference.

[0298] 39. The system according to clause 34, wherein, upon receiving data indicating the difference, the processor is configured to execute the instruction set to cause the system to perform:

[0299] Receive wrap-around offset type flag;

[0300] Determine whether the wrap-around offset type flag is equal to a first value or a second value;

[0301] In response to determining that the wrap-around offset type flag is equal to the first value, data indicating the difference between the width of the image and the offset used to calculate the horizontal wrap-around position is received; and

[0302] In response to determining that the surround offset type flag is equal to the second value, data indicating the offset for calculating the horizontal surround position is received.

[0303] 40. The system according to Clause 39, wherein each of the first value and the second value is 0 or 1.

[0304] 41. The system described in Clause 34, wherein the motion compensation is performed in accordance with a common video coding standard.

[0305] 42. The system according to Clause 34, wherein the image is part of a 360-degree video sequence.

[0306] 43. The system according to Clause 34, wherein the orbital motion compensation flag and the difference are signaled in the Image Parameter Set (PPS).

[0307] 44. A system for performing video data processing, the system comprising:

[0308] Memory for storing instruction sets; and

[0309] A processor configured to execute the instruction set to cause the system to perform the following operations:

[0310] A signal is used to notify a surround motion compensation flag, which indicates whether surround motion compensation is enabled.

[0311] In response to the surround motion compensation flag indicating that surround motion compensation is enabled, data indicating the difference between the width of the image and the offset used to determine the horizontal surround position is signaled; and

[0312] Motion compensation is performed based on the surrounding motion compensation flag and the difference.

[0313] 45. The system according to Clause 44, wherein the difference is in units of the size of the minimum luminance coded block.

[0314] 46. ​​The system according to Clause 45, wherein the difference is less than or equal to (pps_pic_width_in_luma_samples / MinCbSizeY)-(CtbSizeY / MinCbSizeY)–2, where pps_pic_width_in_luma_samples is the width of the image in units of luminance samples, MinCbSizeY is the size of the minimum luminance coding block, and CtbSizeY is the size of the luminance coding tree block.

[0315] 47. The system according to Clause 44, wherein, when performing the motion compensation, the processor is configured to execute the instruction set to cause the system to perform:

[0316] The orbital motion compensation offset is determined based on the width and difference of the image; and

[0317] The motion compensation is performed based on the surrounding motion compensation offset.

[0318] 48. The system according to clause 47, wherein, when determining the surround motion compensation offset based on the width and the difference of the image, the processor is configured to execute the instruction set to cause the system to perform:

[0319] Divide the width of the image, measured in units of luminance samples, by the size of the minimum luminance coding block to generate a first value; and

[0320] The orbital motion compensation offset is determined to be equal to the first value minus the difference.

[0321] 49. The system according to clause 44, wherein, upon receiving data indicating the difference, the processor is configured to execute the instruction set to cause the system to perform:

[0322] The signal notifies the wrap-around offset type flag, wherein the value of the wrap-around offset type flag is a first value or a second value;

[0323] In response to the value of the wrap-around offset type flag being equal to the first value, data indicating the difference between the width of the image and the offset used to calculate the horizontal wrap-around position is signaled; and

[0324] In response to the value of the surround offset type flag being equal to the second value, the data used to calculate the offset of the horizontal surround position is signaled.

[0325] 50. The system according to Clause 49, wherein each of the first value and the second value is 0 or 1.

[0326] 51. The system described in Clause 44, wherein the motion compensation is performed in accordance with a common video coding standard.

[0327] 52. The system described in Clause 44, wherein the image is part of a 360-degree video sequence.

[0328] 3. The system according to Clause 44, wherein the orbital motion compensation flag and the difference are signaled in the Image Parameter Set (PPS).

[0329] 54. A system for performing video encoding, the system comprising:

[0330] Memory for storing instruction sets; and

[0331] A processor configured to execute the instruction set to cause the system to perform the following operations:

[0332] Receive an image for encoding, wherein the image comprises one or more stripes; and

[0333] In the image parameter set of the image, a variable indicating the number of stripes in the video frame minus 2 is used.

[0334] 55. The system according to Clause 54, wherein the image is in a bitstream.

[0335] 56. The system according to Clause 54, wherein the image is encoded according to a common video coding standard.

[0336] 57. The system according to Clause 54, wherein the one or more strips are rectangular strips.

[0337] 58. A system for performing video encoding, the system comprising:

[0338] Memory for storing instruction sets; and

[0339] A processor configured to execute the instruction set to cause the system to perform the following operations:

[0340] Receive an image for encoding, wherein the image includes one or more stripes and one or more sub-images; and

[0341] In the image parameter set of the image, a signal is used to indicate the variable of the number of stripes in the image minus the number of sub-images in the image minus 1.

[0342] 59. The system according to Clause 58, wherein the image is in a bit stream.

[0343] 60. The system according to Clause 58, wherein the processor is configured to execute the instruction set to cause the system to perform:

[0344] The variable indicating the number of bands in the image is determined by subtracting the number of sub-images in the image from the number of bands in the image.

[0345] 61. The system according to Clause 58, wherein the image is encoded according to a common video coding standard.

[0346] 62. The system according to Clause 58, wherein the one or more strips are rectangular strips.

[0347] 63. A system for performing video encoding, the system comprising:

[0348] Memory for storing instruction sets; and

[0349] A processor configured to execute the instruction set to cause the system to perform the following operations:

[0350] Receive an image for encoding, wherein the image comprises one or more stripes;

[0351] The variable indicating whether the image header syntax structure of the image exists in the strip header of one or more strips is signaled; and

[0352] The stripe address is notified using a signal based on the variables.

[0353] 64. The system according to Clause 63, wherein the image is in a bit stream.

[0354] 65. The system according to Clause 63, wherein the image is encoded according to a common video coding standard.

[0355] 66. The system according to Clause 63, wherein one or more stripes are rectangular.

[0356] 67. A non-transitory computer-readable medium storing a set of instructions executable by one or more processors of a device to cause the device to initiate a method for performing video data processing, the method comprising:

[0357] Receive orbital motion compensation markers;

[0358] Whether to enable orbital motion compensation is determined based on the orbital motion compensation flag.

[0359] In response to determining that the surround motion compensation is enabled, data is received indicating the difference between the width of the image and the offset used to determine the horizontal surround position; and

[0360] Motion compensation is performed based on the surrounding motion compensation flag and the difference.

[0361] 68. The non-transitory computer-readable medium as described in Clause 67, wherein the difference is in units of the size of the minimum luminance coded block.

[0362] 69. The non-transitory computer-readable medium according to Clause 68, wherein the difference is less than or equal to (pps_pic_width_in_luma_samples / MinCbSizeY)-(CtbSizeY / MinCbSizeY)–2, where pps_pic_width_in_luma_samples is the width of the image in units of luminance samples, MinCbSizeY is the size of the minimum luminance coding block, and CtbSizeY is the size of the luminance coding tree block.

[0363] 70. The non-transitory computer-readable medium as described in Clause 67, wherein performing the motion compensation further comprises:

[0364] The orbital motion compensation offset is determined based on the width of the image and the difference; and

[0365] The motion compensation is performed based on the surrounding motion compensation offset.

[0366] 71. The non-transitory computer-readable medium according to Clause 70, wherein determining the orbital motion compensation offset based on the width of the image and the difference further comprises:

[0367] Divide the width of the image, measured in units of luminance samples, by the size of the minimum luminance coding block to generate the first value; and

[0368] The orbital motion compensation offset is determined to be equal to the first value minus the difference.

[0369] 72. The non-transitory computer-readable medium as described in Clause 67, wherein the data for receiving the indication difference further comprises:

[0370] Receive wrap-around offset type flag;

[0371] Determine whether the wrap-around offset type flag is equal to a first value or a second value;

[0372] In response to determining that the wrap-around offset type flag is equal to the first value, data indicating the difference between the width of the image and the offset used to calculate the horizontal wrap-around position is received; and

[0373] In response to determining that the surround offset type flag is equal to the second value, data indicating the offset for calculating the horizontal surround position is received.

[0374] 73. The non-transitory computer-readable medium as described in Clause 72, wherein each of the first value and the second value is 0 or 1.

[0375] 74. The non-transitory computer-readable medium as described in Clause 67, wherein the motion compensation is performed in accordance with a common video coding standard.

[0376] 75. The non-transitory computer-readable medium as described in Clause 67, wherein the image is part of a 360-degree video sequence.

[0377] 76. The non-transitory computer-readable medium according to Clause 67, wherein the orbital motion compensation flag and the difference are signaled in the image parameter set (PPS).

[0378] 77. A non-transitory computer-readable medium storing a set of instructions executable by one or more processors of a device to cause the device to initiate a method for performing video data processing, the method comprising:

[0379] A signal is used to notify a surround motion compensation flag, which indicates whether surround motion compensation is enabled.

[0380] In response to the surround motion compensation flag indicating that surround motion compensation is enabled, data indicating the difference between the width of the image and the offset used to determine the horizontal surround position is signaled; and

[0381] Motion compensation is performed based on the surrounding motion compensation flag and the difference.

[0382] 78. The non-transitory computer-readable medium as described in Clause 77, wherein the difference is in units of the size of the minimum luminance coded block.

[0383] 79. The non-transitory computer-readable medium according to Clause 78, wherein the difference is less than or equal to (pps_pic_width_in_luma_samples / MinCbSizeY)-(CtbSizeY / MinCbSizeY)–2, where pps_pic_width_in_luma_samples is the width of the image in units of luminance samples, MinCbSizeY is the size of the minimum luminance coding block, and CtbSizeY is the size of the luminance coding tree block.

[0384] 80. The non-transitory computer-readable medium as described in Clause 77, wherein performing the motion compensation further comprises:

[0385] The orbital motion compensation offset is determined based on the width and difference of the image; and

[0386] The motion compensation is performed based on the surrounding motion compensation offset.

[0387] 81. The non-transitory computer-readable medium according to clause 80, wherein determining the orbital motion compensation offset based on the width of the image and the difference further comprises:

[0388] Divide the width of the image, measured in units of luminance samples, by the size of the minimum luminance coding block to generate a first value; and

[0389] The orbital motion compensation offset is determined to be equal to the first value minus the difference.

[0390] 82. The non-transitory computer-readable medium as described in Clause 77, wherein receiving data indicating a difference further comprises:

[0391] The signal notifies the wrap-around offset type flag, wherein the value of the wrap-around offset type flag is a first value or a second value;

[0392] In response to the value of the wrap-around offset type flag being equal to the first value, data indicating the difference between the width of the image and the offset used to calculate the horizontal wrap-around position is signaled; and

[0393] In response to the value of the surround offset type flag being equal to the second value, the data used to calculate the offset of the horizontal surround position is signaled.

[0394] 83. The non-transitory computer-readable medium as described in Clause 82, wherein each of the first value and the second value is 0 or 1.

[0395] 84. The non-transitory computer-readable medium as described in Clause 77, wherein the motion compensation is performed in accordance with a common video coding standard.

[0396] 85. The non-transitory computer-readable medium as described in Clause 77, wherein the image is part of a 360-degree video sequence.

[0397] 86. The non-transitory computer-readable medium as described in Clause 77, wherein the orbital motion compensation flag and the difference are signaled in the image parameter set (PPS).

[0398] 87. A non-transitory computer-readable medium storing an instruction set executable by one or more processors of a device to cause the device to initiate a method for performing video encoding, the method comprising:

[0399] Receive an image for encoding, wherein the image comprises one or more stripes; and

[0400] In the image parameter set of the image, a variable indicating the number of stripes in the video frame minus 2 is used.

[0401] 88. A non-transitory computer-readable medium as described in Clause 87, wherein the image is in a bitstream.

[0402] 89. The non-transitory computer-readable medium as described in Clause 87, wherein the image is encoded in accordance with a common video coding standard.

[0403] 90. The non-transitory computer-readable medium as described in Clause 87, wherein the one or more stripes are rectangular stripes.

[0404] 91. A non-transitory computer-readable medium storing an instruction set executable by one or more processors of a device to cause the device to initiate a method for performing video encoding, the method comprising:

[0405] Receive an image for encoding, wherein the image includes one or more stripes and one or more sub-images; and

[0406] In the image parameter set of the image, a signal is used to indicate the variable of the number of stripes in the image minus the number of sub-images in the image minus 1.

[0407] 92. The non-transitory computer-readable medium as described in Clause 91, wherein the image is in a bitstream.

[0408] 93. The non-transitory computer-readable medium as described in Clause 91 further includes:

[0409] The variable indicating the number of bands in the image is determined by subtracting the number of sub-images in the image from the number of bands in the image.

[0410] 94. The non-transitory computer-readable medium as described in Clause 91, wherein the image is encoded in accordance with a common video coding standard.

[0411] 95. The non-transitory computer-readable medium as described in Clause 91, wherein the one or more stripes are rectangular stripes.

[0412] 96. A non-transitory computer-readable medium storing an instruction set executable by one or more processors of a device to cause the device to initiate a method for performing video encoding, the method comprising:

[0413] Receive an image for encoding, wherein the image comprises one or more stripes;

[0414] The variable indicating whether the image header syntax structure of the image exists in the strip header of one or more strips is signaled; and

[0415] The stripe address is notified using a signal based on the variables.

[0416] 97. The non-transitory computer-readable medium as described in Clause 96, wherein the image is in a bitstream.

[0417] 98. A non-transitory computer-readable medium as described in Clause 96, wherein the image is encoded in accordance with a common video coding standard.

[0418] 99. The non-transitory computer-readable medium as described in Clause 96, wherein one or more stripes are rectangular.

[0419] Exemplary embodiments have been disclosed in the accompanying drawings and description. However, many variations and modifications can be made to these embodiments. Therefore, although specific terminology has been used, it is used only in a general and descriptive sense and not for limiting purposes.

Claims

1. A video encoding method, comprising: The first flag is notified by a signal in the image parameter set PPS; Based on the first flag, determine whether to use a raster scan strip mode or a rectangular strip mode for each image of the reference PPS; In response to using the rectangular stripe mode, parameters associated with the size of the rectangular stripe are signaled in the PPS, and the stripe address is signaled in the stripe header when the number of stripes in the sub-image is greater than 1; in response to using the raster scan stripe mode, the parameters are not signaled in the PPS, and the stripe address is signaled in the stripe header when the number of patches in the image is greater than 1. The second flag is notified by a signal in the PPS; Based on the second flag, determine whether to enable surround motion compensation, and in response to determining that surround motion compensation is enabled, receive data of the difference between the width of the indication image and the offset used to determine the horizontal surround position; And perform motion compensation based on the orbital motion compensation flag and the difference; Encode the coded blocks within the current stripe based on the second flag; The difference is expressed in units of the size of the minimum luminance coded block.

2. The video encoding method according to claim 1, characterized in that, The step of determining whether to use a raster scan strip mode or a rectangular strip mode for each image of the reference PPS based on the first flag specifically involves: Determine whether the first flag is equal to a first value or a second value. If the first flag is equal to the first value, determine that a raster scan strip pattern is used for each image of the reference PPS. If the first flag is equal to the second value, determine that a rectangular strip pattern is used for each image of the reference PPS.

3. The video encoding method according to claim 2, characterized in that, The first value is 0, and the second value is 1.

4. The video encoding method according to claim 1, characterized in that, The image is part of a 360-degree video sequence.

5. A video decoding method, comprising: Receive the first flag from the image parameter set PPS; Based on the first flag, determine whether to use a raster scan strip mode or a rectangular strip mode for each image of the reference PPS; In response to using the rectangular stripe mode, parameters associated with the size of the rectangular stripe are received from the PPS, and when the number of stripes in the sub-image is greater than 1, a stripe address is received in the stripe header; in response to using the raster scan stripe mode, the reception of the parameters is skipped, and when the number of patches in the image is greater than 1, a stripe address is received in the stripe header. Receive the second flag from the PPS; Based on the second flag, determine whether to enable surround motion compensation, and in response to determining that surround motion compensation is enabled, receive data of the difference between the width of the indication image and the offset used to determine the horizontal surround position; And perform motion compensation based on the orbital motion compensation flag and the difference; Decode the coded block of the current stripe based on the second flag; The difference is expressed in units of the size of the minimum luminance coded block.

6. The video decoding method according to claim 5, characterized in that, The step of determining whether to use a raster scan strip mode or a rectangular strip mode for each image of the reference PPS based on the first flag specifically involves: Determine whether the first flag is equal to a first value or a second value. If the first flag is equal to the first value, determine that a raster scan strip pattern is used for each image of the reference PPS. If the first flag is equal to the second value, determine that a rectangular strip pattern is used for each image of the reference PPS.

7. The video decoding method according to claim 6, characterized in that, The first value is 0, and the second value is 1.

8. The video decoding method according to claim 5, characterized in that, The image is part of a 360-degree video sequence.

9. A non-transitory computer-readable storage medium storing an instruction set executable by one or more processors of a video coding method for encoding a set of images, the method comprising: The first flag is notified by a signal in the image parameter set PPS; Based on the first flag, determine whether to use a raster scan strip mode or a rectangular strip mode for each image of the reference PPS; In response to using the rectangular stripe mode, the parameters associated with the size of the rectangular stripe are signaled in the PPS; and when the number of stripes in the sub-image is greater than 1, the stripe address is signaled in the stripe header; in response to using the raster scan stripe mode, the parameters are not signaled in the PPS, and when the number of patches in the image is greater than 1, the stripe address is signaled in the stripe header. The second flag is notified by a signal in the PPS; Based on the second flag, determine whether to enable surround motion compensation, and in response to determining that surround motion compensation is enabled, receive data of the difference between the width of the indication image and the offset used to determine the horizontal surround position; And perform motion compensation based on the orbital motion compensation flag and the difference; Encode the coded blocks within the current stripe based on the second flag; The difference is expressed in units of the size of the minimum luminance coded block.

10. The non-transitory computer-readable storage medium according to claim 9, wherein determining whether to use a raster scan strip mode or a rectangular strip mode for each image of the reference PPS based on the first flag specifically comprises: Determine whether the first flag is equal to a first value or a second value. If the first flag is equal to the first value, determine that a raster scan strip pattern is used for each image of the reference PPS. If the first flag is equal to the second value, determine that a rectangular strip pattern is used for each image of the reference PPS.

Images