Video code stream encoding and decoding method, corresponding codec and medium

Abstract

A decoding method comprises: receiving an encoded video bitstream, the encoded video bitstream comprising a reference image list; obtaining a reference image of the current image from the reference image list; when the reference image and the current image are in the same layer, setting a motion vector correction flag as a first value so as to start motion vector correction on a current block of the current image; when the reference image and the current image are in different layers, setting the motion vector correction flag as a second value so as to forbid the motion vector correction for the current block of the current image; and when the motion vector correction flag is set as the first value, correcting the motion vector corresponding to the current block.

Description

[0001] This application is a divisional application. The application number of the original application is 202080040219.8, and the original application date is May 27, 2020. The entire content of the original application is incorporated into this application by reference. Technical Field

[0002] This disclosure generally describes techniques for supporting motion vector modification in video coding. More specifically, this disclosure supports disabling motion vector modification when a reference picture is from a different layer than the layer of the current picture. Background Art

[0003] Even relatively short videos require a large amount of video data to describe them, which can create difficulties when the data is to be streamed or otherwise transmitted across communication networks with limited bandwidth capacity. Therefore, video data is often compressed before being sent across modern telecommunications networks. Since memory resources may be limited, the size of the video can also be an issue when storing the video on a storage device. Video compression devices typically use software and / or hardware to encode the video data at the source side before transmission or storage, thereby reducing the amount of data required to represent the digital video image. A video decompression device then decodes the video data and receives the compressed data at the destination side. With limited network resources and the growing demand for higher video quality, there is a need for improved compression and decompression techniques that can increase the compression ratio with little impact on image quality. Summary of the Invention

[0004] A first aspect relates to a method implemented by a video decoder for decoding an encoded video stream. The method comprises: the video decoder receiving an encoded video stream including a reference picture list; the video decoder obtaining a reference picture of a current picture from the reference picture list; when the reference picture and the current picture are in the same layer, the video decoder setting a motion vector correction flag to a first value to enable motion vector correction for a current block of the current picture; when the reference picture and the current picture are in different layers, the video decoder setting the motion vector correction flag to a second value to disable motion vector correction for the current block of the current picture; and when the motion vector correction flag is set to the first value, the video decoder correcting a motion vector corresponding to the current block.

[0005] The method provides a technique for selectively disabling motion vector correction when a reference image is from a different layer than the current image. The ability to selectively disable motion vector correction when the reference image is from a different layer than the current image can avoid video decoding errors (e.g., division by zero). Consequently, the encoder / decoder (also known as "codec") in video decoding is improved relative to current codecs. In effect, the improved video decoding process provides a better user experience for users when transmitting, receiving, and / or viewing video.

[0006] Optionally, in any of the above aspects, another implementation of this aspect is provided: even when the motion vector correction is disabled, reference picture resampling (RPR) is enabled for the entire coded video sequence (CVS) including the current picture.

[0007] Optionally, in any of the above aspects, another implementation of this aspect is provided: using a layer identifier to determine whether the reference image is in the same layer as the current image, or whether the reference image is in a different layer from the current image.

[0008] Optionally, in any of the above aspects, another implementation manner of this aspect is provided: the layer identifier is specified as nuh_layer_id.

[0009] Optionally, in any of the above aspects, another implementation manner of this aspect is provided: the motion vector correction flag is set in a slice header of the encoded video stream.

[0010] Optionally, in any of the above aspects, another implementation manner of this aspect is provided: the motion vector correction flag is a decoder-side motion vector refinement (DMVR) flag or a bi-directional optical flow (BDOF) flag.

[0011] Optionally, in any of the above aspects, another implementation manner of this aspect is provided: displaying an image generated using the current block on a display of an electronic device.

[0012] A second aspect relates to a method for encoding a video stream implemented by a video encoder. The method includes: the video encoder obtaining a reference image for a current image from a reference image list in the video stream; when the reference image and the current image are in the same layer, the video encoder setting a motion vector correction flag to a first value to enable motion vector correction for a current block of the current image; when the reference image and the current image are in different layers, the video encoder setting the motion vector correction flag to a second value to disable motion vector correction for the current block of the current image; and when the motion vector correction flag is set to the first value, the video encoder correcting the motion vector corresponding to the current block.

[0013] The method provides a technique for selectively disabling motion vector correction when a reference image is from a different layer than the current image. The ability to selectively disable motion vector correction when the reference image is from a different layer than the current image can avoid video decoding errors (e.g., division by zero). Consequently, the encoder / decoder (also known as "codec") in video decoding is improved relative to current codecs. In effect, the improved video decoding process provides a better user experience for users when transmitting, receiving, and / or viewing video.

[0014] Optionally, in any of the above aspects, another implementation method of this aspect is provided: the video encoder determines the motion vector of the current image based on the reference image; the video encoder encodes the current image based on the motion vector; the video encoder uses a hypothetical reference decoder to decode the current image.

[0015] Optionally, in any of the above aspects, another implementation of this aspect is provided: even when the motion vector correction is disabled, reference picture resampling (RPR) is enabled for the entire coded video sequence (CVS) including the current picture.

[0016] Optionally, in any of the above aspects, another implementation of this aspect is provided: using a layer identifier to determine whether the reference image is in the same layer as the current image, or whether the reference image is in a different layer from the current image.

[0017] Optionally, in any of the above aspects, another implementation manner of this aspect is provided: the layer identifier is specified as nuh_layer_id.

[0018] Optionally, in any of the above aspects, another implementation manner of this aspect is provided: the motion vector correction flag is set in a slice header of the encoded video stream.

[0019] Optionally, in any of the above aspects, another implementation manner of this aspect is provided: sending the video code stream including the current block to a video decoder.

[0020] A third aspect relates to a decoding device. The decoding device includes: a receiver for receiving an encoded video stream; a memory coupled to the receiver, the memory storing instructions; and a processor coupled to the memory, the processor executing the instructions to cause the decoding device to: receive the encoded video stream including a reference picture list; obtain a reference picture of a current picture from the reference picture list; when the reference picture and the current picture are in the same layer, set a motion vector correction flag to a first value to enable motion vector correction for a current block of the current picture; when the reference picture and the current picture are in different layers, set the motion vector correction flag to a second value to disable motion vector correction for the current block of the current picture; and when the motion vector correction flag is set to the first value, correct the motion vector corresponding to the current block.

[0021] The decoding device provides technology that supports selectively disabling motion vector correction when the reference image is from a different layer than the current image. The ability to selectively disable motion vector correction when the reference image is from a different layer than the current image can avoid video decoding errors (e.g., division by zero). As a result, the encoder / decoder (also known as "codec") in video decoding is improved compared to current codecs. In effect, the improved video decoding process provides users with a better user experience when transmitting, receiving, and / or viewing video.

[0022] Optionally, in any of the above aspects, another implementation of this aspect is provided: even when the motion vector correction is disabled, reference picture resampling (RPR) is enabled for the entire coded video sequence (CVS) including the current picture.

[0023] Optionally, in any of the above aspects, another implementation of this aspect provides a display, wherein the display is used to display an image generated according to the current block.

[0024] A fourth aspect relates to an encoding device. The encoding device includes: a memory including instructions; a processor coupled to the memory, the processor configured to implement the instructions so that the encoding device: obtains a reference image of a current image from a reference image list in the video stream; when the reference image and the current image are in the same layer, sets a motion vector correction flag to a first value to enable motion vector correction for a current block of the current image; when the reference image and the current image are in different layers, sets the motion vector correction flag to a second value to disable motion vector correction for the current block of the current image; when the motion vector correction flag is set to the first value, corrects the motion vector corresponding to the current block; and a transmitter coupled to the processor, the transmitter configured to transmit the video stream including the current block to a video decoder.

[0025] The encoding device provides a technique for selectively disabling motion vector correction when a reference image is from a different layer than the current image. The ability to selectively disable motion vector correction when the reference image is from a different layer than the current image can avoid video decoding errors (e.g., division by zero). As a result, the encoder / decoder (also known as a "codec") in video decoding is improved relative to current codecs. In effect, the improved video decoding process provides users with a better user experience when transmitting, receiving, and / or viewing video.

[0026] Optionally, in any of the above aspects, another implementation of this aspect is provided: even when the motion vector correction is disabled, reference picture resampling (RPR) is enabled for the entire coded video sequence (CVS) including the current picture.

[0027] Optionally, in any of the above aspects, another implementation manner of this aspect is provided: the memory stores the video code stream before the transmitter sends the code stream to the video decoder.

[0028] A fifth aspect relates to a decoding device. The decoding device includes: a receiver configured to receive an image for encoding or a code stream for decoding; a transmitter coupled to the receiver, configured to transmit the code stream to a decoder or transmit the decoded image to a display; a memory coupled to at least one of the receiver or the transmitter, configured to store instructions; and a processor coupled to the memory, configured to execute the instructions stored in the memory to perform any of the methods disclosed herein.

[0029] Optionally, in any of the above aspects, another implementation of this aspect provides a display, wherein the display is used to display an image generated according to the current block.

[0030] The decoding device provides a technique for selectively disabling motion vector correction when the reference image is from a different layer than the current image. The ability to selectively disable motion vector correction when the reference image is from a different layer than the current image can avoid video decoding errors (e.g., division by zero). As a result, the encoder / decoder (also known as "codec") in video decoding is improved compared to current codecs. In effect, the improved video decoding process provides users with a better user experience when transmitting, receiving, and / or viewing video.

[0031] A sixth aspect relates to a system, comprising: an encoder; and a decoder communicating with the encoder, wherein the encoder or the decoder comprises the decoding device, the encoding device, or the decoding apparatus disclosed herein.

[0032] The system provides techniques for selectively disabling motion vector correction when a reference image is from a different layer than the current image. The ability to selectively disable motion vector correction when the reference image is from a different layer than the current image can avoid video decoding errors (e.g., division by zero). Consequently, the encoder / decoder (also known as a "codec") in video decoding is improved relative to current codecs. In effect, the improved video decoding process provides a better user experience for users when transmitting, receiving, and / or viewing video.

[0033] A seventh aspect relates to a decoding module. The decoding module includes: a receiving module for receiving an image for encoding or receiving a code stream for decoding; a sending module coupled to the receiving module, the sending module for sending the code stream to a decoding module or sending the decoded image to a display module; a storage module coupled to at least one of the receiving module or the sending module, the storage module for storing instructions; and a processing module coupled to the storage module, the processing module for executing the instructions stored in the storage module to perform any of the methods disclosed herein.

[0034] The decoding module provides technology that supports selectively disabling motion vector correction when the reference image is from a different layer than the current image. The ability to selectively disable motion vector correction when the reference image is from a different layer than the current image can avoid video decoding errors (e.g., division by zero). As a result, the encoder / decoder (also known as "codec") in video decoding is improved compared to current codecs. In effect, the improved video decoding process provides users with a better user experience when transmitting, receiving, and / or viewing video. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] For a more complete understanding of the present invention, reference is now made to the following brief description taken in conjunction with the accompanying drawings and detailed description, wherein like reference numerals represent like parts.

[0036] Figure 1 is a block diagram of an exemplary coding system that may utilize video coding techniques.

[0037] Figure 2 is a block diagram of an exemplary video encoder that may implement video coding techniques.

[0038] Figure 3 is a block diagram of an example video decoder that can implement video decoding techniques.

[0039] Figure 4 An example of multi-layer coding for achieving spatial adaptability is shown.

[0040] Figure 5 is a schematic diagram of an example of unidirectional inter-frame prediction.

[0041] Figure 6 is a schematic diagram of an example of bidirectional inter-frame prediction.

[0042] Figure 7 The video code stream is shown.

[0043] Figure 8 An image segmentation technique is shown.

[0044] Figure 9 The present invention is an embodiment of a method for decoding an encoded video stream.

[0045] Figure 10 The present invention is an embodiment of a method for encoding a coded video stream.

[0046] Figure 11 It is a schematic diagram of a video decoding device.

[0047] Figure 12 is a schematic diagram of an embodiment of a decoding module. DETAILED DESCRIPTION

[0048] It should be understood at the outset that although illustrative implementations of one or more embodiments are provided below, the disclosed systems and / or methods may be implemented using any number of techniques, whether currently known or in existence. The present invention should in no way be limited to the illustrative implementations, drawings, and techniques illustrated below, including the exemplary designs and implementations illustrated and described herein, but may be modified within the scope of the appended claims along with their full scope of equivalents.

[0049] As used herein, resolution describes the number of pixels in a video file. That is, the resolution is the width and height of the projected image measured in pixels. For example, a video may have a resolution of 1280 (horizontal pixels) × 720 (vertical pixels). This is often abbreviated as 1280×720, or simply 720p. Bidirectional optical flow (BDOF), decoder-side motion vector refinement (DMVR), and merge with motion vector difference (MMVD) are processes, algorithms, or decoding tools used to modify the motion or motion vector of a prediction block. The feature of reference picture resampling (RPR) is to provide the ability to change the spatial resolution of a coded image in the middle of a bitstream without requiring intra-frame decoding of the image at the resolution change location.

[0050] Figure 1 FIG. 1 is a block diagram of an exemplary decoding system 10 in which the video decoding techniques described herein may be employed. Figure 1 As shown, decoding system 10 includes a source device 12 that provides encoded video data to be later decoded by a destination device 14. Specifically, source device 12 can provide the video data to destination device 14 via a computer-readable medium 16. Source device 12 and destination device 14 can include any of a variety of devices, including desktop computers, notebook (e.g., laptop) computers, tablet computers, set-top boxes, telephones such as "smart" phones and "smart" tablets, televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some cases, source device 12 and destination device 14 can be configured for wireless communication.

[0051] The destination device 14 may receive the encoded video data to be decoded via a computer-readable medium 16. The computer-readable medium 16 may include any type of medium or device capable of moving the encoded video data from the source device 12 to the destination device 14. In one example, the computer-readable medium 16 may include a communication medium to enable the source device 12 to send the encoded video data directly to the destination device 14 in real time. The encoded video data may be modulated according to a communication standard (e.g., a wireless communication protocol) and then sent to the destination device 14. The communication medium may include any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network (e.g., the Internet). The communication medium may include routers, switches, base stations, or any other device that can be used to facilitate communication between the source device 12 and the destination device 14.

[0052] In some examples, the encoded data can be output from output interface 22 to a storage device. Similarly, the encoded data can be accessed from the storage device via the input interface. The storage device can include any of a variety of distributed or locally accessed data storage media, such as a hard drive, a Blu-ray disc, a digital video disc (DVD), a compact disc read-only memory (CD-ROM), flash memory, volatile or non-volatile memory, or any other digital storage medium suitable for storing encoded video data. In another example, the storage device can correspond to a file server or another intermediate storage device that can store the encoded video generated by source device 12. Destination device 14 can access the stored video data from the storage device by streaming or downloading. The file server can be any type of server capable of storing encoded video data and sending the encoded video data to destination device 14. Exemplary file servers include a World Wide Web server (e.g., for a website), a File Transfer Protocol (FTP) server, a Network Attached Storage (NAS) device, or a local disk drive. Destination device 14 can access the encoded video data through any standard data connection, including an Internet connection. A standard data connection can include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a digital subscriber line (DSL), a cable modem, etc.), or a combination of both suitable for accessing encoded video data stored in a file server. The transmission of the encoded video data from the storage device can be streaming, downloading, or a combination thereof.

[0053] The techniques of the present invention are not necessarily limited to wireless applications or settings. These techniques can be applied to video decoding to support any of a variety of multimedia applications, such as over-the-air television broadcasting, cable television transmission, satellite television transmission, Internet streaming video transmission such as dynamic adaptive streaming over HTTP (DASH), encoding digital video for storage in a data storage medium, decoding digital video stored in a data storage medium, or other applications. In some examples, decoding system 10 can be used to support one-way or two-way video transmission to support applications such as video streaming, video playback, video broadcasting, and / or video telephony.

[0054] exist Figure 1In the example shown, source device 12 includes a video source 18, a video encoder 20, and an output interface 22. Destination device 14 includes an input interface 28, a video decoder 30, and a display device 32. According to the present invention, the video encoder 20 of source device 12 and / or the video decoder 30 of destination device 14 can be used to apply video decoding techniques. In other examples, the source device and destination device can include other components or devices. For example, source device 12 can receive video data from an external video source (such as an external camera). Similarly, destination device 14 can be connected to an external display device rather than including an integrated display device.

[0055] Figure 1 The illustrated decoding system 10 is merely an example. Video decoding techniques can be performed by any digital video encoding and / or decoding device. While the techniques of the present invention are typically performed by a video decoding device, they can also be performed by a video encoder / decoder (commonly referred to as a "codec"). Furthermore, the techniques of the present invention can also be performed by a video preprocessor. The video encoder and / or decoder can be a graphics processing unit (GPU) or similar device.

[0056] Source device 12 and destination device 14 are merely examples of such encoding devices, where source device 12 generates encoded video data to send to destination device 14. In some examples, source device 12 and destination device 14 can operate substantially symmetrically, such that both source device 12 and destination device 14 include video encoding and decoding components. Thus, encoding system 10 can support one-way or two-way video transmission between video devices 12 and 14, e.g., for video streaming, video playback, video broadcasting, or video telephony.

[0057] Video source 18 of source device 12 may include a video capture device (e.g., a camera), a video archive including previously captured video, and / or a video input interface for receiving video from a video content provider. In another alternative, video source 18 may generate computer graphics-based data as the source video, or a combination of live video, archived video, and computer-generated video.

[0058] In some cases, when video source 18 is a video camera, source device 12 and destination device 14 may constitute a camera phone or videophone. However, as described above, the techniques described herein are generally applicable to video decoding and may be applied to wireless and / or wired applications. In each case, captured, pre-captured, or computer-generated video may be encoded by video encoder 20. The encoded video information may then be output to computer-readable medium 16 via output interface 22.

[0059] Computer-readable media 16 may include transient media, such as wireless broadcasts or wired network transmissions, and may also include storage media (i.e., non-transitory storage media), such as hard disks, flash drives, optical disks, digital video disks, Blu-ray disks, or other computer-readable media. In some examples, a network server (not shown) may receive encoded video data from source device 12 and provide the encoded video data to destination device 14 via network transmission, etc. Similarly, a computing device at a media production facility (e.g., a disc stamping facility) may receive encoded video data from source device 12 and produce a disc including the encoded video data. Therefore, in various examples, computer-readable media 16 may be understood to include one or more computer-readable media in various forms.

[0060] The input interface 28 of the destination device 14 receives information from the computer-readable medium 16. The information of the computer-readable medium 16 may include syntax information defined by the video encoder 20. This syntax information is also used by the video decoder 30 and includes syntax elements that describe the characteristics and / or processing of blocks and other coding units (e.g., group of pictures (GOPs)). The display device 32 displays the decoded video data to a user and may include any of a variety of display devices, such as a cathode ray tube (CRT), a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or other types of display devices.

[0061] The video encoder 20 and the video decoder 30 may operate in accordance with a video coding standard, such as the currently developing efficiency video coding (HEVC) standard, and may comply with the HEVC test model (HM). Alternatively, the video encoder 20 and the video decoder 30 may operate in accordance with other proprietary or industry standards, such as the International Telecommunications Union Telecommunication Standardization Sector (ITU-T) H.264 standard (also known as the Motion Picture Expert Group (MPEG)-4 Part 10, Advanced Video Coding (AVC)), H.265 / HEVC, and extended versions of such standards. However, the techniques of the present invention are not limited to any particular coding standard. Other examples of video coding standards include MPEG-2 and ITU-T H.263. Although in Figure 1 Although not shown, in some aspects, video encoder 20 and video decoder 30 may be integrated with an audio encoder and audio decoder, respectively, and may include suitable multiplexer-demultiplexer (MUX-DEMUX) units or other hardware and software to encode both audio and video in a common data stream or in separate data streams. If applicable, the MUX-DEMUX units may comply with the ITU H.223 multiplexer protocol or other protocols such as the User Datagram Protocol (UDP).

[0062] The video encoder 20 and the video decoder 30 can each be implemented as any of a variety of suitable encoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When the above-mentioned technology is partially implemented in software, a device can store the instructions of the software in a suitable non-transitory computer-readable medium and use one or more processors in hardware to execute these instructions to perform the technology of the present invention. Both the video encoder 20 and the video decoder 30 can be included in one or more encoders or decoders, either of which can be integrated as part of a combined encoder / decoder (CODEC) in the corresponding device. The device including the video encoder 20 and / or the video decoder 30 may include an integrated circuit, a microprocessor, and / or a wireless communication device (e.g., a cellular phone).

[0063] Figure 2 1 is a block diagram of an example of a video encoder 20 that can implement video coding techniques. The video encoder 20 can perform intra-frame decoding and inter-frame decoding on video blocks within a video slice. Intra-frame decoding relies on spatial prediction to reduce or remove spatial redundancy in video within a given video frame or image. Inter-frame decoding relies on temporal prediction to reduce or remove temporal redundancy in video within adjacent frames or images of a video sequence. Intra-frame mode (I-mode) can refer to any of several spatial-based decoding modes. Inter-frame mode (e.g., uni-prediction (P-mode) or bi-prediction (bi-prediction (B-mode)) can refer to any of several temporal-based decoding modes.

[0064] like Figure 2 As shown, the video encoder 20 receives a current video block within a video frame to be encoded. Figure 2In the example of FIG, the video encoder 20 includes a mode selection unit 40, a reference frame memory 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy coding unit 56. The mode selection unit 40 further includes a motion compensation unit 44, a motion estimation unit 42, an intra-predition / intra prediction unit 46, and a segmentation unit 48. For video block reconstruction, the video encoder 20 also includes an inverse quantization unit 58, an inverse transform unit 60, and an adder 62. Deblocking filter ( Figure 2 A filter (not shown) is also included to filter block boundaries and thereby remove blocking artifacts from the reconstructed video. If desired, a deblocking filter is typically applied to the output of adder 62. In addition to the deblocking filter, other (in-loop or post-loop) filters may also be used. Such filters are not shown for simplicity, but such filters (as in-loop filters) may be applied to the output of adder 50 if desired.

[0065] During the encoding process, video encoder 20 receives a video frame or slice to be decoded. The frame or slice may be divided into a plurality of video blocks. Motion estimation unit 42 and motion compensation unit 44 perform inter-frame prediction decoding on the received video block relative to one or more blocks in one or more reference frames to provide temporal prediction. Intra-frame prediction unit 46 may also perform intra-frame prediction decoding on the received video block relative to one or more neighboring blocks in the same frame or slice as the block to be decoded to provide spatial prediction. Video encoder 20 may perform multiple decoding passes, for example, to select an appropriate decoding mode for each block of video data.

[0066] In addition, segmentation unit 48 may segment blocks of video data into sub-blocks based on an evaluation of a previous segmentation scheme in a previous decoding pass. For example, segmentation unit 48 may initially segment a frame or slice into a plurality of largest coding units (LCUs), and segment each LCU into a plurality of sub-coding units (sub-CUs) based on rate-distortion analysis (e.g., rate-distortion optimization). Mode selection unit 40 may further generate a quadtree data structure indicating the segmentation of the LCU into a plurality of sub-CUs. A leaf node CU of the quadtree may include one or more prediction units (PUs) and one or more transform units (TUs).

[0067] The present invention uses the term "block" to refer to any of CU, PU or TU in the context of HEVC or similar data structures in the context of other standards (for example, macroblocks and their sub-blocks in H.264 / AVC). A CU includes a coding node, a PU associated with the coding node, and a TU. The size of the CU corresponds to the size of the coding node and is square. The size of the CU can range from 8×8 pixels to a treeblock size with a maximum value of 64×64 pixels or larger. Each CU may include one or more PUs and one or more TUs. The syntax data associated with the CU may describe (for example) the partitioning of the CU into one or more PUs. The partitioning mode may be different when the CU is encoded in skip mode or direct mode, in intra-frame prediction mode, or in inter-prediction mode. The PU may be partitioned into non-square shapes. The syntax data associated with the CU may also describe (for example) the partitioning of the CU into one or more TUs according to a quadtree. A TU may be square or non-square (for example, rectangular).

[0068] Mode select unit 40 may select one of the intra or inter coding modes based on the error results, etc., and provide the resulting intra or inter coded block to adder 50 to generate residual block data and to adder 62 to reconstruct the coded block for use as a reference frame. Mode select unit 40 also provides syntax elements (e.g., motion vectors, intra mode indicators, partition information, and other such syntax information) to entropy coding unit 56.

[0069] Motion estimation unit 42 and motion compensation unit 44 may be highly integrated, but are described separately for conceptual purposes. Motion estimation performed by motion estimation unit 42 is the process of generating motion vectors, which estimate the motion of video blocks. For example, a motion vector may represent the displacement of a PU of a video block within a current video frame or image relative to a prediction block within a reference frame (or other coded unit), which is relative to the current block being encoded within the current frame (or other coded unit). A prediction block is a block that is found to closely match the block to be coded in terms of pixel difference. This pixel difference may be determined using sum of absolute differences (SAD), sum of square differences (SSD), or other difference metrics. In some examples, video encoder 20 may calculate values ​​for sub-integer pixel positions of a reference image stored in reference frame memory 64. For example, video encoder 20 may interpolate values ​​for quarter-pixel positions, eighth-pixel positions, or other fractional pixel positions of the reference image. Thus, motion estimation unit 42 may perform motion searches relative to both integer and fractional pixel positions and output motion vectors with fractional pixel precision.

[0070] Motion estimation unit 42 calculates a motion vector for the PU of the video block in the inter-coded slice by comparing the position of the PU with the position of the prediction block of the reference picture. The reference picture may be selected from a first reference picture list (list 0) or a second reference picture list (list 1), each of which identifies one or more reference pictures stored in reference frame memory 64. Motion estimation unit 42 sends the calculated motion vector to entropy encoding unit 56 and motion compensation unit 44.

[0071] Motion compensation performed by motion compensation unit 44 may include extracting or generating a prediction block based on the motion vector determined by motion estimation unit 42. Furthermore, in some examples, motion estimation unit 42 and motion compensation unit 44 may be functionally integrated. Upon receiving the motion vector for the PU of the current video block, motion compensation unit 44 may locate the prediction block pointed to by the motion vector in one of the reference picture lists. Adder 50 forms a residual video block by subtracting the pixel values ​​of the prediction block from the pixel values ​​of the current video block being decoded, thereby forming pixel difference values, as described below. Typically, motion estimation unit 42 performs motion estimation for the luma component, and motion compensation unit 44 applies the motion vector calculated based on the luma component to both the chroma and luma components. Mode select unit 40 may also generate syntax elements associated with video blocks and video slices for use by video decoder 30 when decoding video blocks of video slices.

[0072] Intra-prediction unit 46 may perform intra-prediction on the current block, as an alternative to the inter-prediction performed by motion estimation unit 42 and motion compensation unit 44, as described above. Specifically, intra-prediction unit 46 may determine an intra-prediction mode to use for encoding the current block. In some examples, intra-prediction unit 46 may encode the current block using various intra-prediction modes, for example, during a separate encoding process, and intra-prediction unit 46 (or, in some examples, mode selection unit 40) may select an appropriate intra-prediction mode to use from the tested modes.

[0073] For example, intra-frame prediction unit 46 may calculate rate-distortion values ​​using a rate-distortion analysis for various tested intra-frame prediction modes and select the intra-frame prediction mode with the best rate-distortion characteristics among the tested modes. Rate-distortion analysis generally determines the amount of distortion (or error) between a coded block and the original uncoded block that was encoded to produce the coded block, as well as the bit rate (i.e., the number of bits) used to produce the coded block. Intra-frame prediction unit 46 may calculate a ratio based on the distortion and rate for each coded block to determine which intra-frame prediction mode exhibits the best rate-distortion value for that block.

[0074] In addition, intra prediction unit 46 can be used to decode the depth block of the depth image using a depth modeling mode (DMM). Mode selection unit 40 can determine whether the available DMM mode produces better decoding results than the intra prediction mode and other DMM modes (e.g., using rate-distortion optimization (RDO)). Data of the texture image corresponding to the depth image can be stored in reference frame memory 64. Motion estimation unit 42 and motion compensation unit 44 can also be used to perform inter-frame prediction on the depth block of the depth image.

[0075] After selecting an intra-prediction mode for a block (e.g., a conventional intra-prediction mode or one of the DMM modes), intra-prediction unit 46 may provide information indicating the intra-prediction mode selected for the block to entropy coding unit 56. Entropy coding unit 56 may encode the information indicating the selected intra-prediction mode. Video encoder 20 may carry configuration data in the transmitted bitstream, which may include multiple intra-prediction mode index tables and multiple modified intra-prediction mode index tables (also known as codeword mapping tables), definitions of coding contexts for various blocks, an indication of the most probable intra-prediction mode to be used for each coding context, an intra-prediction mode index table, and a modified intra-prediction mode index table.

[0076] Video encoder 20 forms a residual video block by subtracting the prediction data from mode select unit 40 from the original video block being coded. Adder 50 represents one or more components that perform this subtraction operation.

[0077] Transform processing unit 52 applies a transform, such as a discrete cosine transform (DCT) or a conceptually similar transform, to the residual block, thereby producing a video block comprising residual transform coefficient values. Transform processing unit 52 may perform other transforms conceptually similar to the DCT. Wavelet transforms, integer transforms, sub-band transforms, or other types of transforms may also be used.

[0078] Transform processing unit 52 applies a transform to the residual block, thereby generating a block of residual transform coefficients. The transform may convert the residual information from the pixel value domain to a transform domain, such as the frequency domain. Transform processing unit 52 may send the resulting transform coefficients to quantization unit 54. Quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process may reduce the bit depth associated with some or all coefficients. The degree of quantization may be modified by adjusting a quantization parameter. In some examples, quantization unit 54 may then perform a scan on the matrix comprising the quantized transform coefficients. Alternatively, entropy coding unit 56 may perform the scan.

[0079] After quantization, entropy coding unit 56 performs entropy coding on the quantized transform coefficients. For example, entropy coding unit 56 may perform context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or other entropy coding techniques. In the case of context-based entropy coding, the context may be based on neighboring blocks. After entropy coding is performed by entropy coding unit 56, the encoded bitstream may be sent to another device (e.g., video decoder 30) or archived for later transmission or retrieval.

[0080] Inverse quantization unit 58 and inverse transform unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct a residual block in the pixel domain, for example, for later use as a reference block. Motion compensation unit 44 may calculate a reference block by adding the residual block to a prediction block of one of the frames in reference frame memory 64. Motion compensation unit 44 may also apply one or more interpolation filters to the reconstructed residual block to calculate sub-integer pixel values ​​for use in motion estimation. Adder 62 adds the reconstructed residual block to the motion compensated prediction block produced by motion compensation unit 44 to produce a reconstructed video block for storage in reference frame memory 64. The reconstructed video block may be used by motion estimation unit 42 and motion compensation unit 44 as a reference block for inter-coding a block in a subsequent video frame.

[0081] Figure 3 FIG. 1 is a block diagram of an example of a video decoder 30 that can implement video decoding technology. Figure 3 In the example of FIG. 1 , video decoder 30 includes an entropy decoding unit 70, a motion compensation unit 72, an intra-frame prediction unit 74, an inverse quantization unit 76, an inverse transform unit 78, a reference frame memory 82, and an adder 80. In some examples, video decoder 30 may perform the same operations as reference video encoder 20 ( Figure 2 ) is a decoding process that is generally the reverse of the encoding process described in . Motion compensation unit 72 may generate prediction data based on the motion vector received from entropy decoding unit 70, while intra-prediction unit 74 may generate prediction data based on the intra-prediction mode indicator received from entropy decoding unit 70.

[0082] During the decoding process, video decoder 30 receives an encoded video stream from video encoder 20, which represents video blocks of an encoded video slice and associated syntax elements. Entropy decoding unit 70 of video decoder 30 entropy decodes the stream to generate quantization coefficients, motion vectors or intra-prediction mode indicators, and other syntax elements. Entropy decoding unit 70 forwards the motion vectors and other syntax elements to motion compensation unit 72. Video decoder 30 may receive syntax elements at the video slice level and / or the video block level.

[0083] When the video slice is coded as an intra-coded (I) slice, intra-prediction unit 74 may generate prediction data for the video blocks of the current video slice based on the indicated intra-prediction mode and data from previously decoded blocks of the current frame or picture. When the video frame is coded as an inter-coded (e.g., B, P, or GPB) slice, motion compensation unit 72 generates prediction blocks for the video blocks of the current video slice based on the motion vectors and other syntax elements received from entropy decoding unit 70. These prediction blocks may be generated based on one of the reference pictures in one of the reference picture lists. Video decoder 30 constructs reference frame lists: List 0 and List 1, using a default construction technique based on the reference pictures stored in reference frame memory 82.

[0084] Motion compensation unit 72 determines prediction information for video blocks of the current video slice by parsing motion vectors and other syntax elements, and uses the prediction information to generate a prediction block for the current video block being decoded. For example, motion compensation unit 72 uses some of the received syntax elements to determine the prediction mode (e.g., intra prediction or inter prediction) used to code the video blocks of the video slice, the inter-prediction slice type (e.g., B slice, P slice, or GPB slice), construction information for one or more reference picture lists for the slice, motion vectors for each inter-coded video block of the slice, inter-prediction status for each inter-coded video block of the slice, and other information used to decode the video blocks in the current video slice.

[0085] Motion compensation unit 72 may also perform interpolation based on interpolation filters. Motion compensation unit 72 may use interpolation filters used by video encoder 20 during encoding of the video block to calculate interpolated values ​​for sub-integer pixels of the reference block. In this case, motion compensation unit 72 may determine the interpolation filters used by video encoder 20 based on received syntax elements and use these interpolation filters to produce the prediction block.

[0086] Data of the texture image corresponding to the depth image may be stored in the reference frame memory 82. The motion compensation unit 72 may also be configured to perform inter-frame prediction on the depth block of the depth image.

[0087] In one embodiment, the video decoder 30 includes a user interface (UI) 84. The UI 84 is used to receive input from a user of the video decoder 30 (e.g., a network administrator). Through the UI 84, the user can manage or change settings on the video decoder 30. For example, the user can enter or otherwise provide values ​​for parameters (e.g., flags) to control the configuration and / or operation of the video decoder 30 according to the user's preferences. For example, the UI 84 can be a graphical user interface (GUI) that supports user interaction with the video decoder 30 through graphical icons, drop-down menus, check boxes, etc. In some cases, the UI 84 can receive information from the user through a keyboard, mouse, or other peripheral device. In one embodiment, the user can access the UI 84 through a smartphone, tablet device, a personal computer remotely located relative to the video decoder 30, or the like. As used herein, the UI 84 can be referred to as an external input terminal or an external device.

[0088] Taking the above into account, video compression technology performs spatial (intra-frame) prediction and / or temporal (inter-frame) prediction to reduce or remove redundancy inherent in video sequences. For block-based video coding, a video slice (i.e., a video image or a portion of a video image) can be divided into multiple video blocks, which can also be referred to as tree blocks, coding tree blocks (CTB), coding tree units (CTU), coding units (CU) and / or coding nodes. Video blocks in an intra-frame decoding (I) slice of an image are encoded using spatial prediction relative to reference samples in adjacent blocks in the same image. Video blocks in an inter-frame decoding (P or B) slice of an image can use spatial prediction relative to reference samples in adjacent blocks within the same image, or use temporal prediction relative to reference samples in other reference images. An image can be referred to as a frame, and a reference image can be referred to as a reference frame.

[0089] Spatial prediction or temporal prediction produces a prediction block for the block to be decoded. Residual data represents the pixel differences between the original block to be decoded and the prediction block. Inter-coded blocks are encoded based on motion vectors pointing to a block of reference samples that make up the prediction block and residual data representing the difference between the decoded block and the prediction block. Intra-coded blocks are encoded based on an intra-coding mode and the residual data. For further compression, the residual data can be transformed from the pixel domain to the transform domain, producing residual transform coefficients that can then be quantized. The quantized transform coefficients are initially arranged in a two-dimensional array and can be scanned sequentially to produce a one-dimensional vector of transform coefficients. Entropy coding can be used to achieve further compression.

[0090] Image and video compression have evolved rapidly, resulting in a variety of coding standards. These include ITU-T H.261, International Organization for Standardization / International Electrotechnical Commission (ISO / IEC) MPEG-1 Part 2, ITU-T H.262 or ISO / IEC MPEG-2 Part 2, ITU-T H.263, ISO / IEC MPEG-4 Part 2, Advanced Video Coding (AVC) (also known as ITU-T H.264 or ISO / IEC MPEG-4 Part 10), and High Efficiency Video Coding (HEVC) (also known as ITU-T H.265 or MPEG-H Part 2). AVC includes extensions such as scalable video coding (SVC), multiview video coding (MVC), multiview video coding plus depth (MVC+D), and 3D AVC (3D-AVC). HEVC includes extended versions such as scalable HEVC (SHVC), multiview HEVC (MV-HEVC), and 3D HEVC (3D-HEVC).

[0091] There is also a new video coding standard called versatile video coding (VVC) being developed by the joint video experts team (JVET) of ITU-T and ISO / IEC. Although there are multiple working drafts of the VVC standard, this article refers to one of the working drafts (WD) of VVC, namely B. Bross, J. Chen, and S. Liu, "Versatile Video Coding (Draft 5)", JVET-N1001-v3, 13th JVET meeting, March 27, 2019 (VVC Draft 5).

[0092] The techniques disclosed herein are based on the Universal Video Coding (VVC) video coding standard being developed by the Joint Video Experts Team (JVET) of ITU-T and ISO / IEC. However, these techniques are also applicable to other video codecs.

[0093] The reference picture resampling (RPR) feature is the ability to change the spatial resolution of a coded image in the middle of a codestream without requiring intra-frame decoding of the image at the resolution change location. To implement this feature, the image needs to be able to reference one or more reference images with a different spatial resolution than the current image for inter-frame prediction. Therefore, such reference images or parts of them need to be resampled to encode and decode the current image. Hence, it is called RPR. This feature may also be called adaptive resolution change (ARC) or other names. There are some use cases or application scenarios that can benefit from the RPR feature, as shown below.

[0094] Rate adaptation in video calling and conferencing. This allows the encoded video to adapt to changing network conditions. When network conditions deteriorate and available bandwidth decreases, the encoder can adapt by encoding lower-resolution images.

[0095] Active speaker changes in multi-party video conferencing. In multi-party video conferencing, the video size of the active speaker is typically larger than that of other participants. When the active speaker changes, the image resolution of each participant may also need to be adjusted. ARC is particularly useful when the active speaker changes frequently.

[0096] Fast streaming start. For streaming applications, the application typically buffers a certain length of decoded images before starting to display them. Starting the stream at a lower resolution allows the application to have enough images in the buffer to start displaying them faster.

[0097] Adaptive stream switching in streaming. The dynamic adaptive streaming over HTTP (DASH) specification includes a feature called @mediaStreamStructureId. This feature enables switching between different representations at open GOP random access points with non-decodable leading pictures in HEVC (for example, clean random access (CRA) pictures with associated random access skipped leading (RASL) pictures). When two different representations of the same video have different bitrates but the same spatial resolution, and their @mediaStreamStructureId values ​​are the same, switching between the two representations can be performed at the CRA pictures with the associated RASL pictures, and the RASL pictures associated with the switch at the CRA pictures can be decoded with acceptable quality, resulting in seamless switching. With ARC, the @mediaStreamStructureId feature can also be used to switch between DASH representations with different spatial resolutions.

[0098] Adaptability of video decoding is typically supported by using multi-layer decoding techniques. A multi-layer code stream includes a base layer (BL) and one or more enhancement layers (EL). Examples of adaptability include spatial adaptability, quality / signal-to-noise (SNR) adaptability, multi-view adaptability, etc. When using multi-layer decoding techniques, an image or a portion thereof can be decoded (1) without using a reference picture, i.e., using intra-frame prediction, (2) by referring to a reference picture in the same layer, i.e., using inter-frame prediction, or (3) by referring to a reference picture in one or more other layers, i.e., using inter-layer prediction. The reference picture used for inter-layer prediction of the current image is called an inter-layer reference picture (ILRP).

[0099] Figure 4An example of multi-layer decoding for achieving spatial adaptability 400 is shown. The resolution of picture 402 in layer N is different (e.g., lower) than the resolution of picture 404 in layer N+1. In one embodiment, layer N is considered a base layer and layer N+1 is considered an enhancement layer, as described above. In one embodiment, layers N and N+1 are each uniquely identified by a layer identifier, which may be designated as nuh_layer_id. In one embodiment, a long-term reference picture (LTRP) is a picture whose nuh_layer_id is equal to the nuh_layer_id of the current picture. In one embodiment, pictures in different layers are identified as long-term reference pictures. Picture 402 in layer N and picture 404 in layer N+1 may be decoded using inter-frame prediction (as indicated by the solid arrows). Picture 402 may also be decoded using inter-layer prediction (as indicated by the dashed arrows).

[0100] In the context of RPR, the reference picture can be resampled by selecting a reference picture from a lower layer, or by generating a higher layer reference picture from a lower layer reference picture using inter-layer prediction.

[0101] Previous H.26x video coding families supported adaptability to one or more separate profiles from one or more profiles for single-layer decoding. Scalable video coding (SVC) is a scalable version of AVC / H.264 that supports spatial, temporal, and quality adaptability. For SVC, a flag is indicated in each macroblock (MB) in the EL picture to indicate whether the EL MB uses collocated blocks from the lower layer for prediction. The prediction based on collocated blocks can include texture, motion vectors, and / or decoding modes. SVC implementations cannot directly reuse unmodified H.264 / AVC implementations in their design. The SVC EL macroblock syntax and decoding process are different from the H.264 / AVC syntax and decoding process.

[0102] Scalable HEVC (SHVC) is an extension of the HEVC / H.265 standard that supports spatial and quality adaptability; multiview HEVC (MV-HEVC) is an extension of HEVC / H.265 that supports multiview adaptability; and 3D HEVC (3D-HEVC) is an extension of HEVC / H.264 that supports more advanced and efficient three-dimensional (3D) video decoding than MV-HEVC. It should be noted that the single-layer HEVC codec includes temporal adaptability. The multi-layer extensions of HEVC are designed with the following concept: decoded pictures used for inter-layer prediction come from the same access unit (AU) and are treated as long-term reference pictures (LTRPs). They are assigned reference indices from one or more reference picture lists (RPLs) along with other temporal reference pictures in the current layer. Inter-layer prediction (ILP) is implemented at the prediction unit (PU) level by setting the reference index value to refer to one or more inter-layer reference pictures in one or more RPLs.

[0103] When the spatial resolution of the ILRP is different from the spatial resolution of the current picture being encoded or decoded, spatial adaptability uses resampling of the reference picture or its parts. Reference picture resampling can be implemented at the picture level or the coding block level.

[0104] In video codec specifications, pictures need to be identified for various purposes, including use as reference pictures in inter-frame prediction, output of pictures in the decoded picture buffer (DPB), for motion vector scaling, for weighted prediction, etc.

[0105] In AVC and HEVC, pictures can be identified by picture order count (POC).

[0106] In AVC and HEVC, pictures in the DPB can be marked as "used for short-term reference", "used for long-term reference" or "not used for reference". Once a picture is marked as "not used for reference", it can no longer be used for prediction and can be removed from the DPB when it is no longer needed for output.

[0107] There are two types of reference pictures in AVC: short-term reference pictures and long-term reference pictures. When a reference picture is no longer needed for prediction reference, the reference picture can be marked as "unused for reference". The transition between the three states (short-term reference, long-term reference, and unused for reference) is controlled by the decoding reference picture identification process. There are two optional decoding reference picture identification mechanisms: the implicit sliding window process and the explicit memory management control operation (MMCO) process. The sliding window process marks the short-term reference picture as "unused for reference" when the number of reference frames is equal to the given maximum number (max_num_ref_frames in SPS). These short-term reference pictures are stored in a first-in-first-out manner, so that the most recently decoded short-term picture is saved in the DPB.

[0108] The explicit MMCO process may include multiple MMCO commands. The MMCO command may mark one or more short-term or long-term reference pictures as "unused for reference", mark all pictures as "unused for reference", or mark the current reference picture or an existing short-term reference picture as a long-term reference picture and assign a long-term picture index to the long-term reference picture.

[0109] In AVC, the reference picture identification operation and the process of outputting and removing pictures from the DPB are performed after the pictures are decoded.

[0110] HEVC introduces a different approach to reference picture management, called reference pictureset (RPS). The most fundamental difference between the RPS concept and the MMCO / sliding window in AVC is that the complete set of reference pictures used by the current picture or any subsequent picture is provided for each specific slice. Therefore, the complete set of all pictures that must be kept in the DPB for use by the current or subsequent pictures is indicated. This is different from the AVC scheme, which only indicates relative changes to the DPB. With the RPS concept, information from pictures earlier in the decoding order is not required to maintain the correct state of the reference pictures in the DPB.

[0111] To fully leverage the advantages of RPS and improve error resilience, HEVC changes the order of image decoding and DPB operations compared to AVC. In AVC, image identification and buffer operations (outputting and removing decoded images from the DPB) are typically performed after the current image is decoded. In HEVC, the RPS is first decoded from the slice header of the current image, followed by image identification and buffer operations, and finally decoding the current image.

[0112] The latest VVC WD includes a reference picture management method based on two reference picture lists (reference picture list 0 and reference picture list 1). With this method, the reference picture list of a picture is directly constructed without using the reference picture list initialization process and the reference picture list modification process. In addition, the reference picture identification is directly based on the two reference picture lists.

[0113] Motion vector refinement is the process of modifying the motion vector to obtain a better match (e.g., less residual) between the current block and the reference block, thereby achieving better decoding results. VVC includes some inter-frame decoding tools such as bi-directional optical flow (BDOF), decoder-side motion vector refinement (DMVR), and merge with motion vector difference (MMVD), which involve incremental POC-based motion vector scaling and / or sample value scaling. In multi-layer video codecs based on VVC and using inter-layer prediction, the incremental POC-based scaling used in such tools cannot be applied because the collocated image and the current image are associated with the same presentation time or capture time and, therefore, usually have the same POC value, which will lead to the division by zero problem. However, there is no such mechanism to disable this behavior.

[0114] Disclosed herein are techniques for selectively disabling motion vector correction (e.g., DMVR, BDOF, and MMVD) when a reference image is from a different layer than the current image. The ability to selectively disable motion vector correction when the reference image is from a different layer than the current image can avoid video decoding errors (e.g., division by zero). Consequently, the encoder / decoder (also known as "codec") in video decoding is improved relative to current codecs. In effect, the improved video decoding process provides a better user experience for users when transmitting, receiving, and / or viewing video.

[0115] Figure 5 is a diagram illustrating an example of unidirectional inter-frame prediction 500. The unidirectional inter-frame prediction 500 may be used to determine motion vectors for coding blocks and / or decoding blocks created when an image is segmented.

[0116] Unidirectional inter-frame prediction 500 uses a reference frame 530 including a reference block 531 to predict a current block 511 in a current frame 510. As shown, reference frame 530 can be temporally located after current frame 510 (e.g., as the next reference frame), but in some examples, can also be temporally located before current frame 510 (e.g., as the previous reference frame). Current frame 510 is an example frame / image that is encoded / decoded at a particular time. Current frame 510 includes an object in current block 511 that matches an object in reference block 531 in reference frame 530. Reference frame 530 is a frame used as a reference for encoding current frame 510, and reference block 531 is a block in reference frame 530 that includes an object also included in current block 511 in current frame 510.

[0117] Current block 511 is any coding unit that is encoded / decoded at a specified point in time during the decoding process. Current block 511 can be an entire partitioned block or a subblock when affine inter-frame prediction is used. Current frame 510 is separated from reference frame 530 by a temporal distance (TD) 533. TD 533 represents the amount of time between current frame 510 and reference frame 530 in a video sequence and can be measured in frames. Prediction information for current block 511 can reference reference frame 530 and / or reference block 531 using a reference index that represents the direction and temporal distance between frames. During the time period represented by TD 533, an object in current block 511 moves from one location in current frame 510 to another location in reference frame 530 (e.g., the location of reference block 531). For example, the object may move along a trajectory 513, which represents the direction of the object's movement over time. Motion vector 535 describes the direction and magnitude of the object's movement along trajectory 513 within TD 533. Therefore, the encoded motion vector 535 , the reference block 531 , and the residual including the difference between the current block 511 and the reference block 531 provide sufficient information to reconstruct the current block 511 and locate the current block 511 in the current frame 510 .

[0118] Figure 6 6 is a diagram illustrating an example of bidirectional inter prediction 600. Bidirectional inter prediction 600 may be used to determine motion vectors for coding blocks and / or decoding blocks created when an image is segmented.

[0119] Bidirectional inter-frame prediction 600 is similar to unidirectional inter-frame prediction 500, but employs a pair of reference frames to predict a current block 611 in a current frame 610. Thus, current frame 610 and current block 611 are substantially similar to current frame 510 and current block 511, respectively. Current frame 610 is temporally located between a previous reference frame 620, which occurs before current frame 610 in the video sequence, and a next reference frame 630, which occurs after current frame 610 in the video sequence. Previous reference frame 620 and next reference frame 630 are otherwise substantially similar to reference frame 530.

[0120] Current block 611 matches previous reference block 621 in previous reference frame 620 and next reference block 631 in next reference frame 630. This match indicates that, over the course of the video sequence, an object moved along motion path 613 from a position in previous reference block 621, through current block 611, to a position in next reference block 631. Current frame 610 is separated from previous reference frame 620 by a certain previous temporal distance (TD0) 623 and from next reference frame 630 by a certain next temporal distance (TD1) 633. TD0 623 represents the amount of time, in frames, between previous reference frame 620 and current frame 610 in the video sequence. TD1 633 represents the amount of time, in frames, between current frame 610 and next reference frame 630 in the video sequence. Therefore, the object moved along motion path 613 from previous reference block 621 to current block 611 during the time period represented by TD0 623. The object also moves along the motion path 613 from the current block 611 to the next reference block 631 within a time period represented by TD1 633. The prediction information of the current block 611 can refer to the previous reference frame 620 and / or the previous reference block 621 and the next reference frame 630 and / or the next reference block 631 through a pair of reference indexes representing the direction and temporal distance between the frames.

[0121] The previous motion vector (MV0) 625 describes the direction and magnitude of the object's movement along the motion path 613 within TD0 623 (e.g., between the previous reference frame 620 and the current frame 610). The next motion vector (MV1) 635 describes the direction and magnitude of the object's movement along the motion path 613 within TD1 633 (e.g., between the current frame 610 and the next reference frame 630). Therefore, in the bidirectional inter-frame prediction 600, the current block 611 can be decoded and reconstructed using the previous reference block 621 and / or the next reference block 631, MV0 625, and MV1 635.

[0122] In one embodiment, inter-frame prediction and / or bidirectional inter-frame prediction may be performed on a sample-by-sample (e.g., pixel-by-pixel) basis rather than a block-by-block basis. That is, a motion vector pointing to each sample in the previous reference block 621 and / or the next reference block 631 may be determined for each sample in the current block 611. In these embodiments, Figure 6 The motion vector 625 and the motion vector 635 depicted in represent a plurality of motion vectors corresponding to a plurality of samples in the current block 611 , the previous reference block 621 , and the next reference block 631 .

[0123] In the fusion mode and the advanced motion vector prediction (AMVP) mode, the candidate list is generated by adding candidate motion vectors to the candidate list in the order defined by the candidate list determination mode. Such candidate motion vectors may include motion vectors based on unidirectional inter prediction 500, bidirectional inter prediction 600, or a combination thereof. Specifically, the motion vector is generated for such a block when the adjacent block is encoded. Such a motion vector is added to the candidate list of the current block, and the motion vector of the current block is selected from the candidate list. The motion vector can then be indicated as the index of the selected motion vector in the candidate list. The decoder can use the same process as the encoder to construct the candidate list and can determine the selected motion vector from the candidate list based on the indicated index. Therefore, the candidate motion vector includes motion vectors generated based on unidirectional inter prediction 500 and / or bidirectional inter prediction 600, depending on the method used when encoding such adjacent blocks.

[0124] Figure 7 700 is shown. As used herein, the video stream 700 may also be referred to as an encoded video stream, a stream, or a variant thereof. Figure 7 As shown, the code stream 700 includes a sequence parameter set (SPS) 702 , a picture parameter set (PPS) 704 , a slice header 706 , and picture data 708 .

[0125] The SPS 702 includes data common to all images in a sequence of pictures (SOP). In contrast, the PPS 704 includes data common to the entire image. The slice header 706 includes information about the current slice, such as the slice type and the reference image to be used. The SPS 702 and PPS 704 can be collectively referred to as parameter sets. The SPS 702, PPS 704, and slice header 708 are types of network abstraction layer (NAL) units. A NAL unit is a syntactic structure that includes an indication of the type of data to be followed (e.g., decoded video data). NAL units are categorized as video coding layer (VCL) and non-VCL NAL units. VCL NAL units include data representing sample values ​​in a video image, while non-VCL NAL units include any associated additional information, such as parameter sets (important header data that can apply to a large number of VCL NAL units) and supplemental enhancement information (timing information and other supplementary data that can enhance the usability of the decoded video signal but is not required for decoding the values ​​of the samples in the video image). Those skilled in the art will appreciate that the code stream 700 may include other parameters and information in practical applications.

[0126] Figure 7 Image data 708 includes data associated with the image or video being encoded or decoded. Image data 708 can be simply referred to as the payload or data carried in the codestream 700. In one embodiment, image data 708 includes a coded video sequence (CVS) 714 (or CLVS), which includes multiple images 710. CVS 714 is a coded video sequence for each coded layer video sequence (CLVS) in the video codestream 700. It should be noted that when the video codestream 700 includes a single layer, the CVS and CLVS are the same. CVS and CLVS differ only when the video codestream 700 includes multiple layers.

[0127] like Figure 7 As shown, each slice of a picture 710 may be contained in its own VCL NAL unit 712. The collection of VCL NAL units 712 in a CVS 714 may be referred to as an access unit.

[0128] Figure 8 A segmentation technique 800 is shown for an image 810. The image 810 may be similar to Figure 7As shown, the image 810 can be divided into a plurality of slices 812. A slice is a spatially distinct region of a frame (e.g., a picture) that is encoded separately from any other region in the same frame. Figure 8 8. Three stripes 812 are depicted, but more or fewer stripes may be used in actual applications. Each stripe 812 may be divided into a plurality of blocks 814. Figure 8 Block 814 in may be similar to Figure 6 The current block 611, the previous reference block 621 and the next reference block 631 in the block 814 can represent a CU. Figure 8 Four blocks 814 are depicted, but more or fewer blocks may be used in actual applications.

[0129] Each block 814 can be divided into a plurality of samples 816 (e.g., pixels). In one embodiment, the size of each block 814 is measured in luminance samples. Figure 8 Sixteen samples 816 are depicted, but more or fewer samples may be used in actual applications.

[0130] Figure 9 This embodiment of a method 900 for decoding an encoded video stream, implemented by a video decoder (e.g., video decoder 30), is provided. Method 900 may be performed after receiving a decoded stream directly or indirectly from a video encoder (e.g., video encoder 20). Method 900 improves the decoding process by enabling the selective disabling of motion vector correction when the reference image is from a different layer than the current image. Being able to selectively disable motion vector correction in this manner can improve decoding efficiency. Consequently, codec performance is improved, providing a better user experience.

[0131] In step 902, a video decoder receives an encoded video stream (e.g., stream 700). The encoded video stream includes a reference picture list. The reference picture list is a list of reference pictures that can be used for inter-frame prediction. The reference picture list can be designated as reference picture list (RPL) 0 and RPL 1, RefPicList[0] and RefPicList[1], and so on. In one embodiment, the reference picture list structure includes the reference picture list. In one embodiment, the reference picture list is used for bidirectional inter-frame prediction. In step 904, the video decoder obtains a reference picture for the current picture from the reference picture list.

[0132] In step 906, when the reference image and the current image are in the same layer (for example, the reference images are both in Figure 4When the video decoder detects that the motion vector correction flag is set to a first value for the current block of the current picture, the video decoder sets the motion vector correction flag to a first value to enable motion vector correction for the current block of the current picture. In one embodiment, the first value is 1, true (TRUE) or some other value.

[0133] In step 908, when the reference image is located in a different layer than the layer of the current image (e.g., the reference image is located in Figure 4 When the current block is in layer N+1 of the current image, the video decoder sets the motion vector correction flag to a second value to disable motion vector correction for the current block of the current image. In one embodiment, the second value is 0, FALSE, or some other value.

[0134] In one embodiment, the video decoder enables reference picture resampling (RPR) for the entire coded video sequence (CVS) including the current picture even when motion vector correction is disabled.

[0135] In one embodiment, the video decoder uses a layer identifier to determine whether the reference picture is in the same layer as the current picture, or whether the reference picture is in a different layer than the current picture. In one embodiment, the layer identifier is designated as nuh_layer_id. In one embodiment, the motion vector correction flag is set in the slice header of the coded video stream.

[0136] In one embodiment, the motion vector refinement flag is a decoder-side motion vector refinement (DMVR) flag or a bi-directional optical flow (BDOF) flag.

[0137] In step 910, when the motion vector correction flag is set to a first value, the video decoder corrects the motion vector corresponding to the current block. In one embodiment, the motion vector is corrected using DMVR, BDOF, or MMVD. In one embodiment, the DMVR method corrects the initial motion vector (MV) from the fusion mode by searching for the block with the minimum matching cost in the previously decoded reference image. Using BDOF, the bidirectional prediction prediction samples of a coding block are enhanced by the higher-precision MV derived from its two reference blocks. MMVD is a simplified MV representation method based on statistical MVD modeling using three components (e.g., starting point, motion magnitude, and motion direction).

[0138] In one embodiment, an image generated from a current image is displayed to a user of an electronic device (eg, a smartphone, tablet, laptop, personal computer, etc.).

[0139] Figure 10 This embodiment of a method 1000 for encoding a video stream, implemented by a video encoder (e.g., video encoder 20), is provided. Method 1000 may be performed when an image (e.g., in a video) is to be encoded into a video stream and subsequently sent to a video decoder (e.g., video decoder 30). Method 1000 improves the decoding process by enabling the selective disabling of motion vector correction when the reference image is from a different layer than the current image. Being able to selectively disable DMVR in this manner improves decoding efficiency. Consequently, codec performance is improved, providing a better user experience.

[0140] In step 1002, the video encoder obtains a reference picture for the current picture from a reference picture list in the video stream. The reference picture list can be designated as reference picture list (RPL) 0 and RPL 1, RefPicList[0] and RefPicList[1], etc. In one embodiment, after obtaining in step 1002, method 1000 includes determining a motion vector for the current picture based on the reference picture, encoding the current picture based on the motion vector, and decoding the current picture using a hypothetical reference decoder (HRD).

[0141] In step 1004, when the reference image and the current image are in the same layer (for example, the reference images are both in Figure 4 When the video encoder detects a motion vector correction flag in a layer N in the current image, the video encoder sets the motion vector correction flag to a first value to enable motion vector correction for the current block of the current image. In one embodiment, the first value is 1, true (TRUE) or some other value.

[0142] In step 1006, when the reference image is located in a different layer than the layer of the current image (e.g., the reference image is located in Figure 4 When the current block is in layer N+1 of the current image, the video encoder sets the motion vector correction flag to a second value to disable motion vector correction for the current block of the current image. In one embodiment, the second value is 0, FALSE, or some other value.

[0143] In one embodiment, the video decoder enables reference picture resampling (RPR) for the entire coded video sequence (CVS) including the current picture even when motion vector correction is disabled.

[0144] In one embodiment, the video decoder uses a layer identifier to determine whether the reference picture is in the same layer as the current picture, or whether the reference picture is in a different layer than the current picture. In one embodiment, the layer identifier is designated as nuh_layer_id. In one embodiment, the motion vector correction flag is set in the slice header of the coded video stream.

[0145] In one embodiment, the motion vector refinement flag is a decoder-side motion vector refinement (DMVR) flag or a bi-directional optical flow (BDOF) flag.

[0146] In step 1008, when the motion vector correction flag is set to the first value, the video encoder corrects the motion vector corresponding to the current block. In one embodiment, DMVR, BDOF, or MMVD is used to correct the motion vector.

[0147] In one embodiment, the video encoder generates a video stream including the current block and sends the stream to the video decoder. In one embodiment, the video encoder stores the video stream to send to the video decoder.

[0148] In one embodiment, when the reference picture of a CU from RPL 0 or the reference picture of a CU from RPL 1 is from a different layer of the current picture, the flag is set to false. When the flag is equal to 1, it indicates that DMVR is used for the CU.

[0149] In one embodiment, POC delta-based motion vector scaling is applied to a CU in the current picture only if none of the following conditions are true.

[0150] The POC delta between the current picture and the reference picture of a CU from RPL 0 is different from the POC delta between the reference picture of a CU from RPL 1 and the current picture.

[0151] The reference picture is from a different layer than the current picture. Alternatively, the condition is replaced by "the POC delta between the current picture and the reference picture at RPL 0 or the POC delta between the reference picture at RPL 1 and the current picture is equal to 0."

[0152] Alternatively, a flag is derived for each reference picture of the slice to indicate to the reference picture whether the CU in the slice can use motion vector scaling based on POC delta. This flag can be derived during the reference picture list construction process.

[0153] In one embodiment, when the reference picture of a CU from RPL 0 or the reference picture of a CU from RPL 1 is from a different layer of the current picture, the flag is set to false. When the flag is equal to 1, it indicates that BDOF is used for the CU.

[0154] In one embodiment, POC delta-based motion vector scaling is not applied to a CU coded in MMVD mode when the reference picture of a CU from RPL 0 or the reference picture of a CU from RPL 1 is from a different layer of the current picture.

[0155] The following syntax and semantics can be used to implement the embodiments disclosed herein. The following description is relative to the base text, which is the latest VVC draft specification. In other words, only the additions and modifications are described, and text from the base text not mentioned below applies as is. Additions to the base text are shown in bold, and deleted text is shown in italics.

[0156] The overall decoding process for coding units encoded in inter-frame prediction mode is described.

[0157] The input to this process is:

[0158] Luma position (xCb, yCb), which indicates the position of the top-left sample of the current coding block relative to the top-left luma sample of the current image;

[0159] The variable cbWidth represents the width of the current coding block calculated in luminance samples;

[0160] The variable cbHeight represents the height of the current coding block calculated in luminance samples;

[0161] The variable treeType indicates whether a single tree or a dual tree is used. If a dual tree is used, it indicates whether the current tree corresponds to the luma component or the chroma component.

[0162] The output of this process is the modified reconstructed image before in-loop filtering.

[0163] The derivation of quantization parameters as specified in section 8.7.1 of the VVC standard is called with the luma position (xCb, yCb), the width cbWidth of the current coding block in luma samples, and the height cbHeight of the current coding block in luma samples, and the variable treeType as input.

[0164] The decoding process of a coding unit encoded in inter prediction mode consists of the following sequential steps:

[0165] 1. The variable dmvrFlag is set to 0.

[0166] 2. The motion vector components and reference indices of the current coding unit are derived as follows:

[0167] If merge_triangle_flag[xCb][yCb], inter_affine_flag[xCb][yCb], and merge_subblock_flag[xCb][yCb] are all equal to 0, the following applies:

[0168] The derivation process of motion vector components and reference indices specified in section 8.5.2.1 is called with the luma coding block position (xCb, yCb), luma coding block width cbWidth and luma coding block height cbHeight as input, and the luma motion vectors mvL0[0][0] and mvL1[0][0], the reference indices refIdxL0 and refIdxL1 and the prediction list utilization flags predFlagL0[0][0] and predFlagL1[0][0], and the bidirectional prediction weight index gbiIdx as output.

[0169] dmvrFlag is set to 1 when all of the following conditions are met:

[0170] sps_dmvr_enabled_flag equals 1

[0171] merge_flag[xCb][yCb] is equal to 1

[0172] predFlagL0[0][0] and predFlagL1[0][0] are both equal to 1

[0173] mmvd_flag[xCb][yCb] is equal to 0

[0174] DiffPicOrderCnt(currPic,RefPicList[0][refIdxL0]) is equal to DiffPicOrderCnt(RefPicList[1][refIdxL1],currPic)

[0175] cbHeight is greater than or equal to 8

[0176] cbHeight*cbWidth is greater than or equal to 64

[0177] The nuh_layer_id of RefPicList[0][refIdxL0] is the same as the nuh_layer_id of the current picture currPic.

[0178] The nuh_layer_id of RefPicList[1][refIdxL1] is the same as the nuh_layer_id of the current picture currPic.

[0179] If dmvrFlag is equal to 1, the following applies:

[0180] For X 0 and 1, the ordered two-dimensional array refPicLX of luma samples is derived by calling the procedure specified in Section 8.5.7.2 L and two ordered two-dimensional arrays refPicLX of chroma samples Cb and refPicLX Cr The reference image composed of X and refIdxLX is taken as input.

[0181] The number numSbX in the horizontal direction and numSbY in the vertical direction of the luminance coding sub-blocks, the sub-block width sbWidth and the sub-block height sbHeight are derived as follows:

[0182] numSbX = (cbWidth > 16)? ( cbWidth >> 4 ) : 1 (8-255)

[0183] numSbY = (cbHeight > 16)? ( cbHeight >> 4 ) : 1 (8-256)

[0184] sbWidth = (cbWidth > 16)? 16: cbWidth (8-257)

[0185] sbHeight = (cbHeight > 16)? 16: cbHeight (8-258)

[0186] For xSbIdx = 0..numSbX–1 and ySbIdx = 0..numSbY–1, the following applies.

[0187] The luma motion vector mvLX[xSbIdx][ySbIdx] and the prediction list are derived using the flag predFlagLX[xSbIdx][ySbIdx] (with X equal to 0 and 1), and the luma position (xSb[xSbIdx][ySbIdx], ySb[xSbIdx][ySbIdx]) (which represents the position of the top left sample of the coded subblock relative to the top left luma sample of the current image) as follows:

[0188] mvLX[ xSbIdx ][ ySbIdx ] = mvLX[ 0 ][ 0 ] (8-259)

[0189] predFlagLX[ xSbIdx ][ ySbIdx ] = predFlagLX[ 0 ][ 0 ] (8-260)

[0190] xSb[ xSbIdx ][ ySbIdx ] = xCb + xSbIdx * sbWidth (8-261)

[0191] ySb[ xSbIdx ][ ySbIdx ] = yCb + ySbIdx * sbHeight (8-262)

[0192] The decoder-side motion vector correction process specified in clause 8.5.3.1 is called with xSb[xSbIdx][ySbIdx], ySb[xSbIdx][ySbIdx], sbWidth, sbHeight, the motion vector mvLX[xSbIdx][ySbIdx] and the reference picture array refPicLXL as input and the delta motion vector dMvLX[xSbIdx][ySbIdx] as output, where X is equal to 0 and 1.

[0193] The derivation of chroma motion vectors in clause 8.5.2.13 is called with mvLX[xSbIdx][ySbIdx] and refIdxLX as inputs and mvCLX[xSbIdx][ySbIdx] as output, where X is equal to 0 and 1.

[0194] Otherwise (dmvrFlag is equal to 0), the following applies.

[0195] When treeType is equal to SINGLE_TREE and predFlagLX[0][0] (X is 0 or 1) is equal to 1, the derivation process of chroma motion vector in Section 8.5.2.13 is called, where mvLX[0][0] and refIdxLX are used as inputs and mvCLX[0][0] is used as output.

[0196] The number numSbX of luma coding subblocks in the horizontal direction and the number numSbY of luma coding subblocks in the vertical direction are both set to 1.

[0197] Otherwise, if merge_triangle_flag[xCb][yCb] is equal to 1, inter_affine_flag[xCb][yCb] and merge_subblock_flag[xCb][yCb] are both equal to 0, then the derivation process of triangular motion vector components and reference indices specified in Section 8.5.4.1 is called, where the luma coding block position (xCb, yCb), luma coding block width cbWidth and luma coding block height cbHeight are used as input, and the luma motion vectors mvLA and mvLB, the chroma motion vectors mvCLA and mvCLB, the reference indices refIdxLA and refIdxLB and the prediction list flags predFlagLA and predFlagLB are used as output.

[0198] Otherwise (inter_affine_flag[xCb][yCb] or merge_subblock_flag[xCb][yCb] is equal to 1), the derivation process of sub-block motion vector components and reference indices specified in Section 8.5.5.1 is called, where the luma coding block position (xCb, yCb), the luma coding block width cbWidth, the luma coding block height cbHeight are taken as input, the reference indices refIdxL0 and refIdxL1, the number of luma coding sub-blocks in the horizontal direction numS bX and the number numSbY in the vertical direction, the prediction list utilization flag predFlagLX[xSbIdx][ySbIdx], the luma motion vector array mvLX[xSbIdx][ySbIdx], the chroma motion vector array mvCLX[xSbIdx][ySbIdx] (where xSbIdx = 0..(cbWidth>>2)–1 and ySbIdx = 0..(cbHeight>>2)–1 and X is 0 or 1), and the bidirectional prediction weight index gbiIdx as output.

[0199] 3. For xSbIdx=0..numSbX–1, ySbIdx=0..numSbY–1, the luminance motion vector array refMvLX[xSbIdx][ySbIdx] and chrominance motion vector array refMvCLX[xSbIdx][ySbIdx] (X is 0 and 1) after motion vector correction on the decoder side are derived as follows.

[0200] If dmvrFlag is equal to 1, the derivation process of chroma motion vector in clause 8.5.2.13 is called with refMvLX[xSbIdx][ySbIdx] and refIdxLX as inputs and refMvCLX[xSbIdx][ySbIdx] as output, and the input refMvLX[xSbIdx][ySbIdx] is derived as follows:

[0201] refMvLX[xSbIdx][ySbIdx]=mvLX[xSbIdx][ySbIdx]+dMvLX[xSbIdx][ySbIdx]

[0202] (8-263)

[0203] Otherwise (dmvrFlag is equal to 0), the following applies:

[0204] refMvLX[ xSbIdx ][ ySbIdx ] = mvLX[ xSbIdx ][ ySbIdx ] (8-264)

[0205] refMvCLX [ xSbIdx ] [ ySbIdx ] = mvCLX [ xSbIdx ] [ ySbIdx ] (8-265)

[0206] NOTE - The array refMvLX is stored in MvDmvrLX and is used in the derivation of the collocated motion vector in clause 8.5.2.12. The non-modified luma motion vector array MvLX is used in the spatial motion vector prediction and deblocking filter boundary strength derivation processes.

[0207] 4. When ciip_flag[xCb][yCb] is equal to 1, the derivation process of intra prediction mode in combined fusion and intra prediction specified in 8.5.6 is called.

[0208] 5. The predicted samples of the current coding unit are derived as follows.

[0209] If merge_triangle_flag[xCb][yCb] is equal to 0, the prediction samples of the current coding unit are derived as follows.

[0210] Invoke the decoding process for the inter block specified in section 8.5.7.1, where the luma coding block position (xCb, yCb), the luma coding block width cbWidth and the luma coding block height cbHeight, the number of luma coding sub-blocks in the horizontal direction numSbX and the number in the vertical direction numSbY, the luma motion vectors mvL0[xSbIdx][ySbIdx] and mvL1[xSbIdx][ySbIdx], the corrected luma motion vectors refMvL0[xSbIdx][ySbIdx] and refMvL1[xSbIdx][ySbIdx] x][ySbIdx] (xSbIdx = 0..numSbX–1 and ySbIdx = 0..numSbY–1), reference indices refIdxL0 and refIdxL1, prediction list utilization flags predFlagL0[xSbIdx][ySbIdx] and predFlagL1[xSbIdx][ySbIdx], generalized bidirectional prediction weight index gbiIdx, and variable cIdx set to 0, as input, a (cbWidth)×(cbHeight) array predSamples of predicted luma samples. L The inter-frame prediction samples (predSamples) are output.

[0211] Invoke the decoding process for the inter block specified in section 8.5.7.1, where the luma coding block position (xCb, yCb), the luma coding block width cbWidth and the luma coding block height cbHeight, the number of luma coding sub-blocks in the horizontal direction numSbX and the number in the vertical direction numSbY, the chroma motion vectors mvCL0[xSbIdx][ySbIdx] and mvCL1[xSbIdx][ySbIdx], the corrected chroma motion vectors refMvCL0[xSbIdx][ySbIdx] and refMvCL1[xSbIdx][y SbIdx] (xSbIdx = 0..numSbX–1 and ySbIdx = 0..numSbY–1), reference indices refIdxL0 and refIdxL1, prediction list utilization flags predFlagL0[xSbIdx][ySbIdx] and predFlagL1[xSbIdx][ySbIdx], generalized bidirectional prediction weight index gbiIdx and variable cIdx set to 1, as input, a (cbWidth / 2)×(cbHeight / 2) array predSamples of predicted chroma samples for the chroma component Cb Cb The inter-frame prediction samples (predSamples) are output.

[0212] Invoke the decoding process for the inter block specified in section 8.5.7.1, where the luma coding block position (xCb, yCb), the luma coding block width cbWidth and the luma coding block height cbHeight, the number of luma coding sub-blocks in the horizontal direction numSbX and the number in the vertical direction numSbY, the chroma motion vectors mvCL0[xSbIdx][ySbIdx] and mvCL1[xSbIdx][ySbIdx], the corrected chroma motion vectors refMvCL0[xSbIdx][ySbIdx] and refMvCL1[xSbIdx][y SbIdx] (xSbIdx = 0..numSbX–1 and ySbIdx = 0..numSbY–1), reference indices refIdxL0 and refIdxL1, prediction list utilization flags predFlagL0[xSbIdx][ySbIdx] and predFlagL1[xSbIdx][ySbIdx], generalized bidirectional prediction weight index gbiIdx and variable cIdx set to 2, as input, a (cbWidth / 2)×(cbHeight / 2) array predSamples of predicted chroma samples for the chroma component Cr. Cr The inter-frame prediction samples (predSamples) are output.

[0213] Otherwise (merge_triangle_flag[xCb][yCb] is equal to 1), the decoding process for the triangular inter block specified in section 8.5.8.1 is called with the luma coding block position (xCb, yCb), the luma coding block width cbWidth and the luma coding block height cbHeight, the luma motion vectors mvLA and mvLB, the chroma motion vectors mvCLA and mvCLB, the reference indices refIdxLA and refIdxLB, the prediction list flags predFlagLA and predFlagLB as input, and the inter prediction samples (predSamples) as output, which are the (cbWidth) × (cbHeight) array predSamples of predicted luma samples. L and two (cbWidth / 2)×(cbHeight / 2) arrays of predicted chroma samples predSamples Cb and predSamples Cr , each corresponding to each of the chrominance components Cb and Cr.

[0214] 6. The variables NumSbX[xCb][yCb] and NumSbY[xCb][yCb] are set equal to numSbX and numSbY respectively.

[0215] 7. The residual samples of the current coding unit are derived as follows.

[0216] Invokes the decoding process of the residual signal of a coding block coded in inter prediction mode as specified in section 8.5.9, with as input the position (xTb0, yTb0) set equal to the luma position (xCb, yCb), the width nTbW set equal to the luma coding block width cbWidth, the height nTbH set equal to the luma coding block height cbHeight, and the variable cIdx set to 0, the array resSamples L as output.

[0217] Invokes the decoding process of the residual signal of a coding block coded in inter prediction mode as specified in section 8.5.9, with as input the position (xTb0, yTb0) set equal to the chroma position (xCb / 2, yCb / 2), the width nTbW set equal to the chroma coding block width cbWidth / 2, the height nTbH set equal to the chroma coding block height cbHeight / 2, and the variable cIdx set to 1, the array resSamples Cb as output.

[0218] Invokes the decoding process of the residual signal of a coding block coded in inter prediction mode as specified in section 8.5.9, with as input the position (xTb0, yTb0) set equal to the chroma position (xCb / 2, yCb / 2), the width nTbW set equal to the chroma coding block width cbWidth / 2, the height nTbH set equal to the chroma coding block height cbHeight / 2, and the variable cIdx set to 2, the array resSamples Cr as output.

[0219] 8. The reconstructed samples of the current coding unit are derived as follows.

[0220] Invoke the image reconstruction process for the color components specified in Section 8.7.5, with the block position (xB, yB) set equal to (xCb, yCb), the block width bWidth set equal to cbWidth, the block height bHeight set equal to cbHeight, the variable cIdx set to 0, and predSamples set equal to L The (cbWidth) x (cbHeight) array predSamples and is set equal to resSamples L The (cbWidth)×(cbHeight) array resSamples is taken as input and the output is the modified reconstructed image before in-loop filtering.

[0221] Invoke the image reconstruction process for the color components specified in Section 8.7.5, with the block position (xB, yB) set equal to (xCb / 2, yCb / 2), the block width bWidth set equal to cbWidth / 2, the block height bHeight set equal to cbHeight / 2, the variable cIdx set to 1, and predSamples set equal to Cb The (cbWidth / 2) x (cbHeight / 2) array predSamples and is set equal to resSamples Cb The (cbWidth / 2)×(cbHeight / 2) array resSamples is taken as input and the output is the modified reconstructed image before in-loop filtering.

[0222] Invoke the image reconstruction process for the color components specified in Section 8.7.5, with the block position (xB, yB) set equal to (xCb / 2, yCb / 2), the block width bWidth set equal to cbWidth / 2, the block height bHeight set equal to cbHeight / 2, the variable cIdx set to 2, and predSamples set equal to Cr The (cbWidth / 2) x (cbHeight / 2) array predSamples and is set equal to resSamples Cr The (cbWidth / 2)×(cbHeight / 2) array resSamples is taken as input and the output is the modified reconstructed image before in-loop filtering.

[0223] The derivation process of fused motion vector difference is described.

[0224] The input to this process is:

[0225] The luma position (xCb, yCb) of the top-left luma sample of the current luma coding block relative to the top-left luma sample of the current image;

[0226] Reference indexes refIdxL0 and refIdxL1;

[0227] The prediction list utilizes the flags predFlagL0 and predFlagL1.

[0228] The output of this process is the luma-fused motion vector difference in mMvdL0 and mMvdL1 with 1 / 16 fractional sample accuracy.

[0229] The variable currPic represents the current image.

[0230] The luma fused motion vector differences mMvdL0 and mMvdL1 are derived as follows. If both predFlagL0 and predFlagL1 are equal to 1, the following applies:

[0231] currPocDiffL0=DiffPicOrderCnt(currPic,RefPicList[0][refIdxL0]) (8-356)

[0232] currPocDiffL1=DiffPicOrderCnt(currPic,RefPicList[1][refIdxL1]) (8-357)

[0233] If currPocDiffL0 is equal to currPocDiffL1, or the nuh_layer_id of RefPicList[0][refIdxL0] or RefPicList[1][refIdxL1] is the same as the nuh_layer_id of the current picture, then the following applies:

[0234] mMvdL0[0]=MmvdOffset[xCb][yCb][0] (8-358)

[0235] mMvdL0[1]=MmvdOffset[xCb][yCb][1] (8-359)

[0236] mMvdL1[0]=MmvdOffset[xCb][yCb][0] (8-360)

[0237] mMvdL1[1]=MmvdOffset[xCb][yCb][1] (8-361)

[0238] Otherwise, if Abs(currPocDiffL0) is greater than or equal to Abs(currPocDiffL1), the following applies:

[0239] td=Clip3(–128,127,currPocDiffL0) (8-362)

[0240] tb=Clip3(–128,127,currPocDiffL1) (8-363)

[0241] tx=(16384+(Abs(td)>>1)) / td (8-364)

[0242] distScaleFactor = Clip3(–4096, 4095, (tb * tx + 32) >> 6) (8 - 365)

[0243] mMvdL0[0] = mMvdOffset[xCb][yCb][0] (8 - 366)

[0244] mMvdL0[1] = mMvdOffset[xCb][yCb][1] (8 - 367)

[0245] mMvdL1[0] = Clip3(–2 15 , 2 15 –1, Sign(distScaleFactor * mMvdL0[0]) * ((Abs(distScaleFactor * mMvdL0[0]) + 127) >> 8)) (8 - 368)

[0246] mMvdL1[1] = Clip3(–2 15 , 2 15 –1, Sign(distScaleFactor * mMvdL0[1]) * ((Abs(distScaleFactor * mMvdL0[1]) + 127) >> 8)) (8 - 369)

[0247] Otherwise (Abs(currPocDiffL0) is less than Abs(currPocDiffL1)), the following applies:

[0248] td = Clip3(–128, 127, currPocDiffL1) (8 - 370)

[0249] tb = Clip3(–128, 127, currPocDiffL0) (8 - 371)

[0250] tx = (16384 + (Abs(td) >> 1)) / td (8 - 372)

[0251] distScaleFactor = Clip3(–4096, 4095, (tb * tx + 32) >> 6) (8 - 373)

[0252] mMvdL1[0] = mMvdOffset[xCb][yCb][0] (8 - 374)

[0253] mMvdL1[1] = mMvdOffset[xCb][yCb][1] (8 - 375)

[0254] mMvdL0[0]=Clip3(–2 15 ,2 15 –1,Sign(distScaleFactor*mMvdL1[0])*((Abs(distScaleFactor*mMvdL1[0])+127)>>8)) (8-376)

[0255] mMvdL0[1]=Clip3(–2 15 ,2 15 –1,Sign(distScaleFactor*mMvdL1[1])*((Abs(distScaleFactor*mMvdL1[1])+127)>>8)) (8-377)

[0256] Otherwise (predFlagL0 or predFlagL1 is equal to 1), for X 0 and 1, the following applies:

[0257] mMvdLX[0]=(predFlagLX==1)? MmvdOffset[xCb][yCb][0]:0 (8-378)

[0258] mMvdLX[1]=(predFlagLX==1)? MmvdOffset[xCb][yCb][1]:0 (8-379)

[0259] The decoding process of inter-frame blocks is described.

[0260] This process is called when decoding a coding unit that was encoded in inter prediction mode.

[0261] The input to this process is:

[0262] Luma position (xCb, yCb), which indicates the position of the top-left sample of the current coding block relative to the top-left luma sample of the current image;

[0263] The variable cbWidth represents the width of the current coding block calculated in luminance samples;

[0264] The variable cbHeight represents the height of the current coding block calculated in luminance samples;

[0265] The variables numSbX and numSbY represent the number of luminance coding sub-blocks in the horizontal and vertical directions;

[0266] Motion vectors mvL0[xSbIdx][ySbIdx] and mvL1[xSbIdx][ySbIdx], where xSbIdx=0..numSbX–1 and ySbIdx=0..numSbY–1;

[0267] Corrected motion vectors refMvL0[xSbIdx][ySbIdx] and refMvL1[xSbIdx][ySbIdx], where xSbIdx=0..numSbX–1 and ySbIdx=0..numSbY–1;

[0268] Reference indexes refIdxL0 and refIdxL1;

[0269] The prediction list utilizes flags predFlagL0[xSbIdx][ySbIdx] and predFlagL1[xSbIdx][ySbIdx], where xSbIdx=0..numSbX–1 and ySbIdx=0..numSbY–1;

[0270] Bidirectional prediction weight index gbiIdx;

[0271] The variable cIdx represents the color component index of the current block.

[0272] The output of this process is:

[0273] The array of prediction samples, predSamples.

[0274] Let predSamplesL0 L 、predSamplesL1 L and predSamplesIntra L is the (cbWidth)×(cbHeight) array of predicted luminance sample values, predSamplesL0 Cb 、predSamplesL1 Cb 、predSamplesL0 Cr and predSamplesL1 Cr 、predSamplesIntra Cb and predSamplesIntra Cr is a (cbWidth / 2)×(cbHeight / 2) array of predicted chroma sample values.

[0275] The width subCbWidth and height subCbHeight of the current coding subblock calculated using luminance samples are derived as follows:

[0276] sbWidth=cbWidth / numSbX (8-793)

[0277] sbHeight=cbHeight / numSbY (8-794)

[0278] For each coded subblock at subblock index (xSbIdx, ySbIdx), where xSbIdx = 0..numSbX - 1 and ySbIdx = 0..numSbY - 1, the following applies.

[0279] The luma position (xSb, ySb) represents the position of the upper left sample of the current coding sub-block relative to the upper left luma sample of the current image, which is derived as follows:

[0280] (xSb,ySb)=(xCb+xSbIdx*sbWidth,yCb+ySbIdx*sbHeight) (8-795)

[0281] The variable currPic represents the current image, and the variable bdofFlag is derived as follows.

[0282] bdofFlag is set to true if all of the following conditions are met.

[0283] sps_bdof_enabled_flag is equal to 1.

[0284] predFlagL0[xSbIdx][ySbIdx] and predFlagL1[xSbIdx][ySbIdx] are both equal to 1.

[0285] DiffPicOrderCnt(currPic,RefPicList[0][refIdxL0])*DiffPicOrderCnt(currPic,RefPicList[1][refIdxL1]) is less than 0.

[0286] MotionModelIdc[xCb][yCb] is equal to 0.

[0287] merge_subblock_flag[xCb][yCb] is equal to 0.

[0288] GbiIdx[xCb][yCb] is equal to 0.

[0289] cIdx is equal to 0.

[0290] The nuh_layer_id of RefPicList[0][refIdxL0] is the same as the nuh_layer_id of the current picture.

[0291] The nuh_layer_id of RefPicList[1][refIdxL1] is the same as the nuh_layer_id of the current picture.

[0292] Otherwise, bdofFlag is set to false.

[0293] For each of X being 0 and 1, when predFlagLX[xSbIdx][ySbIdx] is equal to 1, the following applies.

[0294] The ordered two-dimensional array refPicLX of luma samples is derived by calling the procedure specified in Section 8.5.7.2 L and two ordered two-dimensional arrays refPicLX of chroma samples Cb and refPicLX Cr The reference image composed of X and refIdxLX is taken as input.

[0295] The motion vector offset mvOffset is set equal to refMvLX[xSbIdx][xSbIdx]–mvLX[xSbIdx][ySbIdx].

[0296] mvOffset[0] is set to 0 when one or more of the following conditions are met:

[0297] xSb is not equal to xCb, mvOffset[0] is less than 0;

[0298] (xSb+sbWidth) is not equal to (xCb+cbWidth) and mvOffset[0] is greater than 0.

[0299] mvOffset[1] is set to 0 when one or more of the following conditions are met:

[0300] ySb is not equal to yCb, mvOffset[1] is less than 0;

[0301] (ySb+sbHeight) is not equal to (yCb+cbHeight) and mvOffset[1] is greater than 0.

[0302] If cIdx is equal to 0, the following applies:

[0303] The array predSamplesLX is derived by calling the fractional sample interpolation procedure specified in Section 8.5.7.3 L, where the luma position (xCb, yCb), the coding subblock width sbWidth and coding subblock height sbHeight calculated in terms of luma samples, the luma motion vector offset mvOffset, the corrected luma motion vector refMvLX[xSb][xSb], the reference array refPicLX L , bdofFlag and cIdx as input.

[0304] Otherwise, if cIdx is equal to 1, the following applies.

[0305] The array predSamplesLX is derived by calling the fractional sample interpolation procedure specified in Section 8.5.7.3 Cb , where the luma position (xCb, yCb), coding subblock width sbWidth / 2, coding subblock height sbHeight / 2, chroma motion vector offset mvOffset, corrected chroma motion vector refMvLX[xSb][xSb], reference array refPicLX Cb , bdofFlag and cIdx as input.

[0306] Otherwise (cIdx equals 2), the following applies.

[0307] The array predSamplesLX is derived by calling the fractional sample interpolation procedure specified in Section 8.5.7.3 Cr , where the luma position (xCb, yCb), coding subblock width sbWidth / 2, coding subblock height sbHeight / 2, chroma motion vector offset mvOffset, corrected chroma motion vector refMvLX[xSb][xSb], reference array refPicLX Cr , bdofFlag and cIdx as input.

[0308] If bdofFlag is equal to true, the following applies.

[0309] Variable shift is set to Max(2,14–BitDepth Y ).

[0310] The variables cuDiffThres, bdofBlkDiffThres, and cuSumDiff are derived as follows:

[0311] cuDiffThres = ( 1 << ( BitDepth Y – 8 + shift ) ) * cbWidth*cbHeight(8-796)

[0312] bdofBlkDiffThres = 1 << ( BitDepth Y – 3 + shift ) (8-797)

[0313] cuSumDiff = 0 (8-798)

[0314] For xIdx=0..(sbWidth>>2)–1 and yIdx=0..(sbHeight>>2)–1, the variable bdofBlkSumDiff and the bidirectional optical flow utilization flag bdofUtilizationFlag[xIdx][yIdx] are derived as follows:

[0315]

[0316] bdofUtilizationFlag[ xIdx ][ yIdx ] = bdofBlkSumDiff > = bdofBlkDiffThres (8-800)

[0317] cuSumDiff += bdofBlkSumDiff (8-801)

[0318] When cuSumDiff is less than cuDiffThres, bdofFlag is set to false.

[0319] The array predSamples of prediction samples is derived as follows.

[0320] If cIdx is equal to 0, the predicted samples predSamples[x L +xSb][y L +ySb](x L =0..sbWidth–1 and y L =0..sbHeight–1) is derived as follows:

[0321] If bdofFlag is equal to true, the bidirectional optical flow sample prediction process specified in Section 8.5.7.4 is called, where nCbW is set equal to the luma coding subblock width sbWidth, nCbH is set equal to the luma coding subblock height sbHeight, and the sample array predSamplesL0 is set equal to the sample array predSamplesL1. L and predSamplesL1 L, and variables predFlagL0[xSbIdx][ySbIdx], predFlagL1[xSbIdx][ySbIdx], refIdxL0, refIdxL1, and bdofUtilizationFlag[xIdx][yIdx] (xIdx=0..(sbWidth>>2)–1, yIdx=0..(sbHeight>>2)–1) as input, predSamples[x L +xSb][y L +ySb] as output.

[0322] Otherwise (bdofFlag is equal to false), the weighted sample prediction process specified in Section 8.5.7.5 is called, where the luma coding subblock width sbWidth, the luma coding subblock height sbHeight, the sample array predSamplesL0 L and predSamplesL1 L , and variables predFlagL0[xSbIdx][ySbIdx], predFlagL1[xSbIdx][ySbIdx], refIdxL0, refIdxL1, gbiIdx, and cIdx as input, predSamples[x L +xSb][y L +ySb] as output.

[0323] Otherwise, if cIdx is equal to 1, the prediction samples predSamples[x C +xCb / 2][y C +yCb / 2](x C =0..cbWidth / 2–1 and y C =0..cbHeight / 2–1), where the coding block width nCbW is set equal to cbWidth / 2, the coding block height nCbH is set equal to cbHeight / 2, and the sample array predSamplesL0 Cb and predSamplesL1 Cb , and variables predFlagL0[xSbIdx][ySbIdx], predFlagL1[xSbIdx][ySbIdx], refIdxL0, refIdxL1, gbiIdx, and cIdx as input.

[0324] Otherwise (cIdx is equal to 2), the prediction samples predSamples[x C +xCb / 2][y C +yCb / 2](x C =0..cbWidth / 2–1 and y C =0..cbHeight / 2–1), where the coding block width nCbW is set equal to cbWidth / 2, the coding block height nCbH is set equal to cbHeight / 2, and the sample array predSamplesL0 Cr and predSamplesL1 Cr , and variables predFlagL0[xSbIdx][ySbIdx], predFlagL1[xSbIdx][ySbIdx], refIdxL0, refIdxL1, gbiIdx, and cIdx as input.

[0325] When cIdx is equal to 0, for x = 0..sbWidth – 1 and y = 0..sbHeight – 1, the following assignments are made:

[0326] MvL0[ xSb + x ][ ySb + y ] = mvL0[ xSbIdx ][ ySbIdx ] (8-802)

[0327] MvL1[ xSb + x ][ ySb + y ] = mvL1[ xSbIdx ][ ySbIdx ] (8-803)

[0328] MvDmvrL0[ xSb + x ][ ySb + y ] = refMvL0[ xSbIdx ][ ySbIdx ] (8-804)

[0329] MvDmvrL1[ xSb + x ][ ySb + y ] = refMvL1[ xSbIdx ][ ySbIdx ] (8-805)

[0330] RefIdxL0[ xSb + x ][ ySb + y ] = refIdxL0 (8-806)

[0331] RefIdxL1[ xSb + x ][ ySb + y ] = refIdxL1 (8-807)

[0332] PredFlagL0[ xSb + x ][ ySb + y ] = predFlagL0[ xSbIdx ][ ySbIdx ](8-808)

[0333] PredFlagL1[ xSb + x ][ ySb + y ] = predFlagL1[ xSbIdx ][ ySbIdx ](8-809)

[0334] GbiIdx[ xSb + x ][ ySb + y ] = gbiIdx (8-810)

[0335] When ciip_flag[xCb][yCb] is equal to 1, the array of prediction samples predSamples is modified as follows.

[0336] If cIdx is equal to 0, the following applies.

[0337] Invokes the general intra sample prediction process specified in section 8.4.4.2.1 with position (xTbCmp, yTbCmp) set equal to (xCb, yCb), intra prediction mode predModeIntra set equal to IntraPredModeY[xCb][yCb], transform block width nTbW set equal to cbWidth and height nTbH equal to cbHeight, coding block width nCbW set equal to cbWidth and height nCbH equal to cbHeight, and variable cIdx as input, and assigns the output to the (cbWidth) × (cbHeight) array predSamplesIntra L .

[0338] Invoke the weighted sample prediction process of the combined fusion and intra prediction specified in Section 8.5.7.6, where the coding block width cbWidth and the coding block height cbHeight are set equal to predSamples and predSamplesIntra respectively. L The sample arrays predSamplesInter and predSamplesIntra, the intra prediction mode predModeIntra set equal to IntraPredModeY[xCb][yCb], and the color component index cIdx are taken as input and the output is assigned to the (cbWidth)×(cbHeight) array predSamples.

[0339] Otherwise, if cIdx is equal to 1, the following applies.

[0340] Invokes the general intra sample prediction process specified in section 8.4.4.2.1 with position (xTbCmp, yTbCmp) set equal to (xCb / 2, yCb / 2), intra prediction mode predModeIntra set equal to IntraPredModeY[xCb][yCb], transform block width nTbW set equal to cbWidth / 2 and height nTbH equal to cbHeight / 2, coding block width nCbW set equal to cbWidth / 2 and height nCbH equal to cbHeight / 2, and variable cIdx as input, and assigns the output to the (cbWidth / 2) × (cbHeight / 2) array predSamplesIntra Cb .

[0341] Invoke the weighted sample prediction process of the combined fusion and intra prediction specified in Section 8.5.7.6, where the coding block width cbWidth / 2, coding block height cbHeight / 2, and predSamples are set equal to each other. Cb and predSamplesIntra Cb The sample arrays predSamplesInter and predSamplesIntra, the intra prediction mode predModeIntra set equal to IntraPredModeY[xCb][yCb], and the color component index cIdx are taken as input and the output is assigned to the (cbWidth / 2)×(cbHeight / 2) array predSamples.

[0342] Otherwise (cIdx equals 2), the following applies:

[0343] Invokes the general intra sample prediction process specified in section 8.4.4.2.1 with position (xTbCmp, yTbCmp) set equal to (xCb / 2, yCb / 2), intra prediction mode predModeIntra set equal to IntraPredModeY[xCb][yCb], transform block width nTbW set equal to cbWidth / 2 and height nTbH equal to cbHeight / 2, coding block width nCbW set equal to cbWidth / 2 and height nCbH equal to cbHeight / 2 / , and variable cIdx as input, and assigns the output to the (cbWidth / 2) × (cbHeight / 2) array predSamplesIntra Cr .

[0344] Invoke the weighted sample prediction process of the combined fusion and intra prediction specified in Section 8.5.7.6, where the coding block width cbWidth / 2, coding block height cbHeight / 2, and predSamples are set equal to each other. Cr and predSamplesIntra Cr The sample arrays predSamplesInter and predSamplesIntra, the intra prediction mode predModeIntra set equal to IntraPredModeY[xCb][yCb], and the color component index cIdx are taken as input and the output is assigned to the (cbWidth / 2)×(cbHeight / 2) array predSamples.

[0345] Figure 11 A schematic diagram of a video decoding device 1100 (e.g., video encoder 20 or video decoder 30) provided in an embodiment of the present invention. Video decoding device 1100 is suitable for implementing the disclosed embodiments described herein. Video decoding device 1100 includes: an input port 1110 and a receiving unit (Rx) 1120 for receiving data; a processor, logic unit, or central processing unit (CPU) 1130 for processing data; a transmitting unit (Tx) 1140 and an output port 1150 for transmitting data; and a memory 1160 for storing data. Video decoding device 1100 may also include optical-to-electrical (OE) components and electrical-to-optical (EO) components coupled to input port 1110, receiving unit 1120, transmitting unit 1140, and output port 1150, serving as an outlet or inlet for optical or electrical signals.

[0346] Processor 1130 is implemented using both hardware and software. Processor 1130 can be implemented as one or more CPU chips, cores (e.g., a multi-core processor), field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), and digital signal processors (DSPs). Processor 1130 communicates with input port 1110, receiving unit 1120, transmitting unit 1140, output port 1150, and memory 1160. Processor 1130 includes a decoding module 1170. Decoding module 1170 implements the disclosed embodiments described above. For example, decoding module 1170 implements, processes, prepares, or provides various codec functions. Therefore, the inclusion of decoding module 1170 significantly improves the functionality of video decoding device 1100 and enables transitions of video decoding device 1100 to different states. Alternatively, decoding module 1170 can be implemented using instructions stored in memory 1160 and executed by processor 1130.

[0347] The video decoding device 1100 may also include input and / or output (I / O) devices 1180 for communicating data with a user. The I / O devices 1180 may include output devices, such as a display for displaying video data and speakers for outputting audio data. The I / O devices 1180 may also include input devices such as a keyboard, mouse, and trackball, and / or corresponding interfaces for interacting with such output devices.

[0348] Memory 1160 includes one or more disks, tape drives, and solid-state drives and can be used as an overflow data storage device for storing programs when such programs are selected for execution and for storing instructions and data read during program execution. Memory 1160 can be volatile and / or non-volatile and can be read-only memory (ROM), random access memory (RAM), ternary content-addressable memory (TCAM), and / or static random-access memory (SRAM).

[0349] Figure 121 is a schematic diagram of an embodiment of a decoding module 1200. In one embodiment, decoding module 1200 is implemented in a video decoding device 1202 (e.g., video encoder 20 or video decoder 30). Video decoding device 1202 includes a receiving module 1201. Receiving module 1201 is configured to receive an image for encoding or a bitstream for decoding. Video decoding device 1202 includes a transmitting module 1207 coupled to receiving module 1201. Transmitting module 1207 is configured to transmit the bitstream to a decoder or transmit the decoded image to a display module (e.g., one of the multiple I / O devices 1180).

[0350] Video decoding device 1202 includes a storage module 1203. Storage module 1203 is coupled to at least one of receiving module 1201 or transmitting module 1207. Storage module 1203 is configured to store instructions. Video decoding device 1202 also includes a processing module 1205. Processing module 1205 is coupled to storage module 1203. Processing module 1205 is configured to execute instructions stored in storage module 1203 to perform the methods disclosed herein.

[0351] It should also be understood that the steps of the exemplary methods described herein do not necessarily need to be performed in the order described, and the order of the steps of these methods should be understood to be merely exemplary. Similarly, in methods consistent with various embodiments of the present invention, these methods may include other steps, and certain steps may be omitted or combined.

[0352] Although the present invention provides several embodiments, it should be understood that the disclosed systems and methods may be embodied in a variety of other specific forms without departing from the spirit or scope of the present invention. The present examples should be considered illustrative rather than restrictive, and the present invention is not limited to the details given herein. For example, various elements or components may be combined or integrated into another system, or some features may be omitted or not implemented.

[0353] In addition, without departing from the scope of the present invention, the techniques, systems, subsystems, and methods described and illustrated as discrete or separate in the various embodiments may be combined or integrated with other systems, modules, techniques, or methods. Other items shown or described as coupled to each other, or directly coupled to each other, or communicating with each other may be indirectly coupled or communicated with each other electrically, mechanically, or otherwise via some interface, device, or intermediate component. Other examples of variations, substitutions, and modifications may be determined by those skilled in the art and may be exemplified without departing from the spirit and scope disclosed herein.

Claims

1. A method for decoding an encoded video stream, characterized in that: The method comprises: Receiving an encoded video stream; Parsing the video code stream to obtain a motion vector, quantized transform coefficients, and syntax elements of a current block, wherein the quantized transform coefficients are used to generate a reconstructed residual block of the current block, and the syntax elements include a layer identifier corresponding to a current image and a layer identifier corresponding to a reference image of the current image, wherein the layer identifier corresponding to the current image and the layer identifier corresponding to the reference image are used to determine whether the reference image is in the same layer as the current image; When at least the reference image and the current image are in the same layer, setting a decoder-side motion vector correction DMVR flag to a first value, enabling DMVR for a current block of the current image, and correcting the motion vector; When the reference picture and the current picture are in different layers, setting the DMVR flag to a second value to disable the DMVR for the current block of the current picture; generating a prediction block of the current block according to the corrected motion vector; A reconstructed block is obtained according to the prediction block and the reconstructed residual block.

2. The method according to claim 1, characterized in that Also included is enabling reference picture resampling (RPR) for the entire coded video sequence (CVS) including the current picture even when the DMVR is disabled.

3. The method according to claim 1 or 2, characterized in that The layer identifier is designated as nuh_layer_id.

4. The method according to claim 1 or 2, characterized in that The DMVR flag is set in the slice header of the encoded video stream.

5. The method according to claim 1 or 2, characterized in that The method further includes displaying an image generated using the current block on a display of the electronic device.

6. A decoding device, characterized in that: The decoding device comprises: A receiver, configured to receive an encoded video stream; a memory coupled to the receiver, the processor configured to: Parsing the video code stream to obtain a motion vector, quantized transform coefficients, and syntax elements of a current block, wherein the quantized transform coefficients are used to generate a reconstructed residual block of the current block, and the syntax elements include a layer identifier corresponding to a current image and a layer identifier corresponding to a reference image of the current image, wherein the layer identifier corresponding to the current image and the layer identifier corresponding to the reference image are used to determine whether the reference image is in the same layer as the current image; When at least the reference image and the current image are in the same layer, setting a decoder-side motion vector correction DMVR flag to a first value, enabling DMVR for a current block of the current image, and correcting the motion vector; When the reference picture and the current picture are in different layers, setting the DMVR flag to a second value to disable the DMVR for the current block of the current picture; generating a prediction block of the current block according to the corrected motion vector; A reconstructed block is obtained according to the prediction block and the reconstructed residual block.

7. The decoding device according to claim 6, characterized in that Even when the motion vector is disabled, the processor enables reference picture resampling RPR for the entire coded video sequence CVS including the current picture.

8. The decoding device according to claim 6 or 7, characterized in that The layer identifier is designated as nuh_layer_id.

9. The decoding device according to claim 6 or 7, characterized in that The DMVR flag is set in the slice header of the encoded video stream.

10. The decoding device according to claim 6 or 7, characterized in that Also included is a display configured to display an image generated according to the current block.

11. A video encoding and decoding system, characterized in that: The system comprises: encoder; A decoder in communication with the encoder, wherein the decoder comprises a decoding device according to any one of claims 6 to 10.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions for being executed by a processor, so as to cause the processor to perform the method according to any one of claims 1 to 5.

13. A computer program product, characterized in that The method comprises computer executable instructions stored in a computer storage medium, which, when executed by a processor, cause a computer to perform the method according to any one of claims 1 to 5.

Images