Intra prediction method and apparatus

Abstract

To provide an intra prediction method and apparatus for reconstruction of unavailable reference samples.SOLUTION: A method includes the steps of: obtaining an intra prediction mode of a current block; deriving availability of a reference sample of a component of the current block; substituting unavailable reference samples by using the available reference samples; deriving a prediction of the current block based on the intra prediction mode and the substituted reference samples; and reconstructing the current block based on the prediction. The component includes a Cb component or a Cr component. Alternatively, the component includes a Chroma component. Since the method derives availability of the reference sample in each component, it can provide available information more accurately.SELECTED DRAWING: Figure 10

Description

[Technical Field]

[0001] [Related Applications] This application claims priority to U.S. Provisional Patent Application No. 62 / 770,736, filed November 21, 2018, entitled "Intra Prediction Method and Device," the disclosure of which is incorporated herein by reference in its entirety for all purposes.

[0002] [Technical field] FIELD OF THE INVENTION Embodiments of the present invention relate generally to the field of video coding, and more particularly to the field of intra prediction methods and apparatus. [Background technology]

[0003] The amount of video data required to render even a relatively short video can be substantial. This can pose challenges when data is streamed or otherwise communicated across communications networks with limited bandwidth capabilities. Therefore, video data is typically compressed before being communicated across today's telecommunications networks. When video is stored on a storage device, video size can also be an issue because memory resources may be limited. Video compression devices often use software and / or hardware to encode video data at the source before transmission or storage, thereby reducing the amount of data needed to represent a digital video image. The compressed data is then received at the destination by a video decompressor, which decodes the video data. With limited network resources and ever-increasing demands for higher video quality, improved compression and decompression techniques that increase compression ratios with little to no sacrifice in image quality are desirable. Summary of the Invention

[0004] The embodiments of the present disclosure provide an intra prediction apparatus and method for encoding and decoding images, and should not be construed as being limited to the examples detailed herein.

[0005] According to a first aspect of the present disclosure, a method includes obtaining an intra-prediction mode of a current block, deriving availability of reference samples for components of the current block, substituting available reference samples for unavailable reference samples, deriving a prediction of the current block based on the intra-prediction mode and the substituting reference samples, and reconstructing the current block based on the prediction. In an embodiment, the components include a Y component, a Cb component, or a Cr component. In another embodiment, the components include a luma component or a chroma component.

[0006] The first aspect of the present disclosure derives the availability of the reference sample in each component, and therefore can provide the availability information more accurately.

[0007] According to an implementation of the first aspect, all Cb samples in the reconstructed block are marked as available, or all Cr samples in the reconstructed block are marked as available.

[0008] The implementation of the first aspect stores the availability information for each sample in each component, and therefore can provide the availability information more accurately in intra prediction processing.

[0009] According to a second aspect of the present disclosure, a decoder includes processing circuitry configured to perform the steps of the above-described method.

[0010] According to a third aspect of the present disclosure, an encoder includes processing circuitry configured to perform the steps of the above-described method.

[0011] According to a fourth aspect of the present disclosure, a computer program product comprises program code which, when executed by a processor, performs the method set out above.

[0012] According to a fifth aspect of the present disclosure, a decoder for intra prediction includes one or more processing units and a non-transitory computer-readable storage medium connected to the one or more processing units, the non-transitory computer-readable storage medium storing program instructions to be executed by the one or more processing units to perform the above-mentioned method.

[0013] According to a sixth aspect of the present disclosure, an encoder for intra prediction includes one or more processing units and a non-transitory computer-readable storage medium connected to the one or more processing units, the non-transitory computer-readable storage medium storing program instructions to be executed by the one or more processing units to perform the above-described method.

[0014] The embodiments of the present disclosure further provide a decoding device and an encoding device for performing the above-mentioned method.

[0015] For purposes of clarity, any one of the above-described embodiments may be combined with any one or more of the other above-described embodiments to create new embodiments that are within the scope of the present disclosure.

[0016] These and other features will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings and claims. [Brief explanation of the drawings]

[0017] For a more complete understanding of the present disclosure, reference is now made to the following brief description taken in conjunction with the accompanying drawings and detailed description, wherein like reference numerals represent like parts.

[0018] [Figure 1A] 1 is a block diagram illustrating an exemplary encoding system in which embodiments of the present disclosure may be implemented.

[0019] [Figure 1B] FIG. 1 is a block diagram illustrating another exemplary encoding system in which embodiments of the present disclosure may be implemented.

[0020] [Figure 2] 1 is a block diagram illustrating an exemplary video encoder capable of implementing embodiments of the present disclosure.

[0021] [Figure 3] FIG. 2 is a block diagram illustrating an example of a video decoder capable of implementing embodiments of the present disclosure.

[0022] [Figure 4] FIG. 1 is a schematic diagram of a network device according to an exemplary embodiment of the present disclosure.

[0023] [Figure 5] 1B is a simplified block diagram of a device that may be used as one or both of the source device and destination device of FIG. 1A in accordance with an exemplary embodiment of the present disclosure.

[0024] [Figure 6] 1 is a description of an intra-prediction algorithm for H.265 / HEVC that may be utilized in the description of this disclosure.

[0025] [Figure 7] FIG. 1 is a graphical diagram illustrating a reference sample according to an exemplary embodiment of the present disclosure.

[0026] [Figure 8] FIG. 10 is a graphical diagram illustrating available and unavailable reference samples according to an exemplary embodiment of the present disclosure.

[0027] [Figure 9] 1 is a simplified flowchart for deriving a reconstructed signal according to an exemplary embodiment of the present disclosure.

[0028] [Figure 10] 1 is a simplified flowchart for deriving a prediction signal according to an exemplary embodiment of the present disclosure.

[0029] [Figure 11] FIG. 10 is a graphical diagram illustrating samples in a current block that are marked as available according to an exemplary embodiment of the present disclosure.

[0030] [Figure 12] FIG. 10 is a graphical diagram illustrating samples in a current block marked as available in a unit N×N according to an exemplary embodiment of the present disclosure.

[0031] [Figure 13] FIG. 10 is a graphical diagram illustrating samples at the right and bottom boundaries of a current block marked as available according to an exemplary embodiment of the present disclosure.

[0032] [Figure 14] FIG. 10 is a graphical diagram illustrating samples at the right and bottom boundaries of a current block marked as available in unit N according to an exemplary embodiment of the present disclosure.

[0033] [Figure 15] FIG. 10 is a graphical diagram illustrating samples at the right and bottom boundaries of a current block marked as available in a unit N×N according to an exemplary embodiment of the present disclosure.

[0034] [Figure 16] FIG. 1 is a graphical diagram illustrating a method for marking samples of chroma components according to an exemplary embodiment of the present disclosure.

[0035] [Figure 17] 31 is a block diagram illustrating an exemplary structure of a content supply system 3100 for implementing a content distribution service.

[0036] [Figure 18] FIG. 2 is a block diagram illustrating the structure of an example terminal device. DETAILED DESCRIPTION OF THE INVENTION

[0037] It should be understood at the outset that, although illustrative implementations of one or more embodiments are provided below, the disclosed systems and / or methods may be implemented using a variety of technologies, whether currently known or existing. The present disclosure should in no way be limited to the illustrative implementations, drawings, and technologies described below, including the exemplary designs and implementations shown and described herein, but may be modified within the scope of the appended claims, along with their full range of equivalents.

[0038] 1A is a schematic block diagram illustrating an exemplary encoding system 10 that may utilize bidirectional prediction techniques. As shown in FIG. 1A, encoding system 10 includes a source device 12 that provides encoded video data to be decoded at a later time by a destination device 14. In particular, source device 12 may provide the video data to destination device 14 via a computer-readable medium 16. Source device 12 and destination device 14 may include any of a wide range of devices, including desktop computers, notebook (i.e., laptop) computers, tablet computers, set-top boxes, telephone handsets such as so-called "smart" phones and so-called "smart" pads, televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some cases, source device 12 and destination device 14 may be equipped for wireless communication.

[0039] Destination device 14 may receive the encoded video data to be decoded via computer-readable medium 16. Computer-readable medium 16 may include any type of medium or device capable of moving encoded video data from source device 12 to destination device 14. In one example, computer-readable medium 16 may include a communications medium that enables source device 12 to transmit encoded video data directly to destination device 14 in real time. The encoded video data may be modulated according to a communications standard, such as a wireless communications protocol, and transmitted to destination device 14. The communications medium may include any wireless or wired communications medium, such as the radio frequency (RF) spectrum or one or more physical transmission lines. The communications medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communications medium may include routers, switches, base stations, or any other equipment that may be useful for enabling communications from source device 12 to destination device 14.

[0040] In some examples, the encoded data may be output from output interface 22 to a storage device. Similarly, the encoded data may be accessed from a storage device by an input interface. The storage device may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, a Blu-ray disc, a digital video disk (DVD), a Compact Disc Read-Only Memory (CD-ROM), flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In further examples, the storage device may correspond to a file server or another intermediate storage device that may store the encoded video generated by source device 12. Destination device 14 may access the stored video data from the storage device via streaming or download. The file server may be any type of server capable of storing and transmitting encoded video data to destination device 14. Exemplary file servers include a web server (e.g., for a website), a file transfer protocol (FTP) server, a network attached storage (NAS) device, or a local disk drive. Destination device 14 may access the encoded video data over any standard data connection, including an Internet connection. This may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a digital subscriber line (DSL), a cable modem, etc.), or a combination of both suitable for accessing encoded video data stored on a file server. The transmission of the encoded video data from the storage device may be a streaming transmission, a download transmission, or a combination thereof.

[0041] The techniques of this disclosure are not necessarily limited to wireless applications or settings. The techniques may be applied to video encoding in support of any of a variety of multimedia applications, such as over-the-air television broadcast, cable television transmission, satellite television transmission, Internet streaming video transmission such as dynamic adaptive streaming over HTTP (DASH), digital video encoded on a data storage medium, decoding of digital video stored on a data storage medium, or other applications. In some examples, encoding system 10 may be configured to support one-way or two-way video transmission to support applications such as video streaming, video playback, video broadcasting, and / or video telephony.

[0042] In the example of FIG. 1A , source device 12 includes video source 18, video encoder 20, and output interface 22. Destination device 14 includes input interface 28, video decoder 30, and display device 32. According to this disclosure, video encoder 200 of source device 12 and / or video decoder 300 of destination device 14 may be configured to apply techniques for bidirectional prediction. In other examples, the source and destination devices may include other components or configurations. For example, source device 12 may receive video data from an external video source, such as an external camera. Similarly, destination device 14 may interface with an external display device rather than including an integrated display device.

[0043] The illustrated encoding system 10 of FIG. 1A is merely an example. The techniques for bidirectional prediction may be performed by any digital video encoding and / or decoding device. While the techniques of this disclosure are generally performed by a video encoding device, the techniques may also be performed by a video encoder / decoder, typically referred to as a "CODEC." Additionally, the techniques of this disclosure may be performed by a video preprocessor. The video encoder and / or decoder may be a graphics processing unit (GPU) or similar device.

[0044] Source device 12 and destination device 14 are merely examples of such encoding devices, with source device 12 generating encoded video data for transmission to destination device 14. In some examples, source device 12 and destination device 14 may operate substantially symmetrically, such that source and destination devices 12, 14 each include video encoding and decoding components. Thus, encoding system 10 may support one-way or two-way video transmission between video devices 12, 14, e.g., for video streaming, video playback, video broadcasting, or video telephony.

[0045] Video source 18 of source device 12 may include a video capture device such as a video camera, a video archive containing previously captured video, and / or a video feed interface for receiving video from a video content provider. As a further alternative, video source 18 may generate computer-graphics-based data as the source video, or a combination of live video, archived video, and computer-generated video.

[0046] In some examples, when video source 18 is a video camera, source device 12 and destination device 14 may form a so-called camera phone or video phone. As noted above, however, the techniques described in this disclosure may be applicable to video encoding generally and may be applied to wireless and / or wired applications. In each case, captured, pre-captured, or computer-generated video may be encoded by video encoder 20. The encoded video information may then be output by output interface 22 to computer-readable medium 16.

[0047] Computer-readable medium 16 may include a transmission medium, such as a wireless broadcast or wired network transmission, or a storage medium (i.e., a non-transitory storage medium), such as a hard disk, flash drive, compact disc, digital video disc, Blu-ray disc, or other computer-readable medium. In some examples, a network server (not shown) may receive the encoded video data from source device 12 and provide the encoded video data to destination device 14, for example, via a network transmission. Similarly, a computing device of a media production facility, such as a disc stamping facility, may receive the encoded video data from source device 12 and produce discs including the encoded video data. Thus, computer-readable medium 16 may be understood to include one or more computer-readable media of various types, in various examples.

[0048] Input interface 28 of destination device 14 receives information from computer-readable medium 16. The information on computer-readable medium 16 may include syntax information defined by video encoder 20, including syntax elements that describe the characteristics and / or processing of blocks and other coding units, such as a group of pictures (GOP), that are also used by video decoder 30. Display device 32 displays the decoded video data to a user and may include any of a variety of display devices, such as a cathode ray tube (CRT), a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device.

[0049] Video encoder 200 and video decoder 300 may operate according to a video coding standard, such as the High Efficiency Video Coding (HEVC) standard currently under development, and may conform to the HEVC Test Model (HM). Alternatively, video encoder 200 and video decoder 300 may operate according to an industry standard, such as Motion Picture Expert Group (MPEG)-4, Part 10, the International Telecommunications Union Telecommunication Standardization Sector (ITU-T) H.264 standard, H.265 / HEVC, or an extension of such a standard, known as Advanced Video Coding (AVC). The techniques of this disclosure, however, are not limited to any particular coding standard. Other examples of video coding standards include MPEG-2 and ITU-T H.263. 1A , in some aspects, video encoder 200 and video decoder 300 may be integrated with an audio encoder and decoder, respectively, and may include a multiplexer-demultiplexer (MUX-DEMUX) unit or other hardware and software appropriate to handle the encoding of both audio and video within a common data stream or to separate the data streams. Where appropriate, the MUX-DEMUX unit may conform to the ITU H.223 multiplexer protocol or other protocols, such as the user datagram protocol (UDP).

[0050] The video encoder 200 and the video decoder 300 may each be implemented as any of a variety of suitable encoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When the techniques are implemented partially in software, a device may store instructions for the software in a suitable non-transitory computer-readable medium and execute the instructions in hardware using one or more processors to perform the techniques of this disclosure. The video encoder 200 and the video decoder 300 may each be included in one or more encoders or decoders, either of which may be integrated within the respective device as part of a combined encoder / decoder (CODEC). A device including the video encoder 200 and / or the video decoder 300 may include an integrated circuit, a microprocessor, and / or a wireless communication device, such as a mobile phone.

[0051] 1B is a schematic block diagram illustrating an example video encoding system 40 including the encoder 200 of FIG. 2 and / or the decoder 300 of FIG. 3, according to an example embodiment. System 40 can implement techniques of this disclosure, such as merge estimation in inter-prediction. In the illustrated implementation, video encoding system 40 may include an image device 41, a video encoder 20, a video decoder 300 (and / or a video coder implemented by logic 47 of a processing unit 46), an antenna 42, one or more processors 43, one or more memory stores 44, and / or a display device 45.

[0052] As shown, imaging device 41, antenna 42, processing unit 46, logic circuitry 47, video encoder 20, video decoder 30, processor 43, memory store 44, and / or display device 45 may be in communication with one another. As discussed, although shown with both video encoder 20 and video decoder 30, video encoding system 40 may include only video encoder 20 or only video decoder 30 in various practical scenarios.

[0053] As shown, in some examples, video encoding system 40 may include antenna 42. Antenna 42 may be configured to transmit or receive, for example, an encoded bitstream of video data. Further, in some examples, video encoding system 40 may include display device 45. Display device 45 may be configured to present the video data. As shown, in some examples, logic 47 may be implemented by processing unit 46. Processing unit 46 may include application-specific integrated circuit (ASIC) logic, a graphics processor, a general-purpose processor, etc. Video encoding system 40 may also include an optional processor 43, which may also include application-specific integrated circuit (ASIC) logic, a graphics processor, a general-purpose processor, etc. In some examples, logic 54 may be implemented by hardware, dedicated video encoding hardware, etc., and processor 43 may implement general-purpose software, an operating system, etc. Furthermore, memory store 44 may be any type of memory, such as volatile memory (e.g., Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), etc.), or non-volatile memory (e.g., flash memory, etc.). In a non-limiting example, memory store 44 may be implemented by a cache memory. In some examples, logic circuitry 47 may access memory store 44 (e.g., for implementing an image buffer). In other examples, logic circuitry 47 and / or processing unit 46 may include a memory store (e.g., a cache, etc.) for implementing an image buffer, etc.

[0054] In some examples, video encoder 200 implemented by logic circuitry may include an image buffer (e.g., by either processing unit 46 or memory store 44) and a graphics processing unit (e.g., by processing unit 46). The graphics processing unit may be communicatively coupled to the image buffer. The graphics processing unit may include video encoder 200 implemented by logic circuitry 47 to implement various modules such as those discussed with respect to FIG. 2 and / or any other encoder system or subsystem described herein. The logic circuitry may be configured to perform various operations as discussed herein.

[0055] Video decoder 30 may be implemented in a manner similar to that implemented by logic circuitry 47 to implement various modules such as those discussed with respect to decoder 300 of FIG. 3 and / or any other decoder system or subsystem described herein. In some examples, video decoder 300 may be implemented by logic circuitry and may include an image buffer (e.g., by either processing unit 46 or memory store 44) and a graphics processing unit (e.g., by processing unit 46). The graphics processing unit may be communicatively coupled to the image buffer. The graphics processing unit may include video decoder 300 as implemented by logic circuitry 47 to implement various modules such as those discussed with respect to FIG. 3 and / or any other decoder system or subsystem described herein.

[0056] In some examples, antenna 42 of video encoding system 40 may be configured to receive an encoded bitstream of video data. As discussed, the encoded bitstream may include data related to the encoding of video frames as discussed herein, such as data related to coding partitions (e.g., transform coefficients or quantized transform coefficients, optional indicators (as discussed), and / or data defining coding partitions), indicators, index values, mode selection data, etc. Video encoding system 40 may also include a video decoder 300 coupled to antenna 42 and configured to decode the encoded bitstream. Display device 45 is configured to present the video frames.

[0057] 2 is a block diagram illustrating an example of a video encoder 200 that may implement the techniques of the present application. The video encoder 200 may perform intra- and inter-coding of video blocks in video slices. Intra-coding relies on spatial prediction to reduce or remove spatial redundancy in video within a given video frame or picture. Inter-coding relies on temporal prediction to reduce or remove temporal redundancy in video within adjacent frames or pictures of a video sequence. Intra-mode (I-mode) may represent any of several spatial-based coding modes. Inter-modes, such as unidirectional prediction (P-mode) or bi-prediction (B-mode), may represent any of several temporal-based coding modes.

[0058] FIG. 2 shows a schematic / conceptual block diagram of an exemplary video encoder 200 configured to implement the techniques of this disclosure. In the example of FIG. 2, the video encoder 200 includes a residual calculation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210 and an inverse transform processing unit 212, a reconstruction unit 214, a buffer 216, a loop filter unit 220, a decoded picture buffer (DPB) 230, a prediction processing unit 260, and an entropy coding unit 270. The prediction processing unit 260 may include an inter estimation unit 242, an inter prediction unit 244, an intra estimation unit 252, an intra prediction unit 254, and a mode selection unit 262. The inter prediction unit 244 may further include a motion estimation unit (not shown). The video encoder 200 shown in FIG. 2 may also be referred to as a hybrid video encoder or a video encoder conforming to a hybrid video codec.

[0059] For example, the residual calculation unit 204, the transform processing unit 206, the quantization unit 208, the prediction processing unit 260, and the entropy coding unit 270 form a forward signal path of the encoder 200. Meanwhile, for example, the inverse quantization unit 210, the inverse transform processing unit 212, the reconstruction unit 214, the buffer 216, the loop filter 220, the decoded picture buffer (DPB) 230, and the prediction processing unit 260 form a backward signal path of the encoder, which corresponds to the signal path of the decoder (see decoder 300 in FIG. 3 ).

[0060] The encoder 200 is configured to receive, for example via an input 202, a picture 201 or a block 203 of a picture 201, e.g. a picture of a video or a sequence of pictures forming a video sequence. The picture block 203 may also be referred to as a current picture block or coded picture block (particularly in video coding, to distinguish the current picture from other pictures, e.g. previously coded and / or decoded pictures of the same video sequence, i.e. the video sequence which also contains the current picture), and the picture 201 as the current picture or coded picture.

[0061] Partitioning

[0062] An embodiment of encoder 200 may include a partition unit (not shown in FIG. 2) configured to partition picture 201 into multiple blocks, such as block 203, typically into multiple non-overlapping blocks. The partition unit may be configured to use the same block size for all pictures in a video sequence and a corresponding grid that defines the block sizes, or to vary the block size between pictures or subsets or groups of pictures, and to partition each picture into corresponding blocks.

[0063] In HEVC and other video coding specifications, a set of coding tree units (CTUs) may be generated to generate a coded representation of a picture. Each CTU may include a coding tree block of luma samples, two corresponding coding tree blocks of chroma samples, and syntax structures used to encode the coding tree block samples. For monochrome pictures or pictures with three distinct color planes, a CTU may include a single coding tree block and syntax structures used to encode the coding tree block samples. A coding tree block may be an N×N block of samples. A CTU may also be referred to as a "tree block" or "largest coding unit (LCU)." A CTU in HEVC may be broadly similar to a macroblock in other standards such as H.264 / AVC. However, a CTU is not necessarily limited to a particular size and may include one or more coding units (CUs). A slice may include an integer number of CTUs ordered consecutively in raster scan order.

[0064] In HEVC, CTUs are divided into CUs using a quadtree structure, denoted as a coding tree, to accommodate various local characteristics. The decision to code a picture region using inter-picture (temporal) or intra-picture (spatial) prediction is made at the CU level. A CU may include a coding block of luma samples and two corresponding coding blocks of chroma samples for a picture having a luma sample array, a Cb sample array, and a Cr sample array, as well as syntax structures used to code the samples of the coding block. For a monochrome picture or a picture with three distinct color planes, a CU may include a single coding block and syntax structures used to code the samples of the coding block. A coding block is an N×N block of samples. In some examples, CUs may be CTUs of the same size. Each CU is coded using one coding mode, which may be, for example, an intra coding mode or an inter coding mode. Other coding modes are also possible. The encoder 200 receives video data. The encoder 200 may code each CTU in a slice of a picture of the video data. As part of encoding a CTU, prediction processing unit 260 or another processing unit of encoder 200 (including but not limited to the units of encoder 200 shown in FIG. 2) may perform partitioning to divide the CTB of the CTU into progressively smaller blocks 203. The smaller blocks may be coding blocks of a CU.

[0065] The syntax data in the bitstream may also define the size of the CTU. A slice includes multiple consecutive CTUs in coding order. A video frame, image, or picture may be partitioned into one or more slices. As described above, each treeblock may be divided into coding units (CUs) according to a quadtree. Typically, a quadtree data structure includes one node per CU, with a root node corresponding to the treeblock (e.g., a CTU). If a CU is divided into four sub-CUs, the node corresponding to the CU includes four child nodes, each of which corresponds to one of the sub-CUs. The nodes in the quadtree structure include leaf nodes and non-leaf nodes. Leaf nodes do not have child nodes in the tree structure (i.e., leaf nodes are not further divided). The non-leaf nodes include the root node of the tree structure. For each non-root node among the nodes, the non-root node corresponds to a sub-CU of the CU corresponding to the parent node in the tree structure of the non-root node. Each non-leaf node has one or more child nodes in the tree structure.

[0066] Each node in the quadtree data structure may provide syntax data for the corresponding CU. For example, a node in the quadtree may include a split flag that indicates whether the CU corresponding to the node is split into sub-CUs. The syntax elements of a CU may be defined recursively and may depend on whether the CU is split into sub-CUs. If a CU is not further split, it is called a leaf CU. If a block of CUs is further split, it may generally be called a non-leaf CU. Each level of partitioning is a quadtree division into four sub-CUs. A black CU is an example of a leaf node (i.e., a block that is not further split).

[0067] A CU has a similar purpose to a macroblock in the H.264 standard, except that CUs have no size difference. For example, a treeblock may be divided into four child nodes (also called sub-CUs), and each child node, on the other hand, may be divided into four other child nodes. The final undivided child node is called a leaf node of the quadtree and includes a coding node, also called a leaf CU. Syntax data associated with the coded bitstream may define the maximum number of times a treeblock can be divided, called the maximum CU depth, and may also define the minimum size of a coding node. Therefore, the bitstream may also define the smallest coding unit (SCU). The term "block" is used to refer to either a CU, PU, ​​or TU in the context of HEVC, or to a similar data structure in the context of other standards (e.g., a macroblock and its sub-blocks in H.264 / AVC).

[0068] In HEVC, each CU can be further divided into 1, 2, or 4 PUs according to a PU partition type. Within one PU, the same prediction process is applied, and related information is transmitted to the decoder for each PU. After obtaining a residual block by applying a prediction process based on the PU partition type, the CU can be partitioned into transform units (TUs) according to another quadtree structure similar to the coding tree of the CU. One of the main features of the HEVC structure is that it has multiple partition concepts, including CUs, PUs, and TUs. A PU may be partitioned to have a non-square shape. Syntax data associated with a CU may, for example, describe the partitioning of the CU into one or more PUs. A TU can have a square or non-square (e.g., rectangular) shape, and syntax data associated with a CU may, for example, describe the partitioning of the CU into one or more TUs according to a quadtree. The partition mode may differ depending on whether the CU is coded in skip or direct mode, in intra prediction mode, or in inter prediction mode.

[0069] Versatile Video Coding (VVC) removes the separation of PU and TU concepts and supports more flexibility in CU partition shape. The size of a CU corresponds to the size of a coding node and may be square or non-square (e.g., rectangular) in shape. The size of a CU may range from 4x4 pixels (or 8x8 pixels) to the size of a treeblock with a maximum of 128x128 pixels or more (e.g., 256x256 pixels).

[0070] After the encoder 200 generates a prediction block (e.g., luma, Cb, and Cr prediction blocks) for a CU, the encoder 200 may generate a residual block for the CU. For example, the encoder 100 may generate a luma residual block for the CU. Each sample in the luma residual block of the CU indicates a difference between a luma sample in the predicted luma block of the CU and a corresponding sample in the original luma coding block of the CU. Further, the encoder 200 may generate a Cb residual block for the CU. Each sample in the Cb residual block of the CU may indicate a difference between a Cb sample in the predicted Cb block of the CU and a corresponding sample in the original Cb coding block of the CU. The encoder 200 may generate a Cr residual block for the CU. Each sample in the Cr residual block of the CU may indicate a difference between a Cr sample in the predicted Cr block of the CU and a corresponding sample in the original Cr coding block of the CU.

[0071] In some examples, encoder 200 skips applying a transform to a transform block. In such examples, encoder 200 may treat residual sample values ​​in the same manner as transform coefficients. Thus, in examples in which encoder 200 skips applying a transform, the following discussion of transform coefficients and coefficient blocks may be applicable to transform blocks of residual samples.

[0072] After generating a coefficient block (e.g., a luma coefficient block, a Cb coefficient block, or a Cr coefficient block), the encoder 200 may quantize the coefficient block to possibly reduce the amount of data used to represent the coefficient block and potentially provide further compression. Quantization typically refers to a process in which a range of values ​​is compressed into a single value. After the encoder 200 quantizes the coefficient block, the encoder 200 may entropy code syntax elements that indicate the quantized transform coefficients. For example, the encoder 200 may perform Context-Adaptive Binary Arithmetic Coding (CABAC) or other entropy coding techniques on the syntax elements that indicate the quantized transform coefficients.

[0073] The encoder 200 may output a bitstream of coded picture data 271, which includes a sequence of bits that form a representation of a coded picture and associated data. The bitstream thus includes a coded representation of the video data.

[0074] J. An et al., “Block partitioning structure for next generation video coding,” International Telecommunication Union, COM16-C966, September 2015 (hereinafter referred to as “VCEG proposal COM16-C966”), proposed a quad-tree-binary-tree (QTBT) partitioning technique for future video coding standards after HEVC. Simulations have shown that the proposed QTBT structure is more efficient than the quad-tree structure used in HEVC. In HEVC, inter-prediction of small blocks is restricted to reduce memory access for motion compensation, so bi-prediction is not supported for 4x8 and 8x4 blocks, and inter-prediction is not supported for 4x4 blocks. JEM's QTBT removes these restrictions.

[0075] In QTBT, a CU can have either a square or rectangular shape. For example, a coding tree unit (CTU) is first partitioned using a quadtree structure. The leaf nodes of the quadtree can be further partitioned using a binary tree structure. There are two types of binary tree partitioning: symmetric horizontal partitioning and symmetric vertical partitioning. In each case, the node is divided either horizontally or vertically by splitting the node in the middle. The leaf nodes of the binary tree are called coding units (CUs), and segmentation is used for prediction and transform processing without further partitioning. This means that CUs, PUs, and TUs have the same block size in the QTBT coding block structure. A CU is sometimes composed of coding blocks (CBs) of different color components, for example, in the case of P and B slices in 4:2:0 chroma format, one CU contains one luma CB and two chroma CBs, and sometimes composed of CBs of a single component, for example, in the case of an I slice, one CU contains only one luma CB or only two chroma CBs.

[0076] The following parameters are defined for the QTBT partitioning method:

[0077] -CTU size: Root node size of quadtree, same concept as HEVC.

[0078] -MinQTSize: The minimum allowed quadtree leaf node size.

[0079] -MaxBTSize: Maximum allowed binary tree root node size.

[0080] -MaxBTDepth: Maximum allowed binary tree depth.

[0081] -MinBTSize: The minimum allowed binary tree leaf node size.

[0082] In one example of a QTBT partition structure, the CTU size is set as 128x128 luma samples with two corresponding 64x64 blocks of chroma samples, MinQTSize is set as 16x16, MaxBTSize is set as 64x64, MinBTSize (for both width and height) is set as 4x4, and MaxBTDepth is set as 4. Quadtree partitioning is first applied to the CTU to generate quadtree leaf nodes. The quadtree leaf nodes may have sizes from 16x16 (i.e., MinQTSize) to 128x128 (i.e., CTU size). When a quadtree leaf node has a size equal to MinQTSize, no further quadtrees are considered. If a leaf quadtree node is 128x128, it is not further divided by a binary tree because its size exceeds MaxBTSize (i.e., 64x64). In other cases, the leaf quadtree node may be further partitioned by a binary tree. Therefore, the quadtree leaf node is also the root node of the binary tree, and it has a binary tree depth such as 0. When the binary tree depth reaches MaxBTDepth (i.e., 4), further divisions are not considered. When a binary tree node has a width equal to MinBTSize (i.e., 4), further horizontal divisions are not considered. Similarly, when a binary tree node has a height equal to MinBTSize, further vertical divisions are not considered. The leaf node of the binary tree is further processed by prediction and transform processing without any further partitions. In JEM, the maximum CTU size is 256x256 luma samples. The leaf node (CU) of the binary tree may be further processed (e.g., by performing prediction and transform processing) without any further partitions.

[0083] Furthermore, the QTBT scheme supports the ability for luma and chroma to have separate QTBT structures. Currently, in P and B slices, the luma and chroma CTBs in one CTU may share the same QTBT structure. However, in an I slice, the luma CTB may be partitioned into CUs by a QTBT structure, and the chroma CTB may be partitioned into chroma CUs by a different QTBT structure. This means that a CU in an I slice consists of a coded block of a luma component or a coded block of two chroma components, and a CU in a P or B slice consists of coded blocks of all three color components.

[0084] The encoder 200 applies a rate-distortion optimization (RDO) process to the QTBT structure to determine the block partitions.

[0085] Furthermore, a block partition structure named multi-type-tree (MTT) has been proposed in U.S. Patent Application Publication No. 20170208336 to replace the CU structures based on QT, BT, and / or QTBT. The MTT partition structure is still a recursive tree structure. In MTT, multiple different partition structures (e.g., three or more) are used. For example, according to the MTT technique, three or more different partition structures may be used for each non-leaf node of the tree structure at each depth of the tree structure. The depth of a node in the tree structure may represent the length of the path from the node to the root of the tree structure (e.g., the number of divisions). The partition structure may typically represent how many different blocks a block can be divided into. The partition structure may be a quad-tree partition structure that may divide a block into four blocks, a binary-tree partition structure that may divide a block into two blocks, or a ternary-tree partition structure that may divide a block into three blocks, or even a ternary-tree partition structure that may not divide a block in the middle. A partition structure may have multiple different partition types, which may further define how blocks are divided, including symmetric or asymmetric partitions, equal or uneven partitions, and / or horizontal or vertical partitions.

[0086] In MTT, at each depth in the tree structure, the encoder 200 may be configured to further divide the subtree using a particular partition type from among one of more than three partition structures. For example, the encoder 100 may be configured to determine a particular partition type from among QT, BT, triple-tree (TT), and other partition structures. In one example, the QT partition structure may include a square quadtree or rectangular quadtree partition type. The encoder 200 may partition a square block using a square quadtree partition by dividing the block into four equally sized square blocks along its center both horizontally and vertically. Similarly, the encoder 200 may partition a rectangular (e.g., non-square) block using a rectangular quadtree partition by dividing the rectangular block into four equally sized rectangular blocks along its center both horizontally and vertically.

[0087] The BT partition structure may include at least one of a horizontally symmetric binary tree, a vertically symmetric binary tree, a horizontally asymmetric binary tree, or a vertically asymmetric binary tree partition type. In the horizontally symmetric binary tree partition type, the encoder 200 may be configured to divide a block horizontally into two symmetric blocks of the same size along the center of the block. In the vertically symmetric binary tree partition type, the encoder 200 may be configured to divide a block vertically into two symmetric blocks of the same size along the center of the block. In the horizontally asymmetric binary tree partition type, the encoder 200 may be configured to divide a block horizontally into two blocks of different sizes. For example, one block may be 1 / 4 the size of the parent block and the other block may be 3 / 4 the size of the parent block, similar to the PART_2N×nU or PART_2N×nD partition type. In the vertically asymmetric binary tree partition type, the encoder 200 may be configured to divide a block vertically into two blocks of different sizes. For example, one block may be 1 / 4 the size of the parent block, and another block may be 3 / 4 the size of the parent block, similar to the PART_nL×2N or PART_nR×2N partition types. In another example, an asymmetric binary tree partition type may divide a parent block into sub-partitions of different sizes. For example, one sub-block may be 3 / 8 the size of the parent block, and another sub-block may be 5 / 8 the size of the parent block. Of course, such partition types may be either vertical or horizontal.

[0088] The TT partition structure differs from the QT or BT structures in that the TT partition structure does not divide blocks in the middle. The central regions of the blocks remain together in the same subblock. Unlike QT, which results in four blocks, or a binary tree, which results in two blocks, a division by the TT partition structure results in three blocks. Exemplary partition types by the TT partition structure include symmetric partition types (both horizontally and vertically) and asymmetric partition types (both horizontally and vertically). Furthermore, the symmetric partition types by the TT partition structure may be uneven / non-uniform or even / uniform. The asymmetric partition type by the TT partition structure is uneven / non-uniform. In one example, the TT partition structure may include at least one of the following partition types: horizontally even / uniform symmetric ternary tree, vertically even / uniform symmetric ternary tree, horizontally uneven / uniform symmetric ternary tree, vertically uneven / uniform symmetric ternary tree, horizontally uneven / uniform asymmetric ternary tree, or vertically uneven / uniform asymmetric ternary tree partition type.

[0089] In general, an uneven / non-uniform symmetric ternary tree partition type is a partition type that is symmetric about the center line of the block, but at least one of the three resulting blocks is not the same size as the other two. One suitable example is when the edge blocks are one-quarter the size of the block and the center block is one-half the size of the block. An even / uniform symmetric ternary tree partition type is a partition type that is symmetric about the center line of the block, but all resulting blocks are the same size. Such partitions are possible when the block height or width is a multiple of three, depending on the vertical or horizontal division. An uneven / non-uniform asymmetric ternary tree partition type is a partition type that is not symmetric about the center line of the block, but at least one of the resulting blocks is not the same size as the other two.

[0090] In examples where a block is divided into an asymmetric ternary tree partition type (e.g., at a subtree node), the encoder 200 and / or decoder 300 may apply a constraint such that two of the three partitions have the same size. Such a constraint may correspond to a restriction that the encoder 200 must follow when encoding the video data. Furthermore, in some examples, the encoder 200 and decoder 300 may apply a constraint such that, when dividing according to an asymmetric ternary tree partition type, the sum of the areas of two partitions is equal to the area of ​​the remaining partition.

[0091] In some examples, the encoder 200 may be configured to select from among all of the aforementioned partition types for each of the QT, BT, and TT partition structures. In other examples, the encoder 200 may be configured to simply determine the partition type from among a subset of the aforementioned partition types. For example, a subset of the aforementioned partition types (or other partition types) may be used for a particular block size or for a particular depth of the quadtree structure. The subset of supported partition types may be signaled in the bitstream for use by the decoder 200, or may be predetermined, allowing the encoder 200 and decoder 300 to determine the subset without any signaling.

[0092] In another example, the number of supported partition types may be fixed for all depths in all CTUs. That is, the encoder 200 and decoder 300 may be pre-configured to use the same number of partition types for any depth of the CTU. In another example, the number of supported partition types may vary and may depend on the depth, slice type, or other pre-encoded information. In one example, at depth 0 or depth 1 in the tree structure, only the QT partition structure is used. At depths greater than 1, each of the QT, BT, and TT partition structures may be used.

[0093] In some examples, the encoder 200 and / or decoder 300 may apply preconfigured constraints to supported partition types to avoid overlapping partitions for specific regions of a video picture or region of a CTU. In one example, when a block is partitioned according to an asymmetric partition type, the encoder 200 and / or decoder 300 may be configured to not further partition the largest sub-block partitioned from the current block. For example, when a square block is partitioned according to an asymmetric partition type (such as the PART_2N×nU partition type), the largest sub-block among all sub-blocks is a designated leaf node (such as the largest sub-block in the PART_2N×nU partition type) and cannot be further divided. However, smaller sub-blocks (such as the smaller sub-blocks in the PART_2N×nU partition type) can be further divided.

[0094] As another example of a constraint on supported partition types that may be applied to avoid overlapping partitions for a particular region, when a block is divided using an asymmetric partition type, the largest sub-block divided from the current block cannot be further divided in the same direction. For example, when a square block is divided using an asymmetric partition type (similar to the PART_2N×nU partition type), the encoder 200 and / or decoder 300 may be configured to not divide a large sub-block among all sub-blocks horizontally (similar to the largest sub-block in the PART_2N×nU partition type).

[0095] As another example where constraints on supported partition types may be applied to avoid difficulties in further partitioning, the encoder 200 and / or decoder 300 may be configured not to partition a block either horizontally or vertically when the width / height of the block is not a power of two (e.g., when the width / height is not 2, 4, 8, 16, etc.).

[0096] The above examples described how the encoder 200 may be configured to perform MTT partitioning. The decoder 300 may therefore also apply the same MTT partitioning as performed by the encoder 200. In some examples, how pictures of the video data are partitioned by the encoder 200 may be determined by applying the same set of predetermined rules in the decoder 300. However, in many situations, the encoder 200 may determine a particular partition structure and partition type to use based on a rate-distortion criterion for a particular picture of the video data being encoded. Thus, in order for the decoder 300 to determine the partitions for a particular picture, the encoder 200 may signal syntax elements in the encoded bitstream that indicate how the picture and the CTUs of the picture should be partitioned. The decoder 200 may parse such syntax elements and partition the picture and CTUs accordingly.

[0097] In one example, prediction processing unit 260 of video encoder 200 may be configured to perform any combination of the above-mentioned partitioning techniques, particularly for motion compensation, as described in more detail below.

[0098] Similar to picture 201, block 203 is again or can be thought of as a two-dimensional array or matrix of samples having intensity values ​​(sample values), but with smaller dimensions than picture 201. In other words, block 203 may contain, for example, one sample array (e.g., a luma array in the case of a monochrome picture 201), or three sample arrays (e.g., a luma and two chroma arrays in the case of a color picture 201), or any other number and / or type of arrays depending on the color format applied. The number of samples in the horizontal and vertical directions (or axes) of block 203 determine the size of block 203.

[0099] 2 is configured to encode a picture 201 block by block, for example, encoding and prediction is performed block by block 203.

[0100] Residual calculation

[0101] The residual calculation unit 204 is configured to calculate the residual block 205 based on the picture block 203 and the prediction block 265 (further details regarding the prediction block 265 will be provided later), for example by subtracting sample values ​​of the prediction block 265 from sample values ​​of the picture block 203 sample by sample (pixel by pixel) to obtain the residual block 205 in the sample domain.

[0102] conversion

[0103] The transform processing unit 206 is configured to apply a transform, such as a discrete cosine transform (DCT) or a discrete sine transform (DST), to the sample values ​​of the residual block 205 to obtain transform coefficients 207 in a transform domain. The transform coefficients 207 may also be referred to as transform residual coefficients and represent the residual block 205 in the transform domain.

[0104] The transform processing unit 206 may be configured to apply an integer approximation of a DCT / DST, such as the transform specified for HEVC / H.265. Compared to an orthogonal DCT transform, such an integer approximation is typically scaled by a specific factor. To maintain the norm of the residual blocks processed by the forward and inverse transforms, an additional scaling factor is applied as part of the transform processing. The scaling factor is typically selected based on specific constraints, such as the scaling factor being a power of two for shift operations, the bit depth of the transform coefficients, and a trade-off between accuracy and implementation cost. A specific scaling factor may be specified, for example, for the inverse transform, e.g., by the inverse transform processing unit 212 in the decoder 300 (and the corresponding inverse transform, e.g., by the inverse transform processing unit 212 in the encoder 200), and a corresponding scaling factor for the forward transform, e.g., by the transform processing unit 206 in the encoder 200, may be specified accordingly.

[0105] quantization

[0106] The quantization unit 208 is configured to quantize the transform coefficients 207 to obtain quantized transform coefficients 209, for example, by applying scalar quantization or vector quantization. The quantized transform coefficients 209 may also be referred to as quantized residual coefficients 209. The quantization process may reduce the bit depth associated with some or all of the transform coefficients 207. For example, an n-bit transform coefficient may be truncated to an m-bit transform coefficient during quantization, where n is greater than m. The degree of quantization may be changed by adjusting a quantization parameter (QP). For example, in scalar quantization, different scaling may be applied to achieve finer or coarser quantization. A smaller quantization step size corresponds to finer quantization, while a larger quantization step size corresponds to coarser quantization. An applicable quantization step size may be indicated by the quantization parameter (QP). The quantization parameter may, for example, be an index into a predetermined set of applicable quantization step sizes. For example, a small quantization parameter may correspond to finer quantization (smaller quantization step size), and a large quantization parameter may correspond to coarser quantization (larger quantization step size), and vice versa. Quantization may involve division by the quantization step size. The corresponding or inverse inverse quantization, e.g., by inverse quantization 210, may involve multiplication by the quantization step size. Embodiments according to some standards, e.g., HEVC, may be configured to use the quantization parameter to determine the quantization step size. Typically, the quantization step size may be calculated based on the quantization parameter using a fixed-point approximation of a formula involving division. Additional scaling factors for quantization and inverse quantization may be introduced to restore the norm of the residual block, which may change due to scaling used in the fixed-point approximation of the quantization step size and quantization parameter formulas. In one exemplary implementation, the scaling of the inverse transform and inverse quantization may be combined. Alternatively, customized quantization tables may be used and signaled from the encoder to the decoder, e.g., in the bitstream.Quantization is a lossy operation, and the loss increases with increasing quantization step size.

[0107] Inverse quantization unit 210 is configured to apply the inverse quantization of quantization unit 208 to the quantized coefficients, e.g., by applying the inverse of the quantization scheme applied by quantization unit 208, based on or using the same quantization step size as quantization unit 208, to obtain inverse quantized coefficients 211. The inverse quantized coefficients 211, also referred to as inverse quantized residual coefficients 211, may correspond to transform coefficients 207, although they are not typically the same as transform coefficients due to loss due to quantization.

[0108] The inverse transform processing unit 212 is configured to apply an inverse transform of the transform applied by the transform processing unit 206, such as an inverse discrete cosine transform (DCT) or an inverse discrete sine transform (DST), to obtain an inverse transformed block 213 in the sample domain. The inverse transformed block 213 may also be referred to as an inverse transformed dequantized block 213 or an inverse transformed residual block 213.

[0109] The reconstruction unit 214 (e.g., adder 214) is configured to add the inverse transform block 213 (i.e., the reconstructed residual block 213) to the prediction block 265 to obtain the reconstructed block 215 in the sample domain, for example, by adding the sample values ​​of the reconstructed residual block 213 and the sample values ​​of the prediction block 265.

[0110] Optionally, a buffer unit 216 (or "buffer" 216), e.g., a line buffer 216, is configured to buffer or store the reconstructed blocks 215 and their respective sample values, e.g., for intra-prediction. In further embodiments, the encoder may be configured to use the unfiltered reconstructed blocks and / or their respective sample values ​​stored in the buffer unit 216 for any kind of estimation and / or prediction, e.g., intra-prediction.

[0111] Embodiments of encoder 200 may be configured, for example, so that buffer unit 216 is used to store reconstructed blocks 215 not only for intra prediction 254 but also for loop filter unit 220 (not shown in FIG. 2), and / or so that buffer unit 216 and decoded picture buffer unit 230 form one buffer. Further embodiments may be configured to use filtered blocks 221 and / or blocks or samples from decoded picture buffer 230 (both not shown in FIG. 2) as input or basis for intra prediction 254.

[0112] Loop filter unit 220 (or “loop filter” 220) is configured to filter reconstructed block 215 to obtain filtered block 221, e.g., to smooth pixel transitions or otherwise improve video quality. Loop filter unit 220 is intended to represent one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or other filters, e.g., a bilateral filter or an adaptive loop filter (ALF), a sharpening or smoothing filter, or a collaborative filter. Although loop filter unit 220 is shown in FIG. 2 as being an in-loop filter, in other configurations, loop filter unit 220 may be implemented as a post-loop filter. Filtered block 221 may also be referred to as filtered reconstructed block 221. Decoded picture buffer 230 may store the reconstructed coded block after loop filter unit 220 performs a filtering operation on the reconstructed coded block.

[0113] An embodiment of the encoder 200 (respectively, the loop filter unit 220) may be configured to output loop filter parameters (such as sample adaptive offset information), e.g., directly or entropy coded by the entropy coding unit 270 or any other entropy coding unit, so that, for example, the decoder 300 may receive and apply the same loop filter parameters for decoding.

[0114] Decoded picture buffer (DPB) 230 may be a reference picture memory that stores reference picture data for use in encoding video data by video encoder 20. DPB 230 may be formed from any of a variety of memory devices, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magneto-resistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. DPB 230 and buffer 216 may be provided by the same memory device or separate memory devices. In some examples, decoded picture buffer (DPB) 230 is configured to store filtered blocks 221. The decoded picture buffer 230 may be further configured to store other previously filtered blocks, e.g., previously reconstructed and filtered block 221, of the same current picture or of a different picture, e.g., a previously reconstructed picture, and may provide a complete previously reconstructed, i.e., decoded, picture (and corresponding reference blocks and samples) and / or a partially reconstructed current picture (and corresponding reference blocks and samples), e.g., for inter prediction. In some examples, the decoded picture buffer (DPB) 230 is configured to store the reconstructed block 215 if the reconstructed block 215 is reconstructed but without in-loop filtering.

[0115] The prediction processing unit 260, also called the block prediction processing unit 260, is configured to receive or retrieve the block 203 (the current block 203 of the current picture 201) and reconstructed picture data, e.g., reference samples of the same (current) picture from the buffer 216 and / or reference picture data 231 from one or more previously decoded pictures from the decoded picture buffer 230, and to process such data for prediction, i.e., to provide a prediction block 265, which may be an inter-prediction block 245 or an intra-prediction block 255.

[0116] The mode selection unit 262 may be configured to select a prediction mode (e.g., intra or inter prediction mode) and / or a corresponding prediction block 245 or 255 to be used as the prediction block 265 for calculation of the residual block 205 and for reconstruction of the reconstruction block 215.

[0117] Embodiments of mode selection unit 262 may be configured to select a prediction mode (e.g., from those supported by prediction processing unit 260) that is most suitable, or in other words, that provides the smallest residual (smallest residual means better compression for transmission or storage) or the smallest signaling overhead (smallest signaling overhead means better compression for transmission or storage), or that considers or balances both. Mode selection unit 262 may be configured to determine the prediction mode based on rate distortion optimization (RDO), i.e., to select a prediction mode that provides the smallest rate distortion optimization or that is associated with a rate distortion that at least satisfies a prediction mode selection criterion.

[0118] The prediction processing (eg, by prediction processing unit 260) and mode selection (eg, by mode selection unit 262) performed by example encoder 200 are described in further detail below.

[0119] As mentioned above, the encoder 200 is configured to determine or select a best or optimal prediction mode from a (predetermined) set of prediction modes, which may include, for example, intra-prediction modes and / or inter-prediction modes.

[0120] The set of intra prediction modes may include 35 different intra prediction modes, e.g., omnidirectional modes such as DC (or average) mode and planar mode, or directional modes such as those defined in H.265, or may include 67 different intra prediction modes, e.g., omnidirectional modes such as DC (or average) mode and planar mode, or directional modes such as those defined in the currently under development H.266.

[0121] The set of inter prediction modes (or possible) depends on the available reference pictures (i.e., previous, at least partially decoded pictures, e.g., stored in DBP 230) and other inter prediction parameters, e.g., whether the entire reference picture or only a portion thereof, e.g., a search window region around the region of the current block of the reference picture, is used to search for the best matching reference block, and / or whether, e.g., pixel interpolation, e.g., half / semi-pel and / or quarter-pel interpolation, is applied.

[0122] In addition to the prediction modes mentioned above, skip mode and / or direct mode may be applied.

[0123] Prediction processing unit 260 may be further configured to partition block 203 into smaller block partitions or sub-blocks, e.g., using quad-tree-partitioning (QT), binary-tree-partitioning (BT), triple-tree-partitioning (TT), or any combination thereof repeatedly, and perform prediction on each block partition or sub-block, e.g., where mode selection includes selecting a prediction mode to be applied to each of the tree structure of partitioned block 203 and the block partitions or sub-blocks.

[0124] The inter prediction unit 244 may include a motion estimation (ME) unit and a motion compensation (MC) unit (not shown in FIG. 2). The motion estimation unit is configured to receive or obtain the picture block 203 (the current picture block 203 of the current picture 201) and at least one or more of the decoded picture 331 or previously reconstructed blocks, such as reconstructed blocks of one or more other / different previously decoded pictures 331, for motion estimation. For example, a video sequence may include the current picture and the previous decoded picture 331. In other words, the current picture and the previous decoded picture 331 may be part of or form a sequence of pictures that form the video sequence. The encoder 200 may be configured to, for example, select a reference block from multiple reference blocks of the same or different pictures among multiple other pictures, and provide the reference picture (or reference picture index, ...) and / or an offset (spatial offset) between the position (x, y coordinates) of the reference block and the position of the current block to a motion estimation unit (not shown in FIG. 2 ) as an inter-prediction parameter. This offset is also called a motion vector (MV). Merging is an important motion estimation tool used in HEVC and inherited to VVC. To perform merge estimation, the first thing to be done is to construct a merge candidate list, where each candidate contains all motion data, including, for each list, information on whether one or two reference picture lists are used as well as reference indexes and motion vectors. The merge candidate list is constructed based on the following candidates: a. up to four spatial merge candidates derived from five spatially neighboring (i.e., nearby) blocks; b. one temporal merge candidate derived from two temporally co-located blocks; and c. additional merge candidates including combined bi-predictive candidates and zero motion vector candidates.

[0125] The intra prediction unit 254 is further configured to determine the intra prediction block 255 based on the intra prediction parameters, e.g., the selected intra prediction mode. In either case, after selecting the intra prediction mode for the block, the intra prediction unit 254 is also configured to provide the intra prediction parameters, i.e., information indicative of the selected intra prediction mode for the block, to the entropy coding unit 270. In one example, the intra prediction unit 254 may be configured to perform any combination of the intra prediction techniques described below.

[0126] The entropy coding unit 270 is configured to apply an entropy coding algorithm or scheme (e.g., a variable length coding (VLC) scheme, a context adaptive VLC scheme (CALVC), an arithmetic coding scheme, a context adaptive binary arithmetic coding (CABAC)), a syntax-based context-adaptive binary arithmetic coding (SBAC), a probability interval partitioning entropy (PIPE) coding or another entropy coding method or technique) to the quantized residual coefficients 209, the inter-prediction parameters, the intra-prediction parameters, and / or the loop filter parameters, individually or jointly (or not at all), to obtain coded picture data 21, which can be output via an output 272, for example in the form of a coded bitstream 21. Encoded bitstream 21 may be transmitted to video decoder 30 or archived for later transmission or retrieval by video decoder 30. Entropy encoding unit 270 may further be configured to entropy encode other syntax elements of the current video slice being encoded.

[0127] Other structural variations of the video encoder 200 may be used to encode the video stream. For example, a non-transform-based encoder 200 may directly quantize the residual signal for a particular block or frame without having a transform processing unit 206. In another implementation, the encoder 200 may have the quantization unit 208 and the inverse quantization unit 210 combined into a single unit.

[0128] 3 shows an example video decoder 300 configured to implement the techniques of this disclosure. Video decoder 300 is configured to receive coded picture data (e.g., coded bitstream) 271, for example, coded by encoder 200, to obtain decoded picture 331. During the decoding process, video decoder 300 receives video data from video encoder 200, for example, a coded video stream representing picture blocks of coded video slices and associated syntax elements.

[0129] 3, the decoder 300 includes an entropy decoding unit 304, an inverse quantization unit 310, an inverse transform processing unit 312, a reconstruction unit 314 (e.g., adder 314), a buffer 316, a loop filter 320, a decoded picture buffer 330, and a prediction processing unit 360. The prediction processing unit 360 may include an inter prediction unit 344, an intra prediction unit 354, and a mode selection unit 362. The video decoder 300 may, in some examples, perform a decoding path that is typically reciprocal to the encoding paths described with respect to the video encoder 200 from FIG.

[0130] The entropy decoding unit 304 is configured to perform entropy decoding on the coded picture data 271 to obtain, e.g., quantized coefficients 309 and / or decoded coding parameters (not shown in FIG. 3), e.g., any or all of inter-prediction parameters, intra-prediction parameters, loop filter parameters, and / or other syntax elements. The entropy decoding unit 304 is further configured to forward the inter-prediction parameters, intra-prediction parameters, and / or other syntax elements to the prediction processing unit 360. The video decoder 300 may receive video slice-level and / or video block-level syntax elements.

[0131] The inverse quantization unit 310 may be functionally identical to the inverse quantization unit 110, the inverse transform processing unit 312 may be functionally identical to the inverse transform processing unit 112, the reconstruction unit 314 may be functionally identical to the reconstruction unit 114, the buffer 316 may be functionally identical to the buffer 116, the loop filter 320 may be functionally identical to the loop filter 120, and the decoded picture buffer 330 may be functionally identical to the decoded picture buffer 130.

[0132] Prediction processing unit 360 may include an inter prediction unit 344 and an intra prediction unit 354, where inter prediction unit 344 may be functionally similar to inter prediction unit 144 and intra prediction unit 354 may be functionally similar to intra prediction unit 154. Prediction processing unit 360 is typically configured to perform block prediction and / or obtain a prediction block 365 from coded data 21, and to receive or obtain (explicitly or implicitly) prediction-related parameters and / or information regarding a selected prediction mode, for example, from entropy decoding unit 304.

[0133] When a video slice is coded as an intra-coded (I) slice, intra prediction unit 354 of prediction processing unit 360 is configured to generate a prediction block 365 for a picture block of the current video slice based on a signaled intra prediction mode and data from a previously decoded block of the current frame or picture. When a video frame is coded as an inter-coded (i.e., B or P) slice, inter prediction unit 344 (e.g., a motion compensation unit) of prediction processing unit 360 is configured to generate a prediction block 365 for a video block of the current video slice based on a motion vector and other syntax elements received from entropy decoding unit 304. In inter prediction, the prediction block may be generated from one of the reference pictures in one of the reference picture lists. Video decoder 300 may construct the reference frame lists: List 0 and List 1, using a specified construction technique based on the reference pictures stored in DPB 330.

[0134] Prediction processing unit 360 is configured to determine prediction information for video blocks of the current video slice by parsing the motion vectors and other syntax elements, and to use the prediction information to generate a predictive block for the current video block being decoded. For example, prediction processing unit 360 uses some of the received syntax elements to determine the prediction mode (e.g., intra- or inter-prediction) used to encode the video blocks of the video slice, the inter-prediction slice type (e.g., B slice, P slice, or GPB slice), configuration information for one or more of the reference picture list of the slice, the motion vectors of each inter-coded video block of the slice, the inter-prediction state of each inter-coded video block of the slice, and other information for decoding the video blocks in the current video slice.

[0135] Inverse quantization unit 310 is configured to inverse quantize, or dequantize, the quantized transform coefficients provided in the bitstream and decoded by entropy decoding unit 304. The inverse quantization process may include using quantization parameters calculated by video encoder 100 to determine, for each video block in a video slice, the degree of quantization, and similarly the degree of inverse quantization, to be applied.

[0136] Inverse transform processing unit 312 is configured to apply an inverse transform, eg, an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process, to the transform coefficients to produce residual blocks in the pixel domain.

[0137] The reconstruction unit 314 (e.g., adder 314) is configured to add the inverse transform block 313 (i.e., the reconstructed residual block 313) to the prediction block 365 to obtain the reconstructed block 315 in the sample domain, for example, by adding the sample values ​​of the reconstructed residual block 313 and the sample values ​​of the prediction block 365.

[0138] The loop filter unit 320 (in the coding loop or after the coding loop) is configured to filter the reconstructed block 315 to obtain a filtered block 321, e.g., to smooth pixel transitions or otherwise improve video quality. In one example, the loop filter unit 320 may be configured to perform any combination of the filtering techniques described below. The loop filter unit 320 is intended to represent one or more filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or other filters, e.g., a bilateral filter or an adaptive loop filter (ALF), or a sharpening or smoothing filter, or a collaborative filter. Although the loop filter unit 320 is shown in FIG. 3 as being an in-loop filter, in other configurations, the loop filter unit 320 may be implemented as a post-loop filter.

[0139] The decoded video blocks 321 in a given frame or picture are then stored in a decoded picture buffer 330, which stores reference pictures used for subsequent motion compensation.

[0140] The decoder 300 is arranged to output the decoded pictures 311, for example via an output 312, for presentation or viewing to a user.

[0141] Other variations of the video decoder 300 may be used to decode the compressed bitstream. For example, the decoder 300 may generate an output video stream without the loop filter unit 320. For example, a non-transform-based decoder 300 may directly inverse quantize the residual signal for a particular block or frame without the inverse transform processing unit 312. In another implementation, the video decoder 300 may have the inverse quantization unit 310 and the inverse transform processing unit 312 combined into a single unit.

[0142] 4 is a schematic diagram of a network device 400 (e.g., an encoding device) according to an embodiment of the present disclosure. Network device 400 is suitable for implementing embodiments of the disclosure as described herein. In an embodiment, network device 400 may be a decoder, such as video decoder 30 of FIG. 1A, or an encoder, such as video encoder 20 of FIG. 1A. In an embodiment, network device 400 may be one or more components of video decoder 30 of FIG. 1A or video encoder 20 of FIG. 1A, as described above.

[0143] Network device 400 includes an ingress port 410 and a receiver unit (Rx) 420 for receiving data, a processor, logic unit, or central processing unit (CPU) 430 for processing the data, a transmitter unit (Tx) 440 and an egress port 450 for transmitting the data, and a memory 460 for storing the data. Network device 400 may also include optical-to-electrical (OE) and electrical-to-optical (EO) components connected to ingress port 410, receiver unit 420, transmitter unit 440, and egress port 450 for egress or ingress of optical or electrical signals.

[0144] The processor 430 is implemented in hardware and software. The processor 430 may be implemented as one or more CPU chips, cores (e.g., multi-core processors), FPGAs, ASICs, and DSPs. The processor 430 communicates with the ingress port 410, the receiver unit 420, the transmitter unit 440, the egress port 450, and the memory 460. The processor 430 includes an encoding module 470. The encoding module 470 implements the above-disclosed embodiments. For example, the encoding module 470 implements, processes, prepares, or provides various encoding operations. The inclusion of the encoding module 470 thus provides substantial improvements to the functionality of the network device 400 and results in the transformation of the network device 400 into a different state. Alternatively, the encoding module 470 is implemented as instructions stored in the memory 460 and executed by the processor 430.

[0145] Memory 460 may include one or more disks, tape drives, and solid state drives, and may be used to store programs when they are selected for execution and as overflow data storage for storing instructions and data read during the execution of the programs. Memory 460 may be volatile and / or non-volatile memory, and may be read-only memory (ROM), random access memory (RAM), ternary content-addressable memory (TCAM), and / or static random-access memory (SRAM).

[0146] 5 is a simplified block diagram of a device 500 that may be used as one or both of source device 12 and destination device 14 from FIG. 1A according to an example embodiment. Device 500 is capable of implementing the techniques of this disclosure. Device 500 may be in the form of a computing system including multiple computing devices, or in the form of a single computing device, such as a mobile phone, tablet computer, laptop computer, notebook computer, desktop computer, etc.

[0147] Processor 502 in device 500 may be a central processing unit. Alternatively, processor 502 may be any other type of device or devices, now existing or later developed, that can manipulate or process information. While implementations of the disclosure may be performed with a single processor, such as processor 502, as shown, benefits in speed and efficiency may be achieved using more than one processor.

[0148] The memory 504 in the device 500 may, in implementation, be a read only memory (ROM) device or a random access memory (RAM) device. Any other suitable type of storage device may be used as the memory 504. The memory 504 may include code and data 506 accessed by the processor 502 using a bus 512. The memory 504 may further include an operating system 508 and application programs 510. The application programs 510 include at least one program that enables the processor 502 to perform methods described herein. For example, the application programs 510 may include applications 1-N, further including a video encoding application that performs methods described herein. The device 500 may also include additional memory in the form of secondary storage 514, which may be, for example, a memory card used with a mobile computing device. Because video communication sessions may contain a significant amount of information, they may be stored in whole or in part in the secondary storage 514 and loaded into the memory 504 as needed for processing.

[0149] Device 500 may also include one or more output devices, such as a display 518. Display 518, in one example, may be a touch-sensitive display that combines a display with touch-sensitive elements operative to sense touch input. Display 518 may be coupled to processor 502 via bus 512. Other output devices that enable a user to program or otherwise use device 500 may be provided in addition to or as an alternative to display 518. When an output device is or includes a display, the display may be implemented in a variety of ways, including a liquid crystal display (LCD), a cathode-ray tube (CRT) display, a plasma display, or a light emitting diode (LED) display, such as an organic LED (OLED) display.

[0150] Device 500 may also include or communicate with image sensing device 520, such as a camera or any other existing or future developed image sensing device 520 capable of sensing images, such as an image of a user operating device 500. Image sensing device 520 may be positioned to be pointed at a user operating device 500. In one example, the position and optical axis of image sensing device 520 may be configured such that the field of view includes an area immediately adjacent to display 518 and is viewable from display 518.

[0151] Device 500 may also include or communicate with audio sensing device 522, such as a microphone or any other existing or future developed audio sensing device capable of sensing audio in the vicinity of device 500. Audio sensing device 522 may be positioned to face a user operating device 500 and may be configured to receive audio, e.g., speech or other utterances, generated by the user while the user is operating device 500.

[0152] While FIG. 5 depicts the processor 502 and memory 504 of device 500 as integrated into a single unit, other configurations are available. The operations of processor 502 can be distributed across multiple machines (each machine having one or more processors), which may be directly connected or across a local area or other network. Memory 504 can be distributed across multiple machines, such as a network-based memory or memory in multiple machines that perform the operations of device 500. While shown here as a single bus, bus 512 of device 500 can be comprised of multiple buses. Furthermore, secondary storage 514 can be directly connected to other components of device 500 or accessed over a network, and can include a single integrated unit, such as a memory card, or multiple units, such as multiple memory cards. Device 500 can therefore be implemented in a variety of configurations.

[0153] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over a computer-readable medium as one or more instructions or code and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media, which correspond to tangible media such as data storage media, or communication media, including any medium that facilitates transfer of a computer program from one place to another, for example, according to a communications protocol. As such, computer-readable media may generally correspond to (1) tangible computer-readable storage media that are non-transitory, or (2) communication media such as a signal or carrier wave. Data storage media may be any available medium accessible by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementing the techniques described in this disclosure. A computer program product may include a computer-readable medium.

[0154] Video compression techniques such as motion compensation, intra prediction, and loop filtering have proven efficient and have therefore been adopted in various video coding standards, such as H.264 / AVC and H.265 / HEVC. Intra prediction can be used when no reference picture is available or when inter-predictive coding is not used for the current block or picture, such as an I-frame or I-slice. Reference samples for intra prediction are usually derived from previously coded (or reconstructed) neighboring blocks within the same picture. For example, in both H.264 / AVC and H.265 / HEVC, boundary samples of neighboring blocks are used as references for intra prediction. There are many different intra prediction modes to cover different textures or structural features. Each mode uses a different prediction signal derivation method. For example, H.265 / HEVC supports a total of 35 intra prediction modes, as shown in Figure 6.

[0155] Explanation of the intra prediction algorithm in H.265 / HEVC

[0156] Intra prediction uses the decoded boundary samples of neighboring blocks as references. The encoder selects the best luma intra prediction mode for each block from 35 options: 33 directional prediction modes, DC mode, and planar mode. The mapping between intra prediction direction and intra prediction mode number is specified in Figure 6. It should be noted that 65 or more intra prediction modes have been developed in modern video coding techniques, such as versatile video coding (VVC), which can capture any edge direction that appears in the raw video.

[0157] As shown in Figure 7, block "CUR" is the current block to be predicted, and the gray samples along the boundaries of adjacent constructed blocks (left and top of the current block) are used as reference samples. The prediction signal can be derived by mapping the reference samples according to a specific method indicated by the intra prediction mode.

[0158] Referring to Figure 7, W is the width of the current block, and H is the height of the current block. W1 is the number of reference samples above. H1 is the number of reference samples to the left. Usually, W1>W and H1>H, which means that the reference sample above also includes the reference (neighboring) sample on the top right, and the reference sample to the left also includes the reference (neighboring) sample on the bottom left. For example, H1=2*H, W1=2*W, or H1=H+W, W1=W+H.

[0159] Reference samples are not always available. For example, as shown in Figure 8, after the availability check process, W2 samples above or above the current block are available, and H2 samples to the left of the current block are available. W3 samples above are unavailable, and H3 samples to the left are unavailable.

[0160] Those unavailable samples need to be replaced (or padded) with available samples before obtaining the predicted signal. For example, unavailable samples are replaced by scanning the reference samples in a clockwise direction and using the latest available sample value for the unavailable ones. For the lower part of the left reference samples, if they are unavailable, they are replaced by the value of the nearest available reference sample.

[0161] Reference sample availability check The reference sample availability check means checking whether the reference sample is available, for example, if the reference sample is reconstructed, then the reference sample is available.

[0162] In existing methods, the availability check process is usually performed by checking the luma samples, which means that for a block of chroma components, the reference sample availability is also derived by checking the availability of the corresponding luma samples.

[0163] In the proposed method of the present disclosure, for a block, the reference sample availability check process is performed by checking samples of its own component, where the component may be a Y component, a Cb component, or a Cr component. According to the proposed method, for a block, the reference sample availability check process is performed by checking samples of its corresponding component, where the component may be a luma component or a chroma component, which may include both a Cb component and a Cr component. Here, the Cb component and the Cr component are called chroma components and are not distinguished in the availability check process.

[0164] A set of methods is presented in this invention, emphasizing the following two aspects:

[0165] According to the first aspect of the present disclosure, for a block, a reference sample availability check process is performed by examining the samples of its own component, where the component may be a Y component, a Cb component, or a Cr component.

[0166] For a Y block, the availability of a reference sample for the Y block is derived by checking whether the reference sample is available, for example, by checking whether the reference sample has been reconstructed.

[0167] For a Cb block, the availability of a reference sample for the Cb block is derived by checking whether the reference sample is available, for example, by checking whether the reference sample is reconstructed.

[0168] For a Cr block, the availability of a reference sample for the Cr block is derived by checking whether the reference sample is available, for example, by checking whether the reference sample is reconstructed.

[0169] According to a second aspect of the present disclosure, for a block, a reference sample availability check process is performed by examining the samples of its corresponding component, where the component may be a luma component or a chroma component. The Y component is referred to as the luma component. Both the Cb component and the Cr component are referred to as the chroma component. Here, the Cb component and the Cr component are not distinguished in the availability check process.

[0170] For a luma block, the availability of a reference sample for the luma block is derived by checking whether the reference sample is available, for example, by checking whether the reference sample is reconstructed.

[0171] For a chroma block, the availability of a reference sample for the chroma block is derived by checking whether the reference sample is available, for example, by checking whether the reference sample is reconstructed.

[0172] In existing solutions, for a block, the reference sample availability check process is always performed by looking at the luma sample, regardless of which component the block belongs to, whereas in the proposed method of the present disclosure, for blocks that belong to different components, their reference sample availability check process is performed by looking at different component samples.

[0173] It should be noted that the method proposed in this disclosure is used to derive the availability of reference samples for a block coded as an intra-prediction mode. The method can be performed by the intra-prediction module 254 or 354 shown in Figures 2 and 3, respectively. Therefore, the method exists on both the decoder side and the encoder side. And the process of checking the availability of reference samples for a block is the same in the encoder and the decoder.

[0174] 9 is a simplified flowchart illustrating a method for deriving a reconstructed signal according to an exemplary embodiment of the present disclosure. Referring to FIG. 9, to obtain a reconstructed block (or signal, or sample) for a block coded in intra-prediction mode, the method first includes a step of obtaining a prediction (or predicted signal or sample) of the block (901). Then, the method includes a step of obtaining a residual (or residual signal) of the block (902). Next, the method includes a step of deriving a reconstructed block by adding the residual to a prediction of the current block (903).

[0175] 10 is a simplified flowchart of deriving a prediction (or prediction signal) according to an exemplary embodiment of the present disclosure. Referring to FIG. 10, to obtain the prediction signal, the method first includes a step (1001) of obtaining an intra-prediction mode of the current block, such as planar mode, DC mode, .... Then, the method includes a step (1002) of deriving the availability of reference samples for components of the current block. In an embodiment, the components include Y components, Cb components, or Cr components. In another embodiment, the components include luma components or chroma components. The method may derive the availability of reference samples by checking the availability of Y samples in a neighboring Y block, which includes the reference samples; alternatively, the method may derive the availability of reference samples by checking the availability of Cb samples in a neighboring Cb block, which includes the reference samples; alternatively, the method may derive the availability of reference samples by checking the availability of Cr samples in a neighboring Cr block, which includes the reference samples.

[0176] If the method determines that unavailable reference samples exist (YES at 1003), the method includes replacing or padding the unavailable reference samples with available reference samples (1004). The method then includes deriving a prediction for the block based on the intra-prediction mode and the replaced reference samples (1005). If the method determines that unavailable reference samples do not exist (NO at 1003), the method proceeds to 1005, which includes deriving a prediction for the current block based on the intra-prediction mode and the available reference samples. At 1006, the method includes reconstructing the current block based on the prediction.

[0177] It is noted here that the embodiments of the present disclosure described below are directed to processes that derive a prediction signal and improve the reference sample availability check process.

[0178] Embodiment 1 In this embodiment, the availability check process is performed by examining a sample of the component itself.

[0179] Referring to FIG. 10, for a block coded as an intra prediction mode, to obtain a prediction signal, the method includes:

[0180] Step 1: Get the intra prediction mode (1001)

[0181] The intra prediction mode is obtained by parsing the syntax related to the intra prediction mode in the bitstream. For example, for a Y block, intra_luma_mpm_flag, intra_luma_mpm_idx, or intra_luma_mpm_remainder is parsed. For a Cb or Cr block, the intra_chroma_pred_mode signal is parsed. Then, the intra prediction mode of the current block can be derived using the parsed syntax.

[0182] Step 2: Run the reference sample availability check process (1002)

[0183] This step involves checking the availability of a reference sample for the current block.

[0184] In an embodiment, when the block is a Y block, the reference sample availability is derived by examining the reference sample of the Y component. Alternatively, the reference sample availability is derived by examining neighboring Y samples, for example, by examining boundary samples of neighboring Y blocks.

[0185] In an embodiment, when the block is a Cb block, the reference sample availability is derived by examining the reference samples of the Cb component. Alternatively, the reference sample availability is derived by examining neighboring Cb samples, for example, by examining boundary samples of neighboring Cb blocks.

[0186] In an embodiment, when the block is a Cr block, the reference sample availability is derived by examining the reference samples of the Cr component. Alternatively, the reference sample availability is derived by examining neighboring Cr samples, for example, by examining boundary samples of neighboring Cr blocks.

[0187] Step 3: Determine if there are any unavailable reference samples (1003)

[0188] At 1003, the method includes determining whether there are unavailable reference samples. If the method determines that there are unavailable reference samples, the method proceeds to step 4 (1004); otherwise, the method proceeds to step 5 (1005).

[0189] Step 4: Reference sample replacement (1004)

[0190] Reference sample replacement means using the values ​​of available reference samples to derive sample values ​​for unavailable reference samples. In an embodiment, unavailable samples are replaced by scanning the reference samples in a clockwise direction and using the latest available sample value for the unavailable ones. For the lower part of the left reference samples, if they are unavailable, they are replaced by the value of the nearest available reference sample.

[0191] Step 5: Derive a predicted signal (1005)

[0192] After obtaining the reference samples and the intra prediction mode, the prediction signal can be derived by mapping the reference samples to the current block, and the mapping method is indicated by the intra prediction mode.

[0193] After the step of deriving a prediction signal for the current block, a reconstructed signal for the current block can be derived by adding a residual signal to the derived prediction signal (903 in FIG. 9).

[0194] It should be noted here that after the current block is reconstructed, the samples in the block region can be marked as "available", as shown in Fig. 11. That is, when the current block is a Y block, all Y samples in the positions covered by the block region are available; when the current block is a Cb block, all Cb samples in the positions covered by the block region are available; and when the current block is a Cr block, all Cr samples in the positions covered by the block region are available.

[0195] An example specification of these sections can be given below: Reference sample availability marking process The inputs to this process are: The sample position (xTbCmp, yTbCmp) that specifies the top-left sample of the current transform block relative to the top-left sample of the current picture. - Variable refIdx that specifies the intra-prediction reference line index -RefW variable that specifies the reference sample width Variable refH that specifies the reference sample height · Variable cIdx that specifies the color component of the current block The output of this process is the reference sample refUnfilt[x][y], and for intra-sample prediction x=-1-refIdx, y=-1-refIdx..refH-1, and x=-refIdx..refW-1, y=-1-refIdx. The refW+refH+1+(2*refIdx) neighboring samples refUnfilt[x][y], which are samples constructed before in-loop filtering, where x=-1-refIdx, y=-1-refIdx..refH-1 and x=-refIdx..refW-1, y=-1-refIdx, are derived as follows: The neighborhood position (xNbCmp, yNbCmp) is specified by: (xNbCmp,yNbCmp) =(xTbCmp+x,yTbCmp+y) (310) The neighbor block availability derivation process specified in Section 6.4.4 is called with the current sample position (xCurr, yCurr) set equal to (xTbCmp, yTbCmp), the neighbor sample positions (xNbCmp, yNbCmp), checkPredModeY set equal to FALSE, and cIdx as input, and the output is assigned to availableN. Each sample refUnfilt[x][y] is derived as follows: If availableN is equal to FALSE, the sample refUnfilt[x][y] is marked as "unavailable for intra prediction". Otherwise, the sample refUnfilt[x][y] is marked as "available for intra prediction" and the sample at position (xNbCmp, yNbCmp) is assigned to refUnfilt[x][y]. Neighboring block availability derivation process The inputs to this process are: The sample position (xCurr, yCurr) of the top-left sample of the current block relative to the top-left sample of the current picture. The sample position (xNbCmp, yNbCmp) covered by the neighboring blocks relative to the top-left luma sample of the current picture, · checkPredModeY variable that specifies whether availability depends on the prediction mode · Variable cIdx that specifies the color component of the current block The output of this process is the availability of neighboring blocks covering position (xNbCmp, yNbCmp), denoted as availableN. The current luma position (xTbY, yTbY) and the neighboring luma position (xNbY, yNbY) are derived as follows: (xTbY,yTbY)=(cIdx==0)?(xCurr,yCurr): (xCurr*SubWidthC,yCurr*SubHeightC) (xNbY,yNbY)=(cIdx==0)?(xNbCmp,yNbCmp): (xNbCmp*SubWidthC,yNbCmp*SubHeightC) The neighboring block availability, availableN, is derived as follows: · If one or more of the following conditions are TRUE, then availableN is set equal to FALSE: xNbCmp is less than 0. yNbCmp is less than 0. xNbY is greater than or equal to pic_width_in_luma_samples. yNbY is greater than or equal to pic_hight_in_luma_samples. IsAvailable[cIdx][xNbCmp] "yNbCmp] is equal to FALSE. The neighboring block is in a different slice than the current block. The neighboring block is in a different tile than the current block. ·entropy_coding_sync_enabled_flag is equal to 1 and (xNbY>>CtbLog2SizeY) is greater than or equal to (xTbY>>CtbLog2SizeY)+1. Otherwise, availableN is set equal to TRUE. If all of the following conditions are true, then availableN is set equal to FALSE: ·checkPredModeY is equal to TRUE. · availableN is set equal to TRUE. CuPredMode[0][xNbY][yNbY] is not equal to CuPredMode[0][xTbY][yTbY]. IsAvailable[cIdx][x][y] used to store the sample availability information for each component cIdx at sample location (x,y) When cIdx=0, 0<=x<=pic_width_in_luma_samples, 0<=y<=pic_height_in_luma_samples, When cIdx=1 or 2, 0<=x<=pic_width_in_chroma_samples, 0<=y<=pic_height_in_chroma_samples. The inputs to this process are: Position (xCurr, yCurr) that specifies the top-left sample of the current block relative to the top-left sample of the current picture component The variables nCurrSw and nCurrSh specify the width and height of the current block, respectively. · Variable cIdx that specifies the color component of the current block A (nCurrSw) x (nCurrSh) array predSamples that specifies the predicted samples for the current block. resamples, a (nCurrSw) x (nCurrSh) array specifying the residual samples of the current block The output of this process is the reconstructed picture sample array recSamples. Depending on the value of the color component cIdx, the following allocation is made: If cIdx is equal to 0, recSamples is the reconstructed picture sample array S L Corresponds to. Otherwise, if cIdx is equal to 1, tuCbfChroma is set equal to tu_cbf_cb[xCurr][yCurr] and recSamples is the reconstructed chroma sample array S Cb Corresponds to. Otherwise (cIdx is equal to 2), tuCbfChroma is set equal to tu_cbf_cr[xCurr][yCurr] and recSamples is the reconstructed chroma sample array S Cr Corresponds to. Depending on the value of pic_lmcs_enabled_flag, the following applies: If pic_lmcs_enabled_flag is equal to 0, the (nCurrSw) × (nCurrSh) block of reconstructed samples recSamples at position (xCurr, yCurr) is derived as follows for i=0..nCurrSw-1, j=0..nCurrSh-1: recSamples[xCurr+i][yCurr+j] =Clip1(predSamples[i][j]+recSamples[i][j]) (1195) Otherwise (pic_lmcs_enabled_flag equals 1), the following applies: If cIdx is equal to 0, the following applies: The picture reconstruction with luma sample mapping process as specified in Section 8.7.5.2 is called with the luma position (xCurr, yCurr), the block width nCurrSw and height nCurrSh, the predicted luma sample array predSamples, and the residual luma sample array resSamples as input, and the output is the reconstructed luma sample array recSamples. Otherwise (cIdx is greater than 0), picture reconstruction with luma-dependent chroma residual scaling of chroma samples as specified in section 8.7.5.3 is called with the chroma position (xCurr, yCurr), the transform block width nCurrSw and height nCurrSh, the coded block flag of the current chroma transform block tuCbfChroma, the predicted chroma sample array predSamples, and the residual chroma sample array resSamples as input, and the output is the reconstructed chroma sample array recSamples. The following assignments are made for i=0..nCurrSw-1, j=0..nCurrSh-1. xVb=(xCurr+i)%((cIdx==0)?IbcBufWidthY:IbcBufWidthC) (1196) yVb=(yCurr+j)%((cIdx==0)?CtbSizeY:(CtbSizeY / subHeightC)) (1197) IbcVirBuf[cIdx][xVb][yVb]=recSamples[xCurr+i][yCurr+j] (1198) IsAvailable[cIdx][xCurr+i][yCurr+j]=TRUE (1199)

[0196] Another example of the specification of these sections can be presented as follows: Reference sample availability marking process The inputs to this process are: The sample position (xTbCmp, yTbCmp) that specifies the top-left sample of the current transform block relative to the top-left sample of the current picture. - Variable refIdx that specifies the intra-prediction reference line index -RefW variable that specifies the reference sample width Variable refH that specifies the reference sample height · Variable cIdx that specifies the color component of the current block The output of this process is the reference sample refUnfilt[x][y], and for intra-sample prediction x=-1-refIdx, y=-1-refIdx..refH-1, and x=-refIdx..refW-1, y=-1-refIdx. The refW+refH+1+(2*refIdx) neighboring samples refUnfilt[x][y], x=-1-refIdx, y=-1-refIdx..refH-1, and x=-refIdx..refW-1, y=-1-refIdx, which are samples constructed before in-loop filtering, are derived as follows: The neighborhood position (xNbCmp, yNbCmp) is specified by: (xNbCmp,yNbCmp) =(xTbCmp+x,yTbCmp+y) (310) The neighbor block availability derivation process specified in Section 6.4.4 is called with the current sample position (xCurr, yCurr) set equal to (xTbCmp, yTbCmp), the neighbor sample positions (xNbCmp, yNbCmp), checkPredModeY set equal to FALSE, and cIdx as input, and the output is assigned to availableN. Each sample of refUnfilt[x][y] is derived as follows: If availableN is equal to FALSE, the sample refUnfilt[x][y] is marked as "unavailable for intra prediction". Otherwise, the sample refUnfilt[x][y] is marked as "available for intra prediction" and the sample at position (xNbCmp, yNbCmp) is assigned to refUnfilt[x][y]. Neighboring block availability derivation process The inputs to this process are: The sample position (xCurr, yCurr) of the top-left sample of the current block relative to the top-left sample of the current picture. The sample position (xNbCmp, yNbCmp) covered by the neighboring blocks relative to the top-left luma sample of the current picture · checkPredModeY variable that specifies whether availability depends on the prediction mode · Variable cIdx that specifies the color component of the current block The output of this process is the availability of neighboring blocks covering position (xNbCmp, yNbCmp), denoted as availableN. The current luma position (xTbY, yTbY) and the neighboring luma position (xNbY, yNbY) are derived as follows: (xTbY,yTbY)=(cIdx==0)?(xCurr,yCurr): (xCurr*SubWidthC,yCurr*SubHeightC) (XXX) (xNbY,yNbY)=(cIdx==0)?(xNbCmp,yNbCmp): (XXX) (xNbCmp*SubWidthC,yNbCmp*SubHeightC) The neighboring block availability, availableN, is derived as follows: · If one or more of the following conditions are TRUE, then availableN is set equal to FALSE: xNbCmp is less than 0. yNbCmp is less than 0. xNbY is greater than or equal to pic_width_in_luma_samples. yNbY is greater than or equal to pic_hight_in_luma_samples. IsAvailable[cIdx][xNbCmp] "yNbCmp] is equal to FALSE. The neighboring block is in a different slice than the current block. The neighboring block is in a different tile than the current block. ·entropy_coding_sync_enabled_flag is equal to 1 and (xNbY>>CtbLog2SizeY) is greater than or equal to (xTbY>>CtbLog2SizeY)+1. Otherwise, availableN is set equal to TRUE. If all of the following conditions are true, then availableN is set equal to FALSE: ·checkPredModeY is equal to TRUE. · availableN is set equal to TRUE. CuPredMode[0][xNbY][yNbY] is not equal to CuPredMode[0][xTbY][yTbY]. The inputs to this process are: Position (xCurr, yCurr) that specifies the top-left sample of the current block relative to the top-left sample of the current picture component The variables nCurrSw and nCurrSh specify the width and height of the current block, respectively. · Variable cIdx that specifies the color component of the current block A (nCurrSw) x (nCurrSh) array predSamples that specifies the predicted samples for the current block. resamples, a (nCurrSw) x (nCurrSh) array specifying the residual samples of the current block The output of this process is the reconstructed picture sample array recSamples. Depending on the value of the color component cIdx, the following allocation is made: If cIdx is equal to 0, recSamples is the reconstructed picture sample array S L Corresponds to. Otherwise, if cIdx is equal to 1, tuCbfChroma is set equal to tu_cbf_cb[xCurr][yCurr] and recSamples is the reconstructed chroma sample array S Cb Corresponds to. Otherwise (cIdx is equal to 2), tuCbfChroma is set equal to tu_cbf_cr[xCurr][yCurr] and recSamples is the reconstructed chroma sample array S Cr Corresponds to. Depending on the value of pic_lmcs_enabled_flag, the following applies: If pic_lmcs_enabled_flag is equal to 0, the (nCurrSw) × (nCurrSh) block of reconstructed samples recSamples at position (xCurr, yCurr) is derived as follows for i=0..nCurrSw-1, j=0..nCurrSh-1: recSamples[xCurr+i][yCurr+j] =Clip1(predSamples[i][j]+recSamples[i][j]) (1195) Otherwise (pic_lmcs_enabled_flag equals 1), the following applies: If cIdx is equal to 0, the following applies: The picture reconstruction with luma sample mapping process as specified in Section 8.7.5.2 is called with the luma position (xCurr, yCurr), the block width nCurrSw and height nCurrSh, the predicted luma sample array predSamples, and the residual luma sample array resSamples as input, and the output is the reconstructed luma sample array recSamples. Otherwise (cIdx is greater than 0), picture reconstruction with luma-dependent chroma residual scaling operation of chroma samples as specified in section 8.7.5.3 is called with inputs chroma position (xCurr, yCurr), transform block width nCurrSw and height nCurrSh, coded block flag of current chroma transform block tuCbfChroma, predicted chroma sample array PredSamples, and residual chroma sample array resSamples, and output is reconstructed chroma sample array recSamples. The following assignments are made for i=0..nCurrSw-1, j=0..nCurrSh-1. xVb=(xCurr+i)%((cIdx==0)?IbcBufWidthY:IbcBufWidthC) (1196) yVb=(yCurr+j)%((cIdx==0)?CtbSizeY:(CtbSizeY / subHeightC)) (1197) IbcVirBuf[cIdx][xVb][yVb]=recSamples[xCurr+i][yCurr+j] (1198) IsAvailable[cIdx][(xCurr+i)*((cIdx==0)?1:SubWidthC)][(yCurr+j)*((cIdx==0)?1:SubHeightC)]=TRUE (1199)

[0197] 12 is a graphic diagram illustrating samples in a current block marked as available in an N×N unit according to an exemplary embodiment of the present disclosure. In an embodiment, referring to FIG. 12, after the current block is reconstructed, samples are marked as "available" in units having a size N*N, such as N=4 or N=2. Alternatively, N=4 for a Y component block and N=2 for a Cb / Cr component block. Any sample in an "available" unit is considered "available." This means that if a unit is marked as "available," any sample in the unit area can be marked as "available." That is, when the current unit is a Y unit, all Y samples in positions covered by the unit area are available; when the current unit is a Cb unit, all Cb samples in positions covered by the unit area are available; and when the current block is a Cr unit, all Cr samples in positions covered by the unit area are available.

[0198] In an embodiment, to check whether a reference sample is available, the process first obtains the position or index of the unit to which the sample belongs, and if the unit is determined to be "available", the reference sample is considered or marked as available.

[0199] 13 is a graphical diagram illustrating samples at the right and bottom boundaries of a current block that are marked as available according to an exemplary embodiment of the present disclosure. In an embodiment, referring to FIG. 13, after the current block is reconstructed, only the right and bottom boundary samples (which are used as reference samples for other blocks) are marked as "available."

[0200] FIG. 14 is a graphic diagram illustrating samples at the right and bottom boundaries of a current block that are marked as available in unit N according to an exemplary embodiment of the present disclosure. In an embodiment, referring to FIG. 14, after the current block is reconstructed, only the right and bottom boundary samples (which are used as reference samples for other blocks) are marked as "available" in a unit size with N, such as N=4 or N=2. Alternatively, N=4 for a Y component block, and N=2 for a Cb / Cr component block. Any sample in an "available" unit is considered "available." This means that when a unit is marked as "available," any sample in the unit area can be marked as "available." In other words, when the current unit is a Y unit, all Y samples at positions covered by the unit area are available; when the current unit is a Cb unit, all Cb samples at positions covered by the unit area are available; and when the current block is a Cr unit, all Cr samples at positions covered by the unit area are available.

[0201] To check whether a reference sample is available, in an embodiment, the process first obtains the position or index of the unit to which the sample belongs, and if the unit is determined to be "available", the reference sample is marked as available.

[0202] FIG. 15 is a graphic diagram illustrating samples at the right and bottom boundaries of a current block that are marked as available in an N×N unit according to an exemplary embodiment of the present disclosure. In an embodiment, referring to FIG. 15, after the current block is reconstructed, only the right and bottom boundary samples (which are used as reference samples for other blocks) are marked as "available" in a unit having a size of N*N, such as N=4 or N=2. Alternatively, N=4 for a Y component block, and N=2 for a Cb / Cr component block. Any sample in an "available" unit is considered "available." This means that once a unit is marked as "available," any sample in the unit area can be marked as "available." That is, when the current unit is a Y unit, all Y samples at positions covered by the unit area are available; when the current unit is a Cb unit, all Cb samples at positions covered by the unit area are available; and when the current block is a Cr unit, all Cr samples at positions covered by the unit area are available.

[0203] To check whether a reference sample is available, in an embodiment, a process or method first obtains the position or index of the unit to which the sample belongs, and if the process or method determines that the unit is "available", the process marks the reference sample as available.

[0204] It is noted here that, according to an embodiment of the present disclosure, the “available” information of samples or units is stored in memory for each component (three components in total): for the Y component, for the Cb component, and for the Cr component.

[0205] It should be noted here that even when a block is coded as an inter prediction mode, the "available" marking method can also be applied after the block is reconstructed.

[0206] Embodiment 2 In this embodiment, the availability check process is performed by examining a sample of its corresponding component.

[0207] The difference between embodiment 2 and embodiment 1 is that the Cb and Cr components are not distinguished in the availability check process. For a block, the reference sample availability check process is performed by checking the samples of its corresponding component. Here, a component can be a luma component or a chroma component. The Y component is called the luma component. Both the Cb and Cr components are called chroma components.

[0208] The difference between embodiment 2 and embodiment 1 occurs only in step 2, and in other steps, they are the same as those in embodiment 1. Step 2 of embodiment 2 will be described in detail below.

[0209] Step 2: Execute a reference sample availability check process, which includes checking the availability of a reference sample for the current block (1002 in FIG. 10).

[0210] In an embodiment, when the block is a luma block, the reference sample availability is derived by looking up the reference luma samples.

[0211] In an embodiment, when the block is a chroma block, the reference sample availability is derived by looking up the reference chroma samples.

[0212] 10 , after checking the availability of reference samples at 1002, the method includes determining whether there are any unavailable reference samples at 1003. If the method determines that there are unavailable reference samples (yes at 1003), the method proceeds to 1004; otherwise, the method proceeds to 1005.

[0213] It should be noted here that the marking method discussed in embodiment 1 can be directly extended to embodiment 2, with the only difference being that for chroma components, chroma samples in a block area can be marked as "available" only after the Cb component block and the Cr component block have been reconstructed.

[0214] For example, for a luma block, once the block is reconstructed, the samples within the block region can be marked as "available." For a chroma block, once both the Cb and Cr blocks are reconstructed, the samples within the block region can be marked as "available." This means that when the current block is a luma block, all luma samples in positions covered by the block region are marked as available. When the current block is a chroma block, once both the Cb and Cr blocks are reconstructed, all chroma samples in positions covered by the block region are marked as available.

[0215] Other marking methods in embodiment 1 can be similarly extended.

[0216] It should be noted here that in the embodiments of the present disclosure, the "availability" information of a sample or unit is stored in memory for each component (two components in total), such as for the luma component and for the chroma component. This means that the Cb and Cr components share the same "availability" or "is available" information.

[0217] It should be noted that even if a block is coded as an inter prediction mode, the "available" marking method can also be applied after the block is reconstructed.

[0218] The following is a description of the application of the encoding and decoding methods as shown in the above embodiments, and the systems that use them.

[0219] 17 is a block diagram showing a content delivery system 3100 that realizes a content distribution service. The content delivery system 3100 includes a capture device 3102, a terminal device 3106, and optionally a display 3126. The capture device 3102 communicates with the terminal device 3106 via a communication link 3104. The communication link may include the communication channel 13 described above. The communication link 3104 includes, but is not limited to, WIFI, Ethernet, cable, wireless (3G / 4G / 5G), USB, or any combination thereof.

[0220] The capture device 3102 may generate data and encode the data using the encoding method described in the above-described embodiments. Alternatively, the capture device 3102 may deliver the data to a streaming server (not shown), which then encodes the data and transmits the encoded data to the terminal device 3106. The capture device 3102 may include, but is not limited to, a camera, a smartphone or Pad, a computer or laptop, a video conferencing system, a PDA, an in-vehicle device, or any combination thereof. For example, the capture device 3102 may include the source device 102 described above. When the data includes video, the video encoder 20 included in the capture device 3102 may actually perform the video encoding process. When the data includes audio (i.e., voice), the audio encoder included in the capture device 3102 may actually perform the audio encoding process. In some practical scenarios, the capture device 3102 delivers the encoded video and audio data by multiplexing them together. In other practical scenarios, for example, in a video conferencing system, the encoded audio data and the encoded video data are not multiplexed. The capture device 3102 delivers the encoded audio data and the encoded video data separately to the terminal device 3106 .

[0221] In the content delivery system 3100, a terminal device 3106 receives and plays encoded data. The terminal device 3106 may be a device with data reception and recovery capabilities, such as a smartphone or Pad 3108, a computer or laptop 3110, a network video recorder (NVR) / digital video recorder (DVR) 3112, a TV 3114, a set-top box (STB) 3116, a video conferencing system 3118, a video surveillance system 3120, a personal digital assistant (PDA) 3122, an in-vehicle device 3124, or any combination thereof, capable of decoding the encoded data described above. For example, the terminal device 3106 may include the destination device 14 described above. When the encoded data includes video, the video decoder 30 included in the terminal device is prioritized to perform video decoding. When the encoded data includes audio, the audio decoder included in the terminal device is prioritized to perform audio decoding.

[0222] In terminal devices with a display, such as a smartphone or Pad 3108, a computer or laptop 3110, a network video recorder (NVR) / digital video recorder (DVR) 3112, a TV 3114, a personal digital assistant (PDA) 3122, or an in-vehicle device 3124, the terminal device can provide the decoded data to its own display. In terminal devices without a display, such as an STB 3116, a video conferencing system 3118, or a video surveillance system 3120, an external display 3126 is connected to receive and display the decoded data.

[0223] When each device in this system performs encoding or decoding, it can use a picture encoding device or a picture decoding device as shown in the above embodiments.

[0224] 18 is a diagram illustrating an example structure of the terminal device 3106. After the terminal device 3106 receives a stream from the capture device 3102, a protocol progression unit 3202 analyzes the transmission protocol of the stream. The protocol includes, but is not limited to, Real Time Streaming Protocol (RTSP), Hyper Text Transfer Protocol (HTTP), HTTP Live streaming protocol (HLS), MPEG-DASH, Real-time Transport protocol (RTP), Real Time Messaging Protocol (RTMP), or any kind of combination thereof, etc.

[0225] After the protocol progression unit 3202 processes the stream, a stream file is generated. The file is output to the demultiplexing unit 3204. The demultiplexing unit 3204 can separate the multiplexed data into encoded audio data and encoded video data. As mentioned above, in some practical scenarios, for example, in a video conferencing system, the encoded audio data and encoded video data are not multiplexed. In this situation, the encoded data is sent to the video decoder 3206 and the audio decoder 3208 without passing through the demultiplexing unit 3204.

[0226] The demultiplexing process generates a video elementary stream (ES), an audio ES, and any subtitles. The video decoder 3206, which includes the video decoder 30 as described in the above embodiment, decodes the video ES using the decoding method as shown in the above embodiment to generate video frames and supplies this data to the synchronization unit 3212. The audio decoder 3208 decodes the audio ES to generate audio frames and supplies this data to the synchronization unit 3212. Alternatively, the video frames may be stored in a buffer (not shown in FIG. 18) before being supplied to the synchronization unit 3212. Similarly, the audio frames may be stored in a buffer (not shown in FIG. 18) before being supplied to the synchronization unit 3212.

[0227] The synchronization unit 3212 synchronizes the video and audio frames and provides the video / audio to the video / audio display 3214. For example, the synchronization unit 3212 synchronizes the presentation of video and audio information, which may be encoded in the syntax with timestamps for the presentation of the encoded audio and visual data, and for the delivery of the data stream itself.

[0228] If subtitles are included in the stream, the subtitle decoder 3210 decodes the subtitles, synchronizes them with the video and audio frames, and provides the video / audio / subtitles to the video / audio / subtitle display 3216.

[0229] The present invention is not limited to the above-mentioned system, and either the picture encoding device or the picture decoding device in the above-mentioned embodiments can be incorporated into other systems, for example, vehicle systems.

[0230] According to the present disclosure, the described methods and processes may be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, the methods or processes may be performed by instructions or program code stored on a computer-readable medium and executed by a hardware processing unit.

[0231] The proposed method stores availability information for each sample in each component, which can provide more accurate availability information in the intra prediction process.

[0232] By way of example, and not limitation, a computer-readable storage medium may include RAM, ROM, EEPROM, CD-ROM, or other optical disk storage, magnetic disk storage, or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair wire, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, the coaxial cable, fiber optic cable, twisted pair wire, DSL, or wireless technologies such as infrared, radio, and microwave are included within the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transitory media, but instead cover non-transitory tangible storage media. As used herein, disk and disc include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray disc, where disks typically reproduce data magnetically and discs reproduce data optically with a laser. Combinations of the above should also be included within the scope of computer-readable media.

[0233] The instructions or program code may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Accordingly, the term "processor," as used herein, may refer to any of the foregoing structures, or any other structure suitable for implementing the techniques described herein. Furthermore, in some aspects, the functionality described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding or incorporated into a combined codec. Furthermore, the techniques may be implemented entirely in one or more circuits or logic elements.

[0234] The techniques of this disclosure may be implemented in a wide variety of devices or equipment, including a wireless handset, an integrated circuit (IC), or a set of ICs (e.g., a chipset). Various components, modules, or units have been described in this disclosure to highlight functional aspects of devices configured to perform the techniques of the disclosure, but do not necessarily require accomplishment by different hardware units. Rather, as described above, the various units may be combined in a codec hardware unit, in combination with appropriate software and / or firmware, or provided by a collection of interoperating hardware units including one or more processors as described above.

[0235] While several embodiments have been provided in this disclosure, it should be understood that the disclosed systems and methods may be embodied in many other specific forms without departing from the spirit or scope of the disclosure. The examples of the present invention should be considered illustrative and not restrictive, and are not intended to be limited to the details given herein. For example, various elements or components may be combined or integrated into another system, or certain features may be omitted or not implemented.

[0236] Furthermore, the techniques, systems, subsystems, and methods described and shown in various embodiments, individually or separately, may be combined or integrated with other systems, modules, techniques, or methods without departing from the scope of the present disclosure. Other items shown or discussed as connected or directly connected or in communication with each other may be indirectly connected or communicate through some interface, device, or intermediate component, whether electrical, mechanical, or otherwise. Other examples of changes, substitutions, and alterations will be ascertainable by those skilled in the art and may be made without departing from the spirit and scope of the present disclosure. [Explanation of symbols]

[0237] 12 Source Device 14 Destination Device 18 Video Sources 20 Video Encoder 22 Output Interface 28 Input Interface 30 Video Decoder 32 Display device

Claims

1. 1. An apparatus for intra prediction, comprising: a memory containing instructions; one or more processors in communication with the memory, the one or more processors executing the instructions to Obtain the intra prediction mode of the current block; Derive availability of neighboring blocks of the current block, and determine that the neighboring block is unavailable when all of the following conditions are true: (i) a variable checkPredModeY that specifies whether availability depends on a prediction mode is equal to true, and (ii) a value of the prediction mode of the neighboring block is not equal to a value of the prediction mode of the current block; deriving a prediction for the current block based on the intra-prediction mode of the current block and availability of the neighboring blocks.

2. The apparatus of claim 1 , wherein the availability of the neighboring blocks is derived for chroma components of the current block.

3. The apparatus of claim 2 , wherein the chroma components include a Cb component and a Cr component.

4. 4. The device of claim 1, wherein CuPredMode[0][xNbY][yNbY] specifies prediction mode values ​​of the neighboring blocks, and CuPredMode[0][xTbY][yTbY] specifies prediction mode values ​​of the current block.

5. The device according to any one of claims 1 to 4, wherein the values ​​of the prediction modes of the neighboring blocks are equal to inter or intra.

6. 1. A method for image intra prediction performed by an encoder or decoder, comprising: obtaining an intra prediction mode of a current block; deriving availability of neighboring blocks of the current block, wherein the neighboring blocks are unavailable when all of the following conditions are true: (i) a variable checkPredModeY that specifies whether availability depends on a prediction mode is equal to true, and (ii) a value of the prediction mode of the neighboring blocks is not equal to a value of the prediction mode of the current block; deriving a prediction of the current block based on the intra-prediction mode of the current block and availability of the neighboring blocks; A method comprising:

7. The method of claim 6 , wherein the availability of the neighboring blocks is derived for chroma components of the current block.

8. The method of claim 7 , wherein the chroma components include a Cb component and a Cr component.

9. 9. The method of claim 6, wherein CuPredMode[0][xNbY][yNbY] specifies the value of the prediction mode of the neighboring blocks and CuPredMode[0][xTbY][yTbY] specifies the value of the prediction mode of the current block.

10. The method according to any one of claims 6 to 9, wherein the value of the prediction mode of the neighboring blocks is equal to inter or intra.

11. A computer program comprising program code for carrying out the method according to any one of claims 6 to 10.

12. A decoding device, comprising: a receiver configured to receive an encoded video bitstream; a memory coupled to the receiver and storing instructions; a processor coupled to said memory, said processor executing said instructions to cause said decoding device to perform the method of any one of claims 6 to 10; A decoding device comprising:

13. 1. An encoding device, comprising: a memory storing instructions; a processor coupled to said memory, said processor executing said instructions to cause said encoding device to perform the method of any one of claims 6 to 10; An encoding device comprising:

Images