Intra prediction method and apparatus

Abstract

To provide an intra prediction method and apparatus for reconstruction of unavailable reference samples.SOLUTION: A method includes the steps of: obtaining an intra prediction mode of a current block; deriving availability of a reference sample of a component of the current block; substituting unavailable reference samples by using the available reference samples; deriving a prediction of the current block based on the intra prediction mode and the substituted reference samples; and reconstructing the current block based on the prediction. The component includes a Cb component or a Cr component. Alternatively, the component includes a Chroma component. Since the method derives availability of the reference sample in each component, it can provide available information more accurately.SELECTED DRAWING: Figure 10

Description

[Technical field]

[0001] [Related Applications] This application claims priority to U.S. Provisional Patent Application No. 62 / 770,736, filed November 21, 2018, entitled "Intra Prediction Method and Device," the disclosure of which is incorporated herein by reference in its entirety for all purposes.

[0002] [Technical field] FIELD OF THE DISCLOSURE Embodiments of the present invention relate generally to the field of video coding, and more particularly to the field of intra prediction methods and apparatus. [Background technology]

[0003] The amount of video data required to render even a relatively short video can be substantial. This can pose challenges when the data is streamed or otherwise communicated across communication networks with limited bandwidth capabilities. Thus, video data is typically compressed before being communicated across today's telecommunications networks. When the video is stored on a storage device, the size of the video can also be an issue since memory resources may be limited. Video compression devices often use software and / or hardware to encode video data at the source before transmission or storage, thereby reducing the amount of data required to represent a digital video image. The compressed data is then received at the destination by a video decompressor device that decodes the video data. With limited network resources and an ever-increasing demand for higher video quality, improved compression and decompression techniques that increase compression ratios with little to no sacrifice in image quality are desirable. Summary of the Invention

[0004] The embodiments of the present disclosure provide an intra prediction apparatus and method for encoding and decoding an image, and the embodiments of the present disclosure should not be construed as being limited to the examples detailed herein.

[0005] According to a first aspect of the present disclosure, a method includes obtaining an intra-prediction mode of a current block, deriving availability of reference samples of components of the current block, substituting unavailable reference samples with available reference samples, deriving a prediction of the current block based on the intra-prediction mode and the substituting reference samples, and reconstructing the current block based on the prediction. In an embodiment, the components include a Y component, a Cb component, or a Cr component. In another embodiment, the components include a luma component, or a chroma component.

[0006] The first aspect of the present disclosure derives the availability of the reference sample in each component, and thus can provide the availability information more accurately.

[0007] According to an implementation of the first aspect, all Cb samples in the reconstructed block are marked as available, or all Cr samples in the reconstructed block are marked as available.

[0008] An implementation of the first aspect stores the availability information for each sample in each component, and thus can provide the availability information more accurately in intra prediction processing.

[0009] According to a second aspect of the present disclosure, a decoder includes a processing circuit configured to perform the steps of the above-mentioned method.

[0010] According to a third aspect of the present disclosure, an encoder includes a processing circuit configured to perform the steps of the above-mentioned method.

[0011] According to a fourth aspect of the present disclosure, a computer program product comprises program code which, when executed by a processor, performs the method described above.

[0012] According to a fifth aspect of the present disclosure, a decoder for intra prediction includes one or more processing units and a non-transitory computer-readable storage medium connected to the one or more processing units, the non-transitory computer-readable storage medium storing program instructions executed by the one or more processing units to perform the above-mentioned method.

[0013] According to a sixth aspect of the present disclosure, an encoder for intra prediction includes one or more processing units and a non-transitory computer-readable storage medium connected to the one or more processing units, the non-transitory computer-readable storage medium storing program instructions executed by the one or more processing units to perform the above-mentioned method.

[0014] The embodiments of the present disclosure further provide a decoding device and an encoding device for performing the above-mentioned method.

[0015] For purposes of clarity, any one of the above-described embodiments may be combined with any one or more of the other above-described embodiments to create new embodiments that are within the scope of the present disclosure.

[0016] These and other features will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings and claims. [Brief description of the drawings]

[0017] For a more complete understanding of the present disclosure, reference is now made to the following brief description taken in conjunction with the accompanying drawings and detailed description, where like reference numerals represent like parts, and in which:

[0018] [Figure 1A] FIG. 1 is a block diagram illustrating an example encoding system in which embodiments of the present disclosure may be implemented.

[0019] [Figure 1B] FIG. 2 is a block diagram illustrating another exemplary encoding system in which embodiments of the present disclosure may be implemented.

[0020] [Diagram 2] 1 is a block diagram illustrating an example video encoder in which embodiments of the present disclosure may be implemented.

[0021] [Diagram 3] 1 is a block diagram illustrating an example of a video decoder capable of implementing embodiments of the present disclosure.

[0022] [Figure 4] FIG. 2 is a schematic diagram of a network device according to an exemplary embodiment of the present disclosure.

[0023] [Diagram 5] 1B is a simplified block diagram of a device that may be used as one or both of the source and destination devices of FIG. 1A in accordance with an exemplary embodiment of the present disclosure.

[0024] [Figure 6] 1 is a description of an intra-prediction algorithm for H.265 / HEVC that may be utilized in the description of this disclosure.

[0025] [Figure 7] FIG. 1 is a graphical diagram illustrating a reference sample according to an exemplary embodiment of the present disclosure.

[0026] [Figure 8] FIG. 1 is a graphical diagram illustrating available and unavailable reference samples according to an exemplary embodiment of the present disclosure.

[0027] [Figure 9] 4 is a simplified flowchart for deriving a reconstructed signal according to an exemplary embodiment of the present disclosure.

[0028] [Figure 10] 4 is a simplified flow chart for deriving a prediction signal according to an exemplary embodiment of the present disclosure.

[0029] [Figure 11] FIG. 13 is a graphical diagram illustrating samples in a current block that are marked as available according to an exemplary embodiment of the present disclosure.

[0030] [Figure 12] FIG. 13 is a graphical diagram illustrating samples in a current block marked as available in a unit N×N according to an exemplary embodiment of the present disclosure.

[0031] [Figure 13] FIG. 13 is a graphical diagram illustrating samples at the right and bottom boundaries of a current block that are marked as available according to an exemplary embodiment of the present disclosure.

[0032] [Figure 14] FIG. 13 is a graphical diagram illustrating samples at the right and bottom boundaries of a current block marked as available in unit N according to an exemplary embodiment of the present disclosure.

[0033] [Figure 15] FIG. 1 is a graphical diagram illustrating samples at the right and bottom boundaries of a current block marked as available in a unit N×N according to an exemplary embodiment of the present disclosure.

[0034] [Figure 16] FIG. 1 is a graphical diagram illustrating a method for marking samples of chroma components according to an exemplary embodiment of the present disclosure.

[0035] [Figure 17] 31 is a block diagram showing an exemplary structure of a content delivery system 3100 for implementing a content distribution service.

[0036] [Figure 18] FIG. 2 is a block diagram showing the structure of an example of a terminal device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0037] It should be understood at the outset that, although illustrative implementations of one or more embodiments are provided below, the disclosed systems and / or methods may be implemented using a variety of technologies, whether currently known or existing, and that the present disclosure should in no way be limited to the illustrative implementations, drawings, and technologies described below, including the exemplary designs and implementations shown and described herein, but may be modified within the scope of the appended claims, along with their full scope of equivalents.

[0038] FIG 1A is a schematic block diagram illustrating an example encoding system 10 that may utilize bidirectional prediction techniques. As shown in FIG 1A, encoding system 10 includes a source device 12 that provides encoded video data to be decoded at a later time by a destination device 14. In particular, source device 12 may provide the video data to destination device 14 via a computer-readable medium 16. Source device 12 and destination device 14 may include any of a wide range of devices, including desktop computers, notebook (i.e., laptop) computers, tablet computers, set-top boxes, so-called "smart" phones, telephone handsets such as so-called "smart" pads, televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, and the like. In some cases, source device 12 and destination device 14 may be equipped for wireless communication.

[0039] The destination device 14 may receive the encoded video data to be decoded via a computer-readable medium 16. The computer-readable medium 16 may include any type of medium or device capable of moving encoded video data from the source device 12 to the destination device 14. In one example, the computer-readable medium 16 may include a communication medium that allows the source device 12 to transmit the encoded video data directly to the destination device 14 in real time. The encoded video data may be modulated according to a communication standard, such as a wireless communication protocol, and transmitted to the destination device 14. The communication medium may include any wireless or wired communication medium, such as the radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include routers, switches, base stations, or any other equipment that may be useful for achieving communication from the source device 12 to the destination device 14.

[0040] In some examples, the encoded data may be output from output interface 22 to a storage device. Similarly, the encoded data may be accessed by the input interface from a storage device. The storage device may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, a Blu-ray disc, a digital video disk (DVD), a Compact Disc Read-Only Memory (CD-ROM), flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In further examples, the storage device may correspond to a file server or another intermediate storage device that may store the encoded video generated by source device 12. Destination device 14 may access the stored video data from the storage device via streaming or download. The file server may be any type of server capable of storing and transmitting the encoded video data to destination device 14. Exemplary file servers include a web server (e.g., for a website), a file transfer protocol (FTP) server, a network attached storage (NAS) device, or a local disk drive. Destination device 14 may access the encoded video data through any standard data connection, including an Internet connection. This may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a digital subscriber line (DSL), a cable modem, etc.), or a combination of both suitable for accessing encoded video data stored on a file server. The transmission of the encoded video data from the storage device may be a streaming transmission, a download transmission, or a combination thereof.

[0041] The techniques of this disclosure are not necessarily limited to wireless applications or settings. The techniques may be applied to video encoding in support of any of a variety of multimedia applications, such as wireless television broadcast, cable television transmission, satellite television transmission, Internet streaming video transmission such as dynamic adaptive streaming over HTTP (DASH), digital video encoded on a data storage medium, decoding of digital video stored on a data storage medium, or other applications. In some examples, encoding system 10 may be configured to support one-way or two-way video transmission to support applications such as video streaming, video playback, video broadcast, and / or video telephony.

[0042] In the example of FIG. 1A, source device 12 includes a video source 18, a video encoder 20, and an output interface 22. Destination device 14 includes an input interface 28, a video decoder 30, and a display device 32. According to this disclosure, the video encoder 200 of source device 12 and / or the video decoder 300 of destination device 14 may be configured to apply techniques for bidirectional prediction. In other examples, the source device and destination device may include other components or configurations. For example, source device 12 may receive video data from an external video source, such as an external camera. Similarly, destination device 14 may interface with an external display device rather than including an integrated display device.

[0043] The illustrated encoding system 10 of FIG. 1A is merely an example. The techniques for bidirectional prediction may be performed by any digital video encoding and / or decoding device. Although the techniques of this disclosure are generally performed by a video encoding device, the techniques may also be performed by a video encoder / decoder, typically referred to as a "CODEC." Additionally, the techniques of this disclosure may be performed by a video pre-processor. The video encoder and / or decoder may be a graphics processing unit (GPU) or similar device.

[0044] Source device 12 and destination device 14 are merely examples of such encoding devices, with source device 12 generating encoded video data for transmission to destination device 14. In some examples, source device 12 and destination device 14 may operate substantially symmetrically, such that each of source and destination devices 12, 14 includes video encoding and decoding components. Thus, encoding system 10 may support one-way or two-way video transmission between video devices 12, 14, e.g., for video streaming, video playback, video broadcast, or video telephony.

[0045] Video source 18 of source device 12 may include a video capture device such as a video camera, a video archive containing previously captured video, and / or a video feed interface for receiving video from a video content provider. As a further alternative, video source 18 may generate computer graphic based data as the source video, or a combination of live video, archived video, and computer generated video.

[0046] In some examples, when video source 18 is a video camera, source device 12 and destination device 14 may form a so-called camera phone or video phone. As noted above, however, the techniques described in this disclosure may be applicable to video encoding generally, and may be applied to wireless and / or wired applications. In each case, captured, pre-captured, or computer-generated video may be encoded by video encoder 20. The encoded video information may then be output by output interface 22 to computer-readable medium 16.

[0047] Computer-readable medium 16 may include a transmission medium, such as a wireless broadcast or wired network transmission, or a storage medium (i.e., a non-transitory storage medium), such as a hard disk, a flash drive, a compact disc, a digital video disc, a Blu-ray disc, or other computer-readable medium. In some examples, a network server (not shown) may receive the encoded video data from source device 12 and provide the encoded video data to destination device 14, for example, via a network transmission. Similarly, a computing device of a media production facility, such as a disc stamping facility, may receive the encoded video data from source device 12 and produce discs including the encoded video data. Thus, computer-readable medium 16 may be understood to include one or more computer-readable media of various types in various examples.

[0048] An input interface 28 of destination device 14 receives information from computer-readable medium 16. The information on computer-readable medium 16 may include syntax information defined by video encoder 20, including syntax elements that describe characteristics and / or processing of blocks and other coding units, such as a group of pictures (GOP), that are also used by video decoder 30. Display device 32 displays the decoded video data to a user and may include any of a variety of display devices, such as a cathode ray tube (CRT), a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device.

[0049] Video encoder 200 and video decoder 300 may operate according to a video coding standard, such as the High Efficiency Video Coding (HEVC) standard currently being developed and may conform to the HEVC Test Model (HM). Alternatively, video encoder 200 and video decoder 300 may operate according to an industry standard, such as the International Telecommunications Union Telecommunications Standardization Sector (ITU-T) H.264 standard, H.265 / HEVC, or an extension of such a standard, proprietary or alternatively referred to as Motion Picture Expert Group (MPEG)-4, Part 10, Advanced Video Coding (AVC). The techniques of this disclosure, however, are not limited to any particular coding standard. Other examples of video coding standards include MPEG-2 and ITU-T H.263. 1A, in some aspects, the video encoder 200 and the video decoder 300 may be integrated with an audio encoder and decoder, respectively, and may include a multiplexer-demultiplexer (MUX-DEMUX) unit, or other hardware and software, appropriate to handle the encoding of both audio and video in a common data stream or to separate the data streams. Where appropriate, the MUX-DEMUX unit may conform to the ITU H.223 multiplexer protocol or other protocols, such as the user datagram protocol (UDP).

[0050] The video encoder 200 and the video decoder 300 may each be implemented as any of a variety of suitable encoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When the techniques are implemented partially in software, a device may store instructions for the software in a suitable non-transitory computer-readable medium and execute the instructions in hardware using one or more processors to perform the techniques of this disclosure. Each of the video encoder 200 and the video decoder 300 may be included in one or more encoders or decoders, either of which may be integrated within a respective device as part of a combined encoder / decoder (CODEC). A device including the video encoder 200 and / or the video decoder 300 may include an integrated circuit, a microprocessor, and / or a wireless communication device, such as a mobile phone.

[0051] 1B is a schematic block diagram illustrating an example video encoding system 40 including the encoder 200 of FIG. 2 and / or the decoder 300 of FIG. 3, according to an example embodiment. The system 40 can implement techniques of this disclosure, such as merge estimation in inter prediction. In the illustrated implementation, the video encoding system 40 can include an image device 41, a video encoder 20, a video decoder 300 (and / or a video coder implemented by logic circuitry 47 of a processing unit 46), an antenna 42, one or more processors 43, one or more memory stores 44, and / or a display device 45.

[0052] As shown, imaging device 41, antenna 42, processing unit 46, logic circuitry 47, video encoder 20, video decoder 30, processor 43, memory store 44, and / or display device 45 may be in communication with one another. As discussed, although shown with both video encoder 20 and video decoder 30, video encoding system 40 may include only video encoder 20 or only video decoder 30 in various practical scenarios.

[0053] As shown, in some examples, the video encoding system 40 may include an antenna 42. The antenna 42 may be configured to transmit or receive, for example, an encoded bitstream of video data. Additionally, in some examples, the video encoding system 40 may include a display device 45. The display device 45 may be configured to present the video data. As shown, in some examples, the logic circuitry 47 may be implemented by a processing unit 46. The processing unit 46 may include application-specific integrated circuit (ASIC) logic, a graphics processor, a general-purpose processor, etc. The video encoding system 40 may also include an optional processor 43, which may also include application-specific integrated circuit (ASIC) logic, a graphics processor, a general-purpose processor, etc. In some examples, the logic circuitry 54 may be implemented by hardware, dedicated video encoding hardware, etc., and the processor 43 may implement general-purpose software, an operating system, etc. Further, memory store 44 may be any type of memory, such as volatile memory (e.g., Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), etc.) or non-volatile memory (e.g., flash memory, etc.). In a non-limiting example, memory store 44 may be implemented by a cache memory. In some examples, logic circuitry 47 may access memory store 44 (e.g., for implementation of an image buffer). In other examples, logic circuitry 47 and / or processing unit 46 may include a memory store (e.g., a cache, etc.) for implementation of an image buffer, etc.

[0054] In some examples, video encoder 200 implemented by logic circuitry may include an image buffer (e.g., by either processing unit 46 or memory store 44) and a graphics processing unit (e.g., by processing unit 46). The graphics processing unit may be communicatively coupled to the image buffer. The graphics processing unit may include video encoder 200 implemented by logic circuitry 47 to implement various modules such as those discussed with respect to FIG. 2 and / or any other encoder system or subsystems described herein. The logic circuitry may be configured to perform various operations as discussed herein.

[0055] Video decoder 30 may be implemented in a manner similar to that implemented by logic circuitry 47 to implement various modules as discussed with respect to decoder 300 of FIG. 3 and / or any other decoder system or subsystem described herein. In some examples, video decoder 300 may be implemented by logic circuitry and may include an image buffer (e.g., by either processing unit 46 or memory store 44) and a graphics processing unit (e.g., by processing unit 46). The graphics processing unit may be communicatively connected to the image buffer. The graphics processing unit may include video decoder 300 as implemented by logic circuitry 47 to implement various modules as discussed with respect to decoder 300 of FIG. 3 and / or any other decoder system or subsystem described herein.

[0056] In some examples, antenna 42 of video encoding system 40 may be configured to receive an encoded bitstream of video data. As discussed, the encoded bitstream may include data related to the encoding of video frames as discussed herein, such as data related to coding partitions (e.g., transform coefficients or quantized transform coefficients, optional indicators (as discussed), and / or data defining coding partitions), indicators, index values, mode selection data, etc. Video encoding system 40 may also include a video decoder 300 coupled to antenna 42 and configured to decode the encoded bitstream. Display device 45 is configured to present the video frames.

[0057] 2 is a block diagram illustrating an example of a video encoder 200 that may implement the techniques of the present application. The video encoder 200 may perform intra- and inter-coding of video blocks in a video slice. Intra-coding relies on spatial prediction to reduce or remove spatial redundancy in video within a given video frame or picture. Inter-coding relies on temporal prediction to reduce or remove temporal redundancy in video within adjacent frames or pictures of a video sequence. Intra-mode (I-mode) may represent any of several spatial-based coding modes. Inter-modes, such as unidirectional prediction (P-mode) or bi-prediction (B-mode), may represent any of several temporal-based coding modes.

[0058] FIG. 2 shows a schematic / conceptual block diagram of an example video encoder 200 configured to implement the techniques of this disclosure. In the example of FIG. 2, the video encoder 200 includes a residual calculation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210 and an inverse transform processing unit 212, a reconstruction unit 214, a buffer 216, a loop filter unit 220, a decoded picture buffer (DPB) 230, a prediction processing unit 260, and an entropy coding unit 270. The prediction processing unit 260 may include an inter estimation 242, an inter prediction unit 244, an intra estimation 252, an intra prediction unit 254, and a mode selection unit 262. The inter prediction unit 244 may further include a motion estimation unit (not shown). The video encoder 200 as shown in FIG. 2 may be referred to as a hybrid video encoder or a video encoder according to a hybrid video codec.

[0059] For example, the residual calculation unit 204, the transform processing unit 206, the quantization unit 208, the prediction processing unit 260, and the entropy coding unit 270 form a forward signal path of the encoder 200. Meanwhile, for example, the inverse quantization unit 210, the inverse transform processing unit 212, the reconstruction unit 214, the buffer 216, the loop filter 220, the decoded picture buffer (DPB) 230, and the prediction processing unit 260 form a backward signal path of the encoder, which corresponds to the signal path of the decoder (see decoder 300 in FIG. 3).

[0060] The encoder 200 is arranged to receive, for example via an input 202, a picture 201 or a block 203 of a picture 201, for example a picture of a video or a sequence of pictures forming a video sequence. The picture block 203 may also be referred to as a current picture block or a coded picture block (particularly in video coding, to distinguish a current picture from other pictures, for example previously coded and / or decoded pictures of the same video sequence, i.e. a video sequence which also contains the current picture), and the picture 201 as a current picture or a coded picture.

[0061] Partitioning

[0062] An embodiment of the encoder 200 may include a partition unit (not shown in FIG. 2) configured to partition picture 201 into multiple blocks, such as block 203, typically into multiple non-overlapping blocks. The partition unit may be configured to use the same block size for all pictures of a video sequence and a corresponding grid that defines the block sizes, or to vary the block size between pictures or subsets or groups of pictures, and to partition each picture into corresponding blocks.

[0063] In HEVC and other video coding specifications, a set of coding tree units (CTUs) may be generated to generate a coded representation of a picture. Each CTU may include a coding tree block of luma samples, two corresponding coding tree blocks of chroma samples, and syntax structures used to code the samples of the coding tree block. For a monochromatic picture or a picture with three separate color planes, a CTU may include a single coding tree block and syntax structures used to code the samples of the coding tree block. A coding tree block may be an N×N block of samples. A CTU may be referred to as a “tree block” or a “largest coding unit (LCU).” A CTU in HEVC may be broadly similar to a macroblock in other standards such as H.264 / AVC. However, a CTU is not necessarily limited to a particular size and may include one or more coding units (CUs). A slice may include an integer number of CTUs ordered consecutively in raster scan order.

[0064] In HEVC, CTUs are divided into CUs using a quadtree structure, denoted as a coding tree, to accommodate different local characteristics. The decision to code a picture region using inter-picture (temporal) or intra-picture (spatial) prediction is made at the CU level. A CU may include a coding block of luma samples and two corresponding coding blocks of chroma samples for a picture having a luma sample array, a Cb sample array, and a Cr sample array, and syntax structures used to code the samples of the coding block. For a monochromatic picture or a picture with three separate color planes, a CU may include a single coding block and syntax structures used to code the samples of the coding block. A coding block is an N×N block of samples. In some examples, the CUs may be CTUs of the same size. Each CU is coded by one coding mode, which may be, for example, an intra coding mode or an inter coding mode. Other coding modes are possible. The encoder 200 receives video data. The encoder 200 may code each CTU in a slice of a picture of the video data. As part of encoding a CTU, prediction processing unit 260 or another processing unit of encoder 200 (including but not limited to the units of encoder 200 shown in FIG. 2) may perform partitioning to divide the CTB of the CTU into progressively smaller blocks 203. The smaller blocks may be coding blocks of CUs.

[0065] The syntax data in the bitstream may also define the size of the CTU. A slice includes a number of consecutive CTUs in coding order. A video frame or image or picture may be partitioned into one or more slices. As described above, each tree block may be divided into coding units (CUs) according to a quadtree. Typically, a quadtree data structure includes one node per CU, with a root node corresponding to a tree block (e.g., a CTU). If a CU is divided into four sub-CUs, the node corresponding to the CU includes four child nodes, each of which corresponds to one of the sub-CUs. The nodes in the quadtree structure include leaf nodes and non-leaf nodes. A leaf node does not have a child node in the tree structure (i.e., a leaf node is not further divided). The non-leaf nodes include a root node of the tree structure. For each non-root node of the nodes, each non-root node corresponds to a sub-CU of a CU corresponding to a parent node in the tree structure of the non-root node. Each non-leaf node has one or more child nodes in the tree structure.

[0066] Each node of the quadtree data structure may provide syntax data for the corresponding CU. For example, a node in the quadtree may include a split flag that indicates whether the CU corresponding to the node is split into sub-CUs. The syntax elements of a CU may be defined recursively and may depend on whether the CU is split into sub-CUs. If a CU is not further split, it is called a leaf CU. If a block of CUs is further split, it may generally be called a non-leaf CU. Each level of partition is a quadtree split into four sub-CUs. Black CUs are examples of leaf nodes (i.e., blocks that are not further split).

[0067] A CU has a similar purpose to a macroblock in the H.264 standard, except that a CU has no size difference. For example, a treeblock may be split into four child nodes (also called sub-CUs), and each child node, on the other hand, may be split into another four child nodes. The final unsplit child node is called a leaf node of the quadtree, and includes a coding node, also called a leaf CU. Syntax data associated with the coded bitstream may define the maximum number of times a treeblock may be split, called the maximum CU depth, and may also define the minimum size of a coding node. Thus, the bitstream may also define the smallest coding unit (SCU). The term "block" is used to represent either a CU, PU, ​​or TU in the context of HEVC, or a similar data structure in the context of other standards (e.g., a macroblock and its sub-blocks in H.264 / AVC).

[0068] In HEVC, each CU can be further divided into 1, 2, or 4 PUs according to a PU partition type. Within one PU, the same prediction process is applied, and related information is sent to the decoder for each PU. After obtaining the residual block by applying the prediction process based on the PU partition type, the CU can be partitioned into transform units (TUs) according to another quadtree structure similar to the coding tree of the CU. One of the main features of the HEVC structure is that it has multiple partition concepts including CU, PU, ​​and TU. The PU may be partitioned to be in a non-square shape. Syntax data associated with a CU may, for example, describe the partition of the CU into one or more PUs. The TU can be in a square or non-square (e.g., rectangular) shape, and syntax data associated with a CU may, for example, describe the partition of the CU into one or more TUs according to a quadtree. The partition mode may differ depending on whether the CU is coded in skip or direct mode, in intra prediction mode, or in inter prediction mode.

[0069] Versatile Video Coding (VVC) removes the separation of PU and TU concepts and supports more flexibility in CU partition shape. The size of a CU corresponds to the size of a coding node and may be square or non-square (e.g., rectangular) in shape. The size of a CU may range from 4×4 pixels (or 8×8 pixels) to the size of a tree block with a maximum of 128×128 pixels or more (e.g., 256×256 pixels).

[0070] After the encoder 200 generates a predictive block (e.g., luma, Cb, and Cr predictive blocks) for the CU, the encoder 200 may generate a residual block for the CU. For example, the encoder 100 may generate a luma residual block for the CU. Each sample in the luma residual block of the CU indicates a difference between a luma sample in the predictive luma block of the CU and a corresponding sample in the original luma coding block of the CU. Furthermore, the encoder 200 may generate a Cb residual block for the CU. Each sample in the Cb residual block of the CU may indicate a difference between a Cb sample in the predictive Cb block of the CU and a corresponding sample in the original Cb coding block of the CU. The encoder 200 may generate a Cr residual block for the CU. Each sample in the Cr residual block of the CU may indicate a difference between a Cr sample in the predictive Cr block of the CU and a corresponding sample in the original Cr coding block of the CU.

[0071] In some examples, the encoder 200 skips applying a transform to a transform block. In such examples, the encoder 200 may treat the residual sample values ​​in the same manner as the transform coefficients. Thus, in examples in which the encoder 200 skips applying a transform, the following discussion of transform coefficients and coefficient blocks may be applicable to the transform blocks of residual samples.

[0072] After generating a coefficient block (e.g., a luma coefficient block, a Cb coefficient block, or a Cr coefficient block), the encoder 200 may quantize the coefficient block to possibly reduce the amount of data used to represent the coefficient block and potentially provide further compression. Quantization typically refers to a process in which a range of values ​​is compressed into a single value. After the encoder 200 quantizes the coefficient block, the encoder 200 may entropy code syntax elements that indicate the quantized transform coefficients. For example, the encoder 200 may perform Context-Adaptive Binary Arithmetic Coding (CABAC) or other entropy coding techniques on the syntax elements that indicate the quantized transform coefficients.

[0073] The encoder 200 may output a bitstream of coded picture data 271 that includes a sequence of bits that form a representation of a coded picture and associated data. The bitstream thus includes a coded representation of the video data.

[0074] J. An et al., “Block partitioning structure for next generation video coding”, International Telecommunication Union, COM16-C966, September 2015 (hereafter, “VCEG proposal COM16-C966”) proposed a quad-tree-binary-tree (QTBT) partitioning technique for future video coding standards after HEVC. Simulations show that the proposed QTBT structure is more efficient than the quad-tree structure used in HEVC. In HEVC, inter prediction of small blocks is constrained to reduce memory access for motion compensation, so bi-prediction is not supported for 4×8 and 8×4 blocks, and inter prediction is not supported for 4×4 blocks. In JEM's QTBT, these constraints are removed.

[0075] In QTBT, a CU can have either a square or a rectangular shape. For example, a coding tree unit (CTU) is first partitioned by a quadtree structure. The leaf nodes of the quadtree can be further partitioned by a binary tree structure. There are two types of binary tree partitioning: symmetric horizontal partitioning and symmetric vertical partitioning. In each case, the node is partitioned by splitting the node in the middle, either horizontally or vertically. The leaf nodes of the binary tree are called coding units (CUs), and segmentation is used for prediction and transform processing without further partitioning. This means that CUs, PUs, and TUs have the same block size in the QTBT coding block structure. A CU is sometimes composed of coding blocks (CBs) of different color components, e.g., in the case of P and B slices with 4:2:0 chroma format, a CU contains one luma CB and two chroma CBs, and sometimes composed of CBs of a single component, e.g., in the case of an I slice, a CU contains only one luma CB or only two chroma CBs.

[0076] The following parameters are defined for the QTBT partition method:

[0077] -CTU size: Root node size of the quadtree, the same concept as HEVC.

[0078] -MinQTSize: The minimum allowed quadtree leaf node size.

[0079] -MaxBTSize: The maximum allowed binary tree root node size.

[0080] -MaxBTDepth: The maximum allowed binary tree depth.

[0081] -MinBTSize: The minimum allowed binary tree leaf node size.

[0082] In one example of a QTBT partition structure, the CTU size is set as 128x128 luma samples with two corresponding 64x64 blocks of chroma samples, MinQTSize is set as 16x16, MaxBTSize is set as 64x64, MinBTSize (for both width and height) is set as 4x4, and MaxBTDepth is set as 4. The quadtree partition is first applied to the CTU to generate quadtree leaf nodes. The quadtree leaf nodes may have sizes from 16x16 (i.e., MinQTSize) to 128x128 (i.e., CTU size). When a quadtree leaf node has a size equal to MinQTSize, no further quadtrees are considered. If a leaf quadtree node is 128x128, it is not further divided by a bi-tree because the size exceeds MaxBTSize (i.e., 64x64). In other cases, the leaf quadtree node may be further partitioned by the binary tree. Thus, the quadtree leaf node is also the root node of the binary tree, which has a binary tree depth like 0. When the binary tree depth reaches MaxBTDepth (i.e. 4), no further splits are considered. When the binary tree node has a width equal to MinBTSize (i.e. 4), no further horizontal splits are considered. Similarly, when the binary tree node has a height equal to MinBTSize, no further vertical splits are considered. The leaf node of the binary tree is further processed by prediction and transformation processes without any further partitions. In JEM, the maximum CTU size is 256x256 luma samples. The leaf node (CU) of the binary tree may be further processed (e.g., by performing prediction and transformation processes) without any further partitions.

[0083] Furthermore, the QTBT scheme supports the ability for luma and chroma to have separate QTBT structures. Currently, in P and B slices, the luma and chroma CTBs in one CTU may share the same QTBT structure. However, in an I slice, the luma CTB may be partitioned into CUs with a QTBT structure, and the chroma CTB may be partitioned into chroma CUs with another QTBT structure. This means that a CU in an I slice is composed of a coded block of a luma component or a coded block of two chroma components, and a CU in a P or B slice is composed of coded blocks of all three color components.

[0084] The encoder 200 applies a rate-distortion optimization (RDO) process to the QTBT structure to determine the block partitions.

[0085] Furthermore, a block partition structure named multi-type-tree (MTT) has been proposed in US Patent Application Publication No. 20170208336 to replace the CU structures based on QT, BT, and / or QTBT. The MTT partition structure is still a recursive tree structure. In MTT, multiple different partition structures (e.g., three or more) are used. For example, according to the MTT technique, three or more different partition structures may be used for each non-leaf node of the tree structure at each depth of the tree structure. The depth of a node in the tree structure may represent the length of the path (e.g., the number of splits) from the node to the root of the tree structure. The partition structure may generally represent how many different blocks a block may be split into. The partition structure may be a quad-tree partition structure that may split a block into four blocks, a binary-tree partition structure that may split a block into two blocks, or a ternary-tree partition structure that may split a block into three blocks, and even a ternary-tree partition structure that may not split a block in the middle. The partition structure may have a number of different partition types, which may further define how the blocks are divided, including symmetric or asymmetric partitions, equal or uneven partitions, and / or horizontal or vertical partitions.

[0086] In MTT, at each depth of the tree structure, the encoder 200 may be configured to further divide the subtree using a particular partition type from among one of more than three partition structures. For example, the encoder 100 may be configured to determine a particular partition type from among QT, BT, triple-tree (TT), and other partition structures. In one example, the QT partition structure may include a square quadtree or rectangular quadtree partition type. The encoder 200 may partition a square block using a square quadtree partition by dividing the block into four equally sized square blocks along the center in both the horizontal and vertical directions. Similarly, the encoder 200 may partition a rectangular (e.g., non-square) block using a rectangular quadtree partition by dividing the rectangular block into four equally sized rectangular blocks along the center in both the horizontal and vertical directions.

[0087] The BT partition structure may include at least one of a horizontal symmetric binary tree, a vertical symmetric binary tree, a horizontal asymmetric binary tree, or a vertical asymmetric binary tree partition type. In the horizontal symmetric binary tree partition type, the encoder 200 may be configured to divide the block into two symmetric blocks of the same size horizontally along the center of the block. In the vertical symmetric binary tree partition type, the encoder 200 may be configured to divide the block into two symmetric blocks of the same size vertically along the center of the block. In the horizontal asymmetric binary tree partition type, the encoder 200 may be configured to divide the block into two blocks of different sizes horizontally. For example, one block may be ¼ the size of the parent block and the other block may be ¾ the size of the parent block, similar to the PART_2N×nU or PART_2N×nD partition types. In the vertical asymmetric binary tree partition type, the encoder 200 may be configured to divide the block into two blocks of different sizes vertically. For example, one block may be 1 / 4 the size of the parent block, and another block may be 3 / 4 the size of the parent block, similar to PART_nL×2N or PART_nR×2N partition types. In another example, an asymmetric binary tree partition type may divide a parent block into sub-parts of different sizes. For example, one sub-block may be 3 / 8 the size of the parent block, and another sub-block may be 5 / 8 the size of the parent block. Of course, such partition types may be either vertical or horizontal.

[0088] The TT partition structure differs from that of the QT or BT structures in that the TT partition structure does not split the blocks in the middle. The central regions of the blocks remain together in the same subblock. Unlike the QT, which results in four blocks, or the binary tree, which results in two blocks, the division by the TT partition structure results in three blocks. Exemplary partition types by the TT partition structure include symmetric partition types (both horizontally and vertically) and asymmetric partition types (both horizontally and vertically). Furthermore, the symmetric partition types by the TT partition structure may be uneven / non-uniform or even / uniform. The asymmetric partition type by the TT partition structure is uneven / non-uniform. In one example, the TT partition structure may include at least one of the following partition types: horizontal even / uniform symmetric ternary tree, vertical even / uniform symmetric ternary tree, horizontal uneven / non-uniform symmetric ternary tree, vertical uneven / non-uniform symmetric ternary tree, horizontal uneven / non-uniform asymmetric ternary tree, or vertical uneven / non-uniform asymmetric ternary tree partition type.

[0089] In general, an uneven / non-homomorphic symmetric ternary tree partition type is a partition type that is symmetric about the center line of a block, but at least one of the three resulting blocks is not the same size as the other two. One suitable example is when the edge blocks are a quarter of the size of the block and the center block is half the size of the block. An even / homomorphic symmetric ternary tree partition type is a partition type that is symmetric about the center line of a block, but all the resulting blocks are the same size. Such partitions are possible when the block height or width is a multiple of three, depending on the vertical or horizontal division. An uneven / non-homomorphic asymmetric ternary tree partition type is a partition type that is not symmetric about the center line of a block, but at least one of the resulting blocks is not the same size as the other two.

[0090] In examples where a block is divided into an asymmetric ternary tree partition type (e.g., at a subtree node), the encoder 200 and / or the decoder 300 may apply a constraint such that two of the three partitions have the same size. Such a constraint may correspond to a restriction that the encoder 200 must follow when encoding the video data. Additionally, in some examples, the encoder 200 and the decoder 300 may apply a constraint such that when dividing according to an asymmetric ternary tree partition type, the sum of the areas of two partitions is equal to the area of ​​the remaining partition.

[0091] In some examples, the encoder 200 may be configured to select from among all of the aforementioned partition types for each of the QT, BT, and TT partition structures. In other examples, the encoder 200 may be configured to simply determine the partition type from among a subset of the aforementioned partition types. For example, a subset of the aforementioned partition types (or other partition types) may be used for a particular block size or for a particular depth of the quadtree structure. The subset of supported partition types may be signaled in the bitstream for use by the decoder 200, or may be predetermined, allowing the encoder 200 and the decoder 300 to determine the subset without any signaling.

[0092] In another example, the number of supported partition types may be fixed for all depths in all CTUs. That is, the encoder 200 and the decoder 300 may be pre-configured to use the same number of partition types for any depth of the CTU. In another example, the number of supported partition types may vary and may depend on the depth, slice type, or other pre-encoded information. In one example, at depth 0 or depth 1 of the tree structure, only the QT partition structure is used. At depths greater than 1, each of the QT, BT, and TT partition structures may be used.

[0093] In some examples, the encoder 200 and / or the decoder 300 may apply preconfigured constraints to the supported partition types to avoid overlapping partitions for certain regions of a video picture or region of a CTU. In one example, when a block is partitioned according to an asymmetric partition type, the encoder 200 and / or the decoder 300 may be configured to not further partition the largest sub-block partitioned from the current block. For example, when a square block is partitioned according to an asymmetric partition type (as with the PART_2N×nU partition type), the largest sub-block among all the sub-blocks is a designated leaf node (as with the largest sub-block of the PART_2N×nU partition type) and cannot be further partitioned. However, the smaller sub-blocks (as with the smaller sub-blocks of the PART_2N×nU partition type) can be further partitioned.

[0094] As another example where constraints on supported partition types may be applied to avoid overlapping partitions for a particular region, when a block is divided by an asymmetric partition type, the largest sub-block divided from the current block cannot be further divided in the same direction. For example, when a square block is divided by an asymmetric partition type (as with the PART_2N×nU partition type), the encoder 200 and / or the decoder 300 may be configured to not divide larger sub-blocks among all sub-blocks horizontally (as with the largest sub-block of the PART_2N×nU partition type).

[0095] As another example where constraints on supported partition types may be applied to avoid difficulties in further partitioning, the encoder 200 and / or decoder 300 may be configured to not partition a block either horizontally or vertically when the width / height of the block is not a power of two (e.g., when the width / height is not 2, 4, 8, 16, etc.).

[0096] The above examples have described how the encoder 200 may be configured to perform MTT partitioning. The decoder 300 may therefore also apply the same MTT partitioning as performed by the encoder 200. In some examples, how a picture of the video data is partitioned by the encoder 200 may be determined by applying the same set of predefined rules in the decoder 300. However, in many situations, the encoder 200 may determine a particular partition structure and partition type to use based on a rate-distortion criterion for a particular picture of the video data being encoded. Thus, in order for the decoder 300 to determine the partitions for a particular picture, the encoder 200 may signal syntax elements in the encoded bitstream that indicate how the picture and the CTUs of the picture should be partitioned. The decoder 200 may parse such syntax elements and partition the picture and the CTUs accordingly.

[0097] In one example, prediction processing unit 260 of video encoder 200 may be configured to perform any combination of the above-mentioned partitioning techniques, particularly for motion compensation, as described in more detail below.

[0098] Similar to picture 201, block 203 is again or can be considered as a two-dimensional array or matrix of samples with intensity values ​​(sample values), but with smaller dimensions than picture 201. In other words, block 203 may contain, for example, one sample array (e.g., a luma array in case of a monochrome picture 201), or three sample arrays (e.g., a luma and two chroma arrays in case of a color picture 201), or any other number and / or type of arrays depending on the color format applied. The number of samples in the horizontal and vertical directions (or axes) of block 203 determine the size of block 203.

[0099] An encoder 200 as shown in Fig. 2 is arranged to encode a picture 201 block by block. For example, encoding and prediction are performed block by block 203.

[0100] Residual calculation

[0101] The residual calculation unit 204 is configured to calculate the residual block 205 based on the picture block 203 and the prediction block 265 (further details regarding the prediction block 265 are provided later), for example by subtracting sample values ​​of the prediction block 265 from sample values ​​of the picture block 203 sample-by-sample (pixel-by-pixel) to obtain the residual block 205 in the sample domain.

[0102] conversion

[0103] The transform processing unit 206 is configured to apply a transform, such as a discrete cosine transform (DCT) or a discrete sine transform (DST), to the sample values ​​of the residual block 205 to obtain transform coefficients 207 in a transform domain. The transform coefficients 207 may also be referred to as transformed residual coefficients and represent the residual block 205 in the transform domain.

[0104] The transform processing unit 206 may be configured to apply an integer approximation of a DCT / DST, such as the transform specified for HEVC / H.265. Compared to an orthogonal DCT transform, such an integer approximation is typically scaled by a specific factor. In order to maintain the norm of the residual blocks processed by the forward and inverse transforms, an additional scaling factor is applied as part of the transform processing. The scaling factor is typically selected based on specific constraints, such as the scaling factor being a power of two for shift operations, the bit depth of the transform coefficients, a trade-off between accuracy and implementation cost, etc. A specific scaling factor may be specified, for example, for the inverse transform, e.g., by the inverse transform processing unit 212 in the decoder 300 (and the corresponding inverse transform, e.g., by the inverse transform processing unit 212 in the encoder 200), and a corresponding scaling factor for the forward transform, e.g., by the transform processing unit 206 in the encoder 200, may be specified accordingly.

[0105] quantization

[0106] The quantization unit 208 is configured to quantize the transform coefficients 207 to obtain quantized transform coefficients 209, for example by applying scalar quantization or vector quantization. The quantized transform coefficients 209 may also be referred to as quantized residual coefficients 209. The quantization process may reduce a bit depth associated with some or all of the transform coefficients 207. For example, an n-bit transform coefficient may be truncated to an m-bit transform coefficient during quantization, where n is greater than m. The degree of quantization may be changed by adjusting a quantization parameter (QP). For example, in scalar quantization, different scaling may be applied to achieve finer or coarser quantization. A smaller quantization step size corresponds to a finer quantization. Meanwhile, a larger quantization step size corresponds to a coarser quantization. The applicable quantization step sizes may be indicated by a quantization parameter (QP). The quantization parameter may, for example, be an index to a predefined set of applicable quantization step sizes. For example, a small quantization parameter may correspond to a finer quantization (smaller quantization step size) and a large quantization parameter may correspond to a coarser quantization (larger quantization step size), and vice versa. Quantization may include division by the quantization step size. The corresponding or inverse inverse quantization, e.g. by the inverse quantizer 210, may include multiplication by the quantization step size. Some standards, e.g., embodiments according to HEVC, may be configured to use the quantization parameter to determine the quantization step size. Typically, the quantization step size may be calculated based on the quantization parameter using a fixed-point approximation of a formula that includes a division. Additional scaling factors for quantization and inverse quantization may be introduced to restore the norm of the residual block that may be changed due to the scaling used in the fixed-point approximation of the formula of the quantization step size and the quantization parameter. In one exemplary implementation, the scaling of the inverse transform and the inverse quantization may be combined. Alternatively, customized quantization tables may be used and signaled from the encoder to the decoder, e.g., in the bitstream.Quantization is a lossy operation, and the loss increases with increasing quantization step size.

[0107] The inverse quantization unit 210 is configured to apply the inverse quantization of the quantization unit 208 to the quantized coefficients, e.g., by applying the inverse of the quantization scheme applied by the quantization unit 208, based on or using the same quantization step size as the quantization unit 208, to obtain the inverse quantized coefficients 211. The inverse quantized coefficients 211, also referred to as the inverse quantized residual coefficients 211, may correspond to the transform coefficients 207, although they are not typically the same as the transform coefficients due to losses due to quantization.

[0108] The inverse transform processing unit 212 is configured to apply an inverse transform of the transform applied by the transform processing unit 206, such as an inverse discrete cosine transform (DCT) or an inverse discrete sine transform (DST), to obtain an inverse transform block 213 in the sample domain. The inverse transform block 213 may also be referred to as an inverse transformed dequantized block 213 or an inverse transformed residual block 213.

[0109] The reconstruction unit 214 (e.g., adder 214) is configured to add the inverse transform block 213 (i.e., the reconstructed residual block 213) to the prediction block 265 to obtain a reconstructed block 215 in the sample domain, for example by adding sample values ​​of the reconstructed residual block 213 and sample values ​​of the prediction block 265.

[0110] Optionally, a buffer unit 216 (or "buffer" 216), e.g. a line buffer 216, is configured to buffer or store the reconstructed blocks 215 and the respective sample values, e.g. for intra prediction. In further embodiments, the encoder may be configured to use the unfiltered reconstructed blocks and / or the respective sample values ​​stored in the buffer unit 216 for any kind of estimation and / or prediction, e.g. intra prediction.

[0111] An embodiment of the encoder 200 may be configured, for example, such that the buffer unit 216 is used to store the reconstructed blocks 215 not only for intra prediction 254 but also for the loop filter unit 220 (not shown in FIG. 2) and / or such that the buffer unit 216 and the decoded picture buffer unit 230 form one buffer. Further embodiments may be configured to use the filtered blocks 221 and / or blocks or samples from the decoded picture buffer 230 (both not shown in FIG. 2) as input or basis for the intra prediction 254.

[0112] The loop filter unit 220 (or “loop filter” 220) is configured to filter the reconstructed block 215 to obtain a filtered block 221, e.g., to smooth pixel transitions or otherwise improve video quality. The loop filter unit 220 is intended to represent one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or other filters, e.g., a bilateral filter or an adaptive loop filter (ALF), or a sharpening or smoothing filter, or a collaborative filter. Although the loop filter unit 220 is illustrated in FIG. 2 as being an in-loop filter, in other configurations, the loop filter unit 220 may be implemented as a post-loop filter. The filtered block 221 may be referred to as a filtered reconstructed block 221. The decoded picture buffer 230 may store the reconstructed coding block after the loop filter unit 220 performs a filtering operation on the reconstructed coding block.

[0113] An embodiment of the encoder 200 (respectively the loop filter unit 220) may be configured to output loop filter parameters (such as sample adaptive offset information), e.g., directly or entropy coded by the entropy coding unit 270 or any other entropy coding unit, so that the decoder 300, for example, may receive and apply the same loop filter parameters for decoding.

[0114] The decoded picture buffer (DPB) 230 may be a reference picture memory that stores reference picture data for use in encoding the video data by the video encoder 20. The DPB 230 may be formed of any of a variety of memory devices, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magneto resistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. The DPB 230 and the buffer 216 may be provided by the same memory device or separate memory devices. In some examples, the decoded picture buffer (DPB) 230 is configured to store the filtered blocks 221. The decoded picture buffer 230 may further be configured to store other previously filtered blocks of the same current picture or of a different picture, e.g., a previously reconstructed picture, e.g., previously reconstructed and filtered block 221, and may provide a complete previously reconstructed, i.e., decoded picture (and corresponding reference blocks and samples) and / or a partially reconstructed current picture (and corresponding reference blocks and samples), e.g., for inter prediction. In some examples, the decoded picture buffer (DPB) 230 is configured to store the reconstructed block 215 if the reconstructed block 215 is reconstructed but without in-loop filtering.

[0115] The prediction processing unit 260, also referred to as the block prediction processing unit 260, is configured to receive or obtain the block 203 (the current block 203 of the current picture 201) and reconstructed picture data, e.g. reference samples of the same (current) picture from the buffer 216 and / or reference picture data 231 from one or more previously decoded pictures from the decoded picture buffer 230, and to process such data for prediction, i.e. to provide a prediction block 265, which may be an inter-prediction block 245 or an intra-prediction block 255.

[0116] The mode selection unit 262 may be configured to select a prediction mode (e.g., intra or inter prediction mode) and / or a corresponding prediction block 245 or 255 to be used as the prediction block 265 for calculation of the residual block 205 and for reconstruction of the reconstruction block 215.

[0117] An embodiment of the mode selection unit 262 may be configured to select a prediction mode (e.g., from those supported by the prediction processing unit 260) that is most suitable, or in other words provides the smallest residual (smallest residual means better compression for transmission or storage) or the smallest signaling overhead (smallest signaling overhead means better compression for transmission or storage), or that considers or balances both. The mode selection unit 262 may be configured to determine the prediction mode based on rate distortion optimization (RDO), i.e., select the prediction mode that provides the smallest rate distortion optimization or is associated with a rate distortion that at least satisfies a prediction mode selection criterion.

[0118] The prediction process (eg, by prediction processing unit 260) and mode selection (eg, by mode selection unit 262) performed by example encoder 200 are described in further detail below.

[0119] As mentioned above, the encoder 200 is configured to determine or select a best or optimal prediction mode from a (predetermined) set of prediction modes, which may include, for example, intra-prediction modes and / or inter-prediction modes.

[0120] The set of intra prediction modes may include 35 different intra prediction modes, e.g., omnidirectional modes such as DC (or average) mode and planar mode, or directional modes such as those defined in H.265, or may include 67 different intra prediction modes, e.g., omnidirectional modes such as DC (or average) mode and planar mode, or directional modes such as those defined in the currently under development H.266.

[0121] The set of inter prediction modes (or possible) depends on the available reference pictures (i.e., previous, at least partially decoded pictures, e.g., stored in DBP 230) and other inter prediction parameters, e.g., whether the entire reference picture or only a portion of it is used to search for the best matching reference block, e.g., a search window region around the region of the current block of the reference picture, and / or whether, e.g., pixel interpolation is applied, e.g., half / semi-pel and / or quarter-pel interpolation.

[0122] In addition to the prediction modes mentioned above, skip mode and / or direct mode may be applied.

[0123] Prediction processing unit 260 may be further configured to partition block 203 into smaller block partitions or sub-blocks, e.g., using quad-tree-partitioning (QT), binary-tree-partitioning (BT), triple-tree-partitioning (TT), or any combination thereof iteratively, and perform prediction for each block partition or sub-block, e.g., where mode selection includes selecting a prediction mode to be applied to each of the tree structure of partitioned block 203 and the block partitions or sub-blocks.

[0124] The inter prediction unit 244 may include a motion estimation (ME) unit and a motion compensation (MC) unit (not shown in FIG. 2). The motion estimation unit is configured to receive or obtain the picture block 203 (the current picture block 203 of the current picture 201) and at least one or more of the decoded picture 331 or previously reconstructed blocks, such as reconstructed blocks of one or more other / different previously decoded pictures 331, for motion estimation. For example, a video sequence may include the current picture and the previous decoded picture 331. In other words, the current picture and the previous decoded picture 331 may be part of or form a sequence of pictures forming a video sequence. The encoder 200 may be configured to, for example, select a reference block from multiple reference blocks of the same or different pictures of multiple other pictures, and provide the reference picture (or reference picture index, ...) and / or the offset (spatial offset) between the position (x, y coordinates) of the reference block and the position of the current block as an inter prediction parameter to a motion estimation unit (not shown in Fig. 2). This offset is also called a motion vector (MV). Merging is an important motion estimation tool used in HEVC and inherited to VVC. To perform merge estimation, the first thing to be done is to construct a merge candidate list, where each of the candidates contains the full motion data including, for each list, information on whether one or two reference picture lists are used as well as the reference index and motion vector. The merge candidate list is constructed based on the following candidates: a. up to four spatial merge candidates derived from five spatially neighboring (i.e., nearby) blocks, b. one temporal merge candidate derived from two temporally co-located blocks, c. additional merge candidates including combined bi-predictive candidates and zero motion vector candidates.

[0125] The intra prediction unit 254 is further configured to determine an intra prediction block 255 based on the intra prediction parameters, e.g., a selected intra prediction mode. In any case, after selecting an intra prediction mode for the block, the intra prediction unit 254 is also configured to provide the intra prediction parameters, i.e., information indicative of the selected intra prediction mode for the block, to the entropy coding unit 270. In one example, the intra prediction unit 254 may be configured to perform any combination of the intra prediction techniques described below.

[0126] The entropy encoding unit 270 is configured to apply an entropy encoding algorithm or scheme (e.g. a variable length coding (VLC) scheme, a context adaptive VLC scheme (CALVC), an arithmetic coding scheme, a context adaptive binary arithmetic coding (CABAC)), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding or another entropy encoding method or technique) to the quantized residual coefficients 209, the inter prediction parameters, the intra prediction parameters, and / or the loop filter parameters, individually or jointly (or not at all) to obtain encoded picture data 21, which can be output via an output 272, for example in the form of an encoded bitstream 21. Encoded bitstream 21 may be transmitted to video decoder 30 or archived for later transmission or retrieval by video decoder 30. Entropy encoding unit 270 may further be configured to entropy encode other syntax elements of the current video slice being encoded.

[0127] Other structural variations of the video encoder 200 may be used to encode the video stream. For example, a non-transform-based encoder 200 may directly quantize the residual signal for a particular block or frame without having a transform processing unit 206. In another implementation, the encoder 200 may have the quantization unit 208 and the inverse quantization unit 210 combined into a single unit.

[0128] 3 shows an example video decoder 300 configured to implement the techniques of this disclosure. The video decoder 300 is configured to receive encoded picture data (e.g., encoded bitstream) 271, e.g., encoded by encoder 200, to obtain decoded pictures 331. During the decoding process, the video decoder 300 receives video data from video encoder 200, e.g., an encoded video stream representing picture blocks of encoded video slices and associated syntax elements.

[0129] 3, the decoder 300 includes an entropy decoding unit 304, an inverse quantization unit 310, an inverse transform processing unit 312, a reconstruction unit 314 (e.g., summer 314), a buffer 316, a loop filter 320, a decoded picture buffer 330, and a prediction processing unit 360. The prediction processing unit 360 may include an inter prediction unit 344, an intra prediction unit 354, and a mode selection unit 362. The video decoder 300 may, in some examples, perform a decoding path that is generally reciprocal to the encoding paths described with respect to the video encoder 200 from FIG.

[0130] The entropy decoding unit 304 is configured to perform entropy decoding on the coded picture data 271 to obtain, e.g., quantized coefficients 309, and / or decoded coding parameters (not shown in FIG. 3), such as inter-prediction parameters, intra-prediction parameters, loop filter parameters, and / or other syntax elements. The entropy decoding unit 304 is further configured to forward the inter-prediction parameters, intra-prediction parameters, and / or other syntax elements to the prediction processing unit 360. The video decoder 300 may receive video slice-level and / or video block-level syntax elements.

[0131] The inverse quantization unit 310 may be functionally identical to the inverse quantization unit 110, the inverse transform processing unit 312 may be functionally identical to the inverse transform processing unit 112, the reconstruction unit 314 may be functionally identical to the reconstruction unit 114, the buffer 316 may be functionally identical to the buffer 116, the loop filter 320 may be functionally identical to the loop filter 120, and the decoded picture buffer 330 may be functionally identical to the decoded picture buffer 130.

[0132] Prediction processing unit 360 may include an inter prediction unit 344 and an intra prediction unit 354, where the inter prediction unit 344 may be functionally similar to the inter prediction unit 144 and the intra prediction unit 354 may be functionally similar to the intra prediction unit 154. Prediction processing unit 360 is typically configured to perform block prediction and / or obtain a prediction block 365 from the encoded data 21, and to receive or obtain (explicitly or implicitly) prediction related parameters and / or information regarding a selected prediction mode, for example from entropy decoding unit 304.

[0133] When a video slice is coded as an intra coded (I) slice, intra prediction unit 354 of prediction processing unit 360 is configured to generate a prediction block 365 for a picture block of the current video slice based on a signaled intra prediction mode and data from a previously decoded block of the current frame or picture. When a video frame is coded as an inter coded (i.e., B or P) slice, inter prediction unit 344 (e.g., a motion compensation unit) of prediction processing unit 360 is configured to generate a prediction block 365 for a video block of the current video slice based on a motion vector and other syntax elements received from entropy decoding unit 304. In inter prediction, a prediction block may be generated from one of the reference pictures in one of the reference picture lists. Video decoder 300 may construct the reference frame lists: List 0 and List 1 using a specified construction technique based on the reference pictures stored in DPB 330.

[0134] Prediction processing unit 360 is configured to determine prediction information for video blocks of the current video slice by parsing the motion vectors and other syntax elements, and to use the prediction information to generate a predictive block for the current video block being decoded. For example, prediction processing unit 360 uses some of the received syntax elements to determine a prediction mode (e.g., intra or inter prediction) used to encode video blocks of the video slice, an inter-prediction slice type (e.g., a B slice, a P slice, or a GPB slice), configuration information for one or more of the reference picture lists of the slice, motion vectors for each inter-coded video block of the slice, an inter-prediction state for each inter-coded video block of the slice, and other information for decoding video blocks in the current video slice.

[0135] The inverse quantization unit 310 is configured to inverse quantize, or dequantize, the quantized transform coefficients provided in the bitstream and decoded by the entropy decoding unit 304. The inverse quantization process may include determining, for each video block in a video slice, the degree of quantization, and likewise the degree of inverse quantization to be applied, using a quantization parameter calculated by the video encoder 100.

[0136] Inverse transform processing unit 312 is configured to apply an inverse transform, eg, an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process, to the transform coefficients to produce residual blocks in the pixel domain.

[0137] The reconstruction unit 314 (e.g., adder 314) is configured to add the inverse transform block 313 (i.e., the reconstructed residual block 313) to the prediction block 365 to obtain a reconstructed block 315 in the sample domain, for example by adding sample values ​​of the reconstructed residual block 313 and sample values ​​of the prediction block 365.

[0138] The loop filter unit 320 (in the coding loop or after the coding loop) is configured to filter the reconstructed block 315 to obtain a filtered block 321, e.g., to smooth pixel transitions or otherwise improve video quality. In one example, the loop filter unit 320 may be configured to perform any combination of the filtering techniques described below. The loop filter unit 320 is intended to represent one or more filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or other filters, e.g., a bilateral filter or an adaptive loop filter (ALF), or a sharpening or smoothing filter, or a collaborative filter. Although the loop filter unit 320 is illustrated in FIG. 3 as being an in-loop filter, in other configurations, the loop filter unit 320 may be implemented as a post-loop filter.

[0139] The decoded video blocks 321 in a given frame or picture are then stored in a decoded picture buffer 330, which stores reference pictures used for subsequent motion compensation.

[0140] The decoder 300 is arranged to output the decoded pictures 311, for presentation or viewing to a user, eg via an output 312.

[0141] Other variations of the video decoder 300 may be used to decode the compressed bitstream. For example, the decoder 300 may generate an output video stream without having a loop filter unit 320. For example, a non-transform-based decoder 300 may directly inverse quantize the residual signal for a particular block or frame without having an inverse transform processing unit 312. In another implementation, the video decoder 300 may have the inverse quantization unit 310 and the inverse transform processing unit 312 combined into a single unit.

[0142] 4 is a schematic diagram of a network device 400 (e.g., an encoding device) according to an embodiment of the present disclosure. The network device 400 is suitable for implementing embodiments of the disclosure as described herein. In an embodiment, the network device 400 may be a decoder, such as the video decoder 30 of FIG. 1A, or an encoder, such as the video encoder 20 of FIG. 1A. In an embodiment, the network device 400 may be one or more components of the video decoder 30 of FIG. 1A or the video encoder 20 of FIG. 1A, as described above.

[0143] Network device 400 includes an ingress port 410 and a receiver unit (Rx) 420 for receiving data, a processor, logic unit, or central processing unit (CPU) 430 for processing data, a transmitter unit (Tx) 440 and an egress port 450 for transmitting data, and a memory 460 for storing data. Network device 400 may also include optical-to-electrical (OE) and electrical-to-optical (EO) components for egress or ingress of optical or electrical signals connected to ingress port 410, receiver unit 420, transmitter unit 440, and egress port 450.

[0144] The processor 430 is implemented by hardware and software. The processor 430 may be implemented as one or more CPU chips, cores (such as a multi-core processor), FPGAs, ASICs, and DSPs. The processor 430 communicates with the ingress port 410, the receiver unit 420, the transmitter unit 440, the egress port 450, and the memory 460. The processor 430 includes an encoding module 470. The encoding module 470 implements the above-disclosed embodiments. For example, the encoding module 470 implements, processes, prepares, or provides various encoding operations. The inclusion of the encoding module 470 thus provides substantial improvements to the functionality of the network device 400 and results in the transformation of the network device 400 into a different state. Alternatively, the encoding module 470 is implemented as instructions stored in the memory 460 and executed by the processor 430.

[0145] Memory 460 may include one or more disks, tape drives, and solid state drives, and may be used to store programs when they are selected for execution and as overflow data storage for storing instructions and data read during execution of the programs. Memory 460 may be volatile and / or non-volatile memory, and may be read-only memory (ROM), random access memory (RAM), ternary content-addressable memory (TCAM), and / or static random-access memory (SRAM).

[0146] 5 is a simplified block diagram of a device 500 that may be used as one or both of source device 12 and destination device 14 from FIG. 1A in accordance with an exemplary embodiment. Device 500 is capable of implementing the techniques of this disclosure. Device 500 may be in the form of a computing system including multiple computing devices, or in the form of a single computing device, such as a mobile phone, tablet computer, laptop computer, notebook computer, desktop computer, etc.

[0147] The processor 502 in the device 500 may be a central processing unit. Alternatively, the processor 502 may be any other type of device or devices capable of manipulating or processing information, either now existing or developed in the future. Although implementations of the disclosure may be performed with a single processor, such as the processor 502, as shown, benefits in speed and efficiency may be achieved using more than one processor.

[0148] The memory 504 in the device 500 may be a read only memory (ROM) device or a random access memory (RAM) device in implementation. Any other suitable type of storage device may be used as the memory 504. The memory 504 may include code and data 506 that is accessed by the processor 502 using a bus 512. The memory 504 may further include an operating system 508 and application programs 510. The application programs 510 include at least one program that enables the processor 502 to perform the methods described herein. For example, the application programs 510 may include applications 1-N, further including a video encoding application that performs the methods described herein. The device 500 may also include additional memory in the form of a secondary storage 514, which may be, for example, a memory card used with a mobile computing device. Because video communication sessions may include a significant amount of information, they may be stored in whole or in part in the secondary storage 514 and loaded into the memory 504 as needed for processing.

[0149] The device 500 may also include one or more output devices, such as a display 518. The display 518, in one example, may be a touch-sensitive display that combines a display with touch-sensitive elements operative to sense touch input. The display 518 may be coupled to the processor 502 via the bus 512. Other output devices that enable a user to program or otherwise use the device 500 may be provided in addition to or as an alternative to the display 518. When the output device is or includes a display, the display may be implemented in a variety of ways, including a liquid crystal display (LCD), a cathode-ray tube (CRT) display, a plasma display, or a light emitting diode (LED) display, such as an organic LED (OLED) display.

[0150] Device 500 may also include or communicate with an image sensing device 520, such as a camera or any other existing or future developed image sensing device 520 capable of sensing images, such as an image of a user operating device 500. Image sensing device 520 may be positioned to be pointed at a user operating device 500. In one example, the position and optical axis of image sensing device 520 may be configured such that the field of view includes an area immediately adjacent to display 518 and from which display 518 is visible.

[0151] Device 500 may also include or communicate with a voice sensing device 522, such as a microphone or any other existing or future developed voice sensing device capable of sensing sound proximate device 500. Voice sensing device 522 may be positioned to face a user operating device 500 and configured to receive sound, e.g., speech or other utterances, generated by the user while the user is operating device 500.

[0152] Although FIG. 5 depicts the processor 502 and memory 504 of device 500 as integrated in a single unit, other configurations are available. The operations of processor 502 may be distributed across multiple machines (each machine having one or more processors) that may be directly connected or across a local area or other network. Memory 504 may be distributed across multiple machines, such as a network-based memory or memory among multiple machines that perform the operations of device 500. Although shown here as a single bus, bus 512 of device 500 may be comprised of multiple buses. Additionally, secondary storage 514 may be directly connected to other components of device 500 or may be accessible over a network, and may include a single integrated unit such as a memory card or multiple units such as multiple memory cards. Device 500 may thus be implemented in a variety of configurations.

[0153] In one or more examples, the functions described may be implemented by hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over a computer-readable medium as one or more instructions or code and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium, which corresponds to a tangible medium, such as a data storage medium, or a communication medium, which includes any medium that facilitates the transfer of a computer program from one place to another, for example according to a communication protocol. As such, a computer-readable medium may generally correspond to (1) a tangible computer-readable storage medium that is non-transitory, or (2) a communication medium, such as a signal or carrier wave. A data storage medium may be any available medium accessible by one or more computers or one or more processors to read instructions, code, and / or data structures for implementing the techniques described in this disclosure. A computer program product may include a computer-readable medium.

[0154] Video compression techniques such as motion compensation, intra prediction, and loop filter have proven to be efficient and therefore have been adopted in various video coding standards such as H.264 / AVC and H.265 / HEVC. Intra prediction can be used when there is no reference picture available or when inter-predictive coding is not used for the current block or picture, e.g., an I-frame or an I-slice. Reference samples for intra prediction are usually derived from previously coded (or reconstructed) neighboring blocks in the same picture. For example, in both H.264 / AVC and H.265 / HEVC, boundary samples of adjacent blocks are used as references for intra prediction. There are many different intra prediction modes to cover different textures or structural features. In each mode, a different prediction signal derivation method is used. For example, H.265 / HEVC supports a total of 35 intra prediction modes, as shown in FIG. 6.

[0155] Explanation of the intra prediction algorithm in H.265 / HEVC

[0156] In intra prediction, the decoded boundary samples of neighboring blocks are used as references. The encoder selects the best luma intra prediction mode for each block from 35 options: 33 directional prediction modes, DC mode, and planar mode. The mapping between intra prediction direction and intra prediction mode number is specified in Figure 6. It should be noted that 65 or even more intra prediction modes have been developed in modern video coding techniques, such as versatile video coding (VVC), which can capture any edge direction that appears in the raw video.

[0157] As shown in Figure 7, block "CUR" is the current block to be predicted, and the gray samples along the boundaries of adjacent constructed blocks (left and top of the current block) are used as reference samples. The prediction signal can be derived by mapping the reference samples according to a specific method indicated by the intra prediction mode.

[0158] Referring to Figure 7, W is the width of the current block, and H is the height of the current block. W1 is the number of reference samples above. H1 is the number of reference samples to the left. Usually, W1>W and H1>H, which means that the reference samples above also include the reference (neighboring) samples on the top right, and the reference samples to the left also include the reference (neighboring) samples on the bottom left. For example, H1=2*H, W1=2*W, or H1=H+W, W1=W+H.

[0159] Reference samples are not always available. For example, as shown in Figure 8, after the availability check process, W2 samples above or above the current block are available, and H2 samples to the left of the current block are available. W3 samples above are unavailable, and H3 samples to the left are unavailable.

[0160] Those unavailable samples need to be replaced (or padded) with available samples before obtaining the predicted signal. For example, unavailable samples are replaced by scanning the reference samples in a clockwise direction and using the latest available sample value for the unavailable one. For the lower part of the left reference samples, if they are unavailable, they are replaced by the value of the nearest available reference sample.

[0161] Reference sample availability check The reference sample availability check means checking whether the reference sample is available, e.g., if the reference sample is reconstructed, then the reference sample is available.

[0162] In existing methods, the availability check process is usually performed by checking the luma samples, which means that for a block of chroma components, the reference sample availability is also derived by checking the availability of the corresponding luma sample.

[0163] In the proposed method of the present disclosure, for a block, a reference sample availability check process is performed by checking samples of its own component, where the component may be a Y component, a Cb component or a Cr component. According to the proposed method, for a block, a reference sample availability check process is performed by checking samples of its corresponding component, where the component may be a luma component or a chroma component, which may include both a Cb component and a Cr component. Here, the Cb component and the Cr component are called chroma components and are not distinguished in the availability check process.

[0164] A set of methods is presented in this invention, emphasizing the following two aspects:

[0165] According to a first aspect of the present disclosure, for a block, a reference sample availability check process is performed by checking the samples of its own component, where the component may be a Y component, a Cb component or a Cr component.

[0166] For a Y block, the availability of a reference sample for the Y block is derived by checking whether the reference sample is available, for example by checking whether the reference sample has been reconstructed.

[0167] For a Cb block, the availability of a reference sample for the Cb block is derived by checking whether a reference sample is available, for example, by checking whether the reference sample has been reconstructed.

[0168] For a Cr block, the availability of a reference sample for the Cr block is derived by checking whether the reference sample is available, for example, by checking whether the reference sample has been reconstructed.

[0169] According to a second aspect of the present disclosure, for a block, a reference sample availability check process is performed by checking the samples of its corresponding components, where the components can be luma or chroma components. The Y component is referred to as the luma component. Both the Cb and Cr components are referred to as the chroma components. Here, the Cb and Cr components are not distinguished in the availability check process.

[0170] For a luma block, the availability of a reference sample for the luma block is derived by checking whether the reference sample is available, for example, by checking whether the reference sample is reconstructed.

[0171] For a chroma block, the availability of a reference sample for the chroma block is derived by checking whether the reference sample is available, for example by checking whether the reference sample has been reconstructed.

[0172] In existing solutions, for a block, the reference sample availability check process is always performed by checking the luma sample, regardless of which component the block belongs to, whereas in the proposed method of this disclosure, for blocks that belong to different components, their reference sample availability check process is performed by checking different component samples.

[0173] It should be noted here that the method proposed in this disclosure is used to derive the availability of reference samples for blocks coded as intra-prediction modes. The method can be performed by the intra-prediction modules 254 or 354 shown in Figures 2 and 3, respectively. Therefore, the method exists on both the decoder side and the encoder side. And the process of checking the availability of reference samples for a block is the same in the encoder and the decoder.

[0174] Fig. 9 is a simplified flowchart illustrating a method for deriving a reconstructed signal according to an exemplary embodiment of the present disclosure. Referring to Fig. 9, for a block coded as an intra prediction mode, in order to obtain a reconstructed block (or signal, or sample), the method first includes a step of obtaining a prediction (or prediction signal or sample) of the block (901). Then, the method includes a step of obtaining a residual (or residual signal) of the block (902). Next, the method includes a step of deriving a reconstructed block by adding the residual to a prediction of the current block (903).

[0175] FIG. 10 is a simplified flowchart of deriving a prediction (or a prediction signal) according to an exemplary embodiment of the present disclosure. Referring to FIG. 10, to obtain a prediction signal, the method first includes a step (1001) of obtaining an intra-prediction mode of a current block, such as planar mode or DC mode, .... Then, the method includes a step (1002) of deriving the availability of a reference sample of a component of the current block. In an embodiment, the component includes a Y component, a Cb component, or a Cr component. In another embodiment, the component includes a luma component or a chroma component. The method may derive the availability of the reference sample by checking the availability of a Y sample in a neighboring Y block, which includes the reference sample, or may derive the availability of a reference sample by checking the availability of a Cb sample in a neighboring Cb block, which includes the reference sample, or may derive the availability of a reference sample by checking the availability of a Cr sample in a neighboring Cr block, which includes the reference sample.

[0176] If the method determines that unavailable reference samples exist (1003, yes), the method includes replacing or padding the unavailable reference samples with available reference samples (1004). The method then includes deriving a prediction for the block based on the intra-prediction mode and the replaced reference samples (1005). If the method determines that unavailable reference samples do not exist (1003, no), the method proceeds to 1005, which includes deriving a prediction for the current block based on the intra-prediction mode and the available reference samples. At 1006, the method includes reconstructing the current block based on the prediction.

[0177] It is noted here that the embodiments of the present disclosure described below are directed to a process for deriving a prediction signal and improving the reference sample availability check process.

[0178] EMBODIMENT 1 In this embodiment, the availability check process is performed by examining a sample of the component itself.

[0179] Referring to FIG. 10, for a block coded as an intra prediction mode, to obtain a prediction signal, the method includes:

[0180] Step 1: Get the intra prediction mode (1001)

[0181] The intra prediction mode is obtained by parsing the syntax related to the intra prediction mode in the bitstream. For example, for a Y block, intra_luma_mpm_flag, intra_luma_mpm_idx, or intra_luma_mpm_remainder is parsed. For a Cb or Cr block, the intra_chroma_pred_mode signal is parsed. Then, the intra prediction mode of the current block can be derived using the parsed syntax.

[0182] Step 2: Run the reference sample availability check process (1002)

[0183] This step involves checking the availability of a reference sample for the current block.

[0184] In an embodiment, when the block is a Y block, the reference sample availability is derived by looking at the reference sample of the Y component. Alternatively, the reference sample availability is derived by looking at neighboring Y samples, for example, by looking at the boundary samples of neighboring Y blocks.

[0185] In an embodiment, when the block is a Cb block, the reference sample availability is derived by looking at reference samples of the Cb component. Alternatively, the reference sample availability is derived by looking at neighboring Cb samples, for example by looking at boundary samples of neighboring Cb blocks.

[0186] In an embodiment, when the block is a Cr block, the reference sample availability is derived by looking at the reference samples of the Cr component. Alternatively, the reference sample availability is derived by looking at neighboring Cr samples, for example, by looking at the boundary samples of the neighboring Cr blocks.

[0187] Step 3: Determine if there are any unavailable reference samples (1003)

[0188] At 1003, the method includes determining whether there is an unavailable reference sample. If the method determines that there is an unavailable reference sample, the method proceeds to step 4 (1004), otherwise the method proceeds to step 5 (1005).

[0189] Step 4: Reference sample replacement (1004)

[0190] Reference sample replacement means deriving sample values ​​for unavailable reference samples using values ​​of available reference samples. In an embodiment, unavailable samples are replaced by scanning the reference samples in a clockwise direction and using the latest available sample value for the unavailable one. For the lower part of the left reference samples, if they are unavailable, they are replaced by the value of the nearest available reference sample.

[0191] Step 5: Derive a predicted signal (1005)

[0192] After the steps of obtaining the reference samples and the intra prediction mode, the prediction signal can be derived by mapping the reference samples to the current block, where the mapping method is indicated by the intra prediction mode.

[0193] After the step of deriving a prediction signal for the current block, a reconstructed signal for the current block can be derived by adding the residual signal to the derived prediction signal (903 in FIG. 9).

[0194] Note that after the current block is reconstructed, the samples in the block region can be marked as "available", as shown in Fig. 11. That is, when the current block is a Y block, all Y samples in the positions covered by the block region are available, when the current block is a Cb block, all Cb samples in the positions covered by the block region are available, and when the current block is a Cr block, all Cr samples in the positions covered by the block region are available.

[0195] An example specification of these sections could be given as follows: Reference sample availability marking process The inputs to this process are: - Sample position (xTbCmp, yTbCmp) that specifies the top-left sample of the current transform block relative to the top-left sample of the current picture. - Variable refIdx that specifies the intra prediction reference line index ·RefW variable that specifies the reference sample width · Variable refH that specifies the reference sample height · Variable cIdx that specifies the color component of the current block The output of this process is the reference sample refUnfilt[x][y], and in the case of intra sample prediction, x=-1-refIdx, y=-1-refIdx..refH-1, and x=-refIdx..refW-1, y=-1-refIdx. The refW+refH+1+(2*refIdx) neighboring samples refUnfilt[x][y], which are samples constructed before in-loop filtering, where x=-1-refIdx, y=-1-refIdx..refH-1, and x=-refIdx..refW-1, y=-1-refIdx, are derived as follows: The neighborhood position (xNbCmp, yNbCmp) is specified by: (xNbCmp,yNbCmp) =(xTbCmp+x,yTbCmp+y) (310) The neighboring block availability derivation procedure specified in Section 6.4.4 is called with inputs the current sample position (xCurr,yCurr) set equal to (xTbCmp,yTbCmp), the neighboring sample positions (xNbCmp,yNbCmp), checkPredModeY set equal to FALSE, and cIdx, and the output is assigned to availableN. Each sample refUnfilt[x][y] is derived as follows: If availableN is equal to FALSE, the sample refUnfilt[x][y] is marked as "unavailable for intra prediction". Otherwise, the sample refUnfilt[x][y] is marked as “available for intra prediction” and the sample at position (xNbCmp, yNbCmp) is assigned to refUnfilt[x][y]. Neighboring block availability derivation process The inputs to this process are: The sample position (xCurr, yCurr) of the top-left sample of the current block relative to the top-left sample of the current picture. The sample position (xNbCmp, yNbCmp) covered by the neighboring blocks for the top-left luma sample of the current picture, · checkPredModeY variable that specifies whether availability depends on the prediction mode · Variable cIdx that specifies the color component of the current block The output of this process is the availability of neighboring blocks covering position (xNbCmp, yNbCmp), denoted as availableN. The current luma position (xTbY, yTbY) and the neighboring luma positions (xNbY, yNbY) are derived as follows: (xTbY,yTbY)=(cIdx==0)?(xCurr,yCurr): (xCurr*SubWidthC,yCurr*SubHeightC) (xNbY,yNbY)=(cIdx==0)?(xNbCmp,yNbCmp): (xNbCmp*SubWidthC,yNbCmp*SubHeightC) The neighboring block availability, availableN, is derived as follows: · If one or more of the following conditions are TRUE, then availableN is set equal to FALSE: xNbCmp is less than 0. yNbCmp is less than 0. xNbY is greater than or equal to pic_width_in_luma_samples. yNbY is greater than or equal to pic_hight_in_luma_samples. IsAvailable[cIdx][xNbCmp]"yNbCmp] is equal to FALSE. The neighboring block is in a different slice than the current block. The neighboring block is contained in a different tile than the current block. entropy_coding_sync_enabled_flag is equal to 1 and (xNbY>>CtbLog2SizeY) is greater than or equal to (xTbY>>CtbLog2SizeY)+1. Otherwise, availableN is set equal to TRUE. If all of the following conditions are TRUE, then availableN is set equal to FALSE. ·checkPredModeY is equal to TRUE. · availableN is set equal to TRUE. CuPredMode[0][xNbY][yNbY] is not equal to CuPredMode[0][xTbY][yTbY]. IsAvailable[cIdx][x][y] used to store the sample availability information of each component cIdx at sample position (x,y). When cIdx=0, 0<=x<=pic_width_in_luma_samples, 0<=y<=pic_height_in_luma_samples, When cIdx=1 or 2, 0<=x<=pic_width_in_chroma_samples, 0<=y<=pic_height_in_chroma_samples. The inputs to this process are: A position (xCurr, yCurr) that specifies the top-left sample of the current block relative to the top-left sample of the current picture component. The variables nCurrSw and nCurrSh specify the width and height of the current block, respectively. · Variable cIdx that specifies the color component of the current block A (nCurrSw) x (nCurrSh) array predSamples that specifies the predicted samples for the current block. resamples, a (nCurrSw) x (nCurrSh) array that specifies the residual samples for the current block The output of this process is the reconstructed picture sample array recSamples. Depending on the value of the color component cIdx, the following assignments are made: If cIdx is equal to 0, recSamples is the reconstructed picture sample array S L Corresponds to. Otherwise, if cIdx is equal to 1, tuCbfChroma is set equal to tu_cbf_cb[xCurr][yCurr] and recSamples is the reconstructed chroma sample array S Cb Corresponds to. Otherwise (cIdx is equal to 2), tuCbfChroma is set equal to tu_cbf_cr[xCurr][yCurr] and recSamples is the reconstructed chroma sample array S Cr Corresponds to. Depending on the value of pic_lmcs_enabled_flag the following applies: If pic_lmcs_enabled_flag is equal to 0, the (nCurrSw) × (nCurrSh) block of reconstructed samples recSamples at position (xCurr, yCurr) is derived as follows for i = 0..nCurrSw-1, j = 0..nCurrSh-1: recSamples[xCurr+i][yCurr+j] =Clip1(predSamples[i][j]+recSamples[i][j]) (1195) Otherwise (pic_lmcs_enabled_flag is equal to 1), the following applies: If cIdx is equal to 0, the following applies: The picture reconstruction with luma sample mapping process as specified in subclause 8.7.5.2 is called with the luma position (xCurr, yCurr), the block width nCurrSw and height nCurrSh, the predicted luma sample array predSamples, and the residual luma sample array resSamples; the output is the reconstructed luma sample array recSamples. Otherwise (cIdx is greater than 0), the picture reconstruction with luma dependent chroma residual scaling operation of chroma samples as specified in clause 8.7.5.3 is called with inputs the chroma position (xCurr, yCurr), the transform block width nCurrSw and height nCurrSh, the coded block flag of the current chroma transform block tuCbfChroma, the predicted chroma sample array predSamples, and the residual chroma sample array resSamples, and the output is the reconstructed chroma sample array recSamples. The following assignments are made for i=0..nCurrSw-1, j=0..nCurrSh-1. xVb=(xCurr+i)%((cIdx==0)?IbcBufWidthY:IbcBufWidthC) (1196) yVb=(yCurr+j)%((cIdx==0)?CtbSizeY:(CtbSizeY / subHeightC)) (1197) IbcVirBuf[cIdx][xVb][yVb]=recSamples[xCurr+i][yCurr+j] (1198) IsAvailable[cIdx][xCurr+i][yCurr+j]=TRUE (1199)

[0196] Another example of the specification of these sections can be given as follows: Reference sample availability marking process The inputs to this process are: - Sample position (xTbCmp, yTbCmp) that specifies the top-left sample of the current transform block relative to the top-left sample of the current picture. - Variable refIdx that specifies the intra prediction reference line index ·RefW variable that specifies the reference sample width · Variable refH that specifies the reference sample height · Variable cIdx that specifies the color component of the current block The output of this process is the reference sample refUnfilt[x][y], and in the case of intra sample prediction, x=-1-refIdx, y=-1-refIdx..refH-1, and x=-refIdx..refW-1, y=-1-refIdx. The refW+refH+1+(2*refIdx) neighboring samples refUnfilt[x][y], x=-1-refIdx, y=-1-refIdx..refH-1, and x=-refIdx..refW-1, y=-1-refIdx, which are samples constructed before in-loop filtering, are derived as follows: The neighborhood position (xNbCmp, yNbCmp) is specified by: (xNbCmp,yNbCmp) =(xTbCmp+x,yTbCmp+y) (310) The neighboring block availability derivation procedure specified in Section 6.4.4 is called with inputs the current sample position (xCurr,yCurr) set equal to (xTbCmp,yTbCmp), the neighboring sample positions (xNbCmp,yNbCmp), checkPredModeY set equal to FALSE, and cIdx, and the output is assigned to availableN. Each sample of refUnfilt[x][y] is derived as follows: If availableN is equal to FALSE, the sample refUnfilt[x][y] is marked as "unavailable for intra prediction". Otherwise, the sample refUnfilt[x][y] is marked as “available for intra prediction” and the sample at position (xNbCmp, yNbCmp) is assigned to refUnfilt[x][y]. Neighboring block availability derivation process The inputs to this process are: The sample position (xCurr, yCurr) of the top-left sample of the current block relative to the top-left sample of the current picture. The sample position (xNbCmp, yNbCmp) covered by the neighboring blocks relative to the top-left luma sample of the current picture. · checkPredModeY variable that specifies whether availability depends on the prediction mode · Variable cIdx that specifies the color component of the current block The output of this process is the availability of neighboring blocks covering position (xNbCmp, yNbCmp), denoted as availableN. The current luma position (xTbY, yTbY) and the neighboring luma positions (xNbY, yNbY) are derived as follows: (xTbY,yTbY)=(cIdx==0)?(xCurr,yCurr): (xCurr*SubWidthC,yCurr*SubHeightC) (XXX) (xNbY,yNbY)=(cIdx==0)?(xNbCmp,yNbCmp): (XXX) (xNbCmp*SubWidthC,yNbCmp*SubHeightC) The neighboring block availability, availableN, is derived as follows: · If one or more of the following conditions are TRUE, then availableN is set equal to FALSE: xNbCmp is less than 0. yNbCmp is less than 0. xNbY is greater than or equal to pic_width_in_luma_samples. yNbY is greater than or equal to pic_hight_in_luma_samples. IsAvailable[cIdx][xNbCmp]"yNbCmp] is equal to FALSE. The neighboring block is in a different slice than the current block. The neighboring block is contained in a different tile than the current block. entropy_coding_sync_enabled_flag is equal to 1 and (xNbY>>CtbLog2SizeY) is greater than or equal to (xTbY>>CtbLog2SizeY)+1. Otherwise, availableN is set equal to TRUE. If all of the following conditions are TRUE, then availableN is set equal to FALSE. ·checkPredModeY is equal to TRUE. · availableN is set equal to TRUE. CuPredMode[0][xNbY][yNbY] is not equal to CuPredMode[0][xTbY][yTbY]. The inputs to this process are: A position (xCurr, yCurr) that specifies the top-left sample of the current block relative to the top-left sample of the current picture component. The variables nCurrSw and nCurrSh, which specify the width and height of the current block, respectively. · Variable cIdx that specifies the color component of the current block A (nCurrSw) x (nCurrSh) array predSamples that specifies the predicted samples for the current block. resamples, a (nCurrSw) x (nCurrSh) array that specifies the residual samples for the current block The output of this process is the reconstructed picture sample array recSamples. Depending on the value of the color component cIdx, the following assignments are made: If cIdx is equal to 0, recSamples is the reconstructed picture sample array S L Corresponds to. Otherwise, if cIdx is equal to 1, tuCbfChroma is set equal to tu_cbf_cb[xCurr][yCurr] and recSamples is the reconstructed chroma sample array S Cb Corresponds to. Otherwise (cIdx is equal to 2), tuCbfChroma is set equal to tu_cbf_cr[xCurr][yCurr] and recSamples is the reconstructed chroma sample array S Cr Corresponds to. Depending on the value of pic_lmcs_enabled_flag the following applies: If pic_lmcs_enabled_flag is equal to 0, the (nCurrSw) × (nCurrSh) block of reconstructed samples recSamples at position (xCurr, yCurr) is derived as follows for i = 0..nCurrSw-1, j = 0..nCurrSh-1: recSamples[xCurr+i][yCurr+j] =Clip1(predSamples[i][j]+recSamples[i][j]) (1195) Otherwise (pic_lmcs_enabled_flag is equal to 1), the following applies: If cIdx is equal to 0, the following applies: The picture reconstruction with luma sample mapping process as specified in subclause 8.7.5.2 is called with the luma position (xCurr, yCurr), the block width nCurrSw and height nCurrSh, the predicted luma sample array predSamples, and the residual luma sample array resSamples; the output is the reconstructed luma sample array recSamples. Otherwise (cIdx is greater than 0), the picture reconstruction with luma dependent chroma residual scaling operation of chroma samples as specified in clause 8.7.5.3 is called with inputs the chroma position (xCurr, yCurr), the transform block width nCurrSw and height nCurrSh, the coded block flag of the current chroma transform block tuCbfChroma, the predicted chroma sample array PredSamples, and the residual chroma sample array resSamples, and the output is the reconstructed chroma sample array recSamples. The following assignments are made for i=0..nCurrSw-1, j=0..nCurrSh-1. xVb=(xCurr+i)%((cIdx==0)?IbcBufWidthY:IbcBufWidthC) (1196) yVb=(yCurr+j)%((cIdx==0)?CtbSizeY:(CtbSizeY / subHeightC)) (1197) IbcVirBuf[cIdx][xVb][yVb]=recSamples[xCurr+i][yCurr+j] (1198) IsAvailable[cIdx][(xCurr+i)*((cIdx==0)?1:SubWidthC)][(yCurr+j)*((cIdx==0)?1:SubHeightC)]=TRUE (1199)

[0197] FIG. 12 is a graphic diagram illustrating samples in a current block marked as available in a unit N×N according to an exemplary embodiment of the present disclosure. In an embodiment, referring to FIG. 12, after the current block is reconstructed, samples are marked as "available" in a unit having a size N*N, such as N=4 or N=2. Alternatively, N=4 for a Y component block and N=2 for a Cb / Cr component block. Any sample in an "available" unit is considered to be "available". This means that if a unit is marked as "available", any sample in the unit area can be marked as "available". That is, when the current unit is a Y unit, all Y samples in positions covered by the unit area are available, when the current unit is a Cb unit, all Cb samples in positions covered by the unit area are available, and when the current block is a Cr unit, all Cr samples in positions covered by the unit area are available.

[0198] In an embodiment, to check whether a reference sample is available, the process first obtains the position or index of the unit to which the sample belongs, and if the unit is determined to be "available", the reference sample is considered or marked as available.

[0199] 13 is a graphic diagram illustrating samples at the right and bottom boundaries of a current block that are marked as available according to an exemplary embodiment of the present disclosure. In an embodiment, referring to FIG. 13, after the current block is reconstructed, only the right and bottom boundary samples (which are used as reference samples for other blocks) are marked as "available".

[0200] FIG. 14 is a graphic diagram illustrating samples at the right and bottom boundaries of a current block marked as available in unit N according to an exemplary embodiment of the present disclosure. In an embodiment, referring to FIG. 14, after the current block is reconstructed, only the right and bottom boundary samples (which are used as reference samples for other blocks) are marked as "available" in a unit size with N, such as N=4 or N=2. Alternatively, N=4 for a Y component block, and N=2 for a Cb / Cr component block. Any sample in an "available" unit is considered to be "available". This means that if a unit is marked as "available", any sample in the unit area can be marked as "available". In other words, when the current unit is a Y unit, all Y samples at positions covered by the unit area are available, when the current unit is a Cb unit, all Cb samples at positions covered by the unit area are available, and when the current block is a Cr unit, all Cr samples at positions covered by the unit area are available.

[0201] To check whether a reference sample is available, in an embodiment, the process first obtains the position or index of the unit to which the sample belongs, and if the unit is determined to be "available", the reference sample is marked as available.

[0202] FIG. 15 is a graphic diagram illustrating samples at the right and bottom boundaries of a current block marked as available in a unit N×N according to an exemplary embodiment of the present disclosure. In an embodiment, referring to FIG. 15, after the current block is reconstructed, only the right and bottom boundary samples (which are used as reference samples for other blocks) are marked as "available" in a unit having a size N*N, such as N=4 or N=2. Alternatively, N=4 for a Y component block, and N=2 for a Cb / Cr component block. Any sample in an "available" unit is considered to be "available". This means that once a unit is marked as "available", any sample in the unit area can be marked as "available". That is, when the current unit is a Y unit, all Y samples at positions covered by the unit area are available, when the current unit is a Cb unit, all Cb samples at positions covered by the unit area are available, and when the current block is a Cr unit, all Cr samples at positions covered by the unit area are available.

[0203] To check whether a reference sample is available, in an embodiment, a process or method first obtains the position or index of the unit to which the sample belongs, and if the process or method determines that the unit is "available", the process marks the reference sample as available.

[0204] It is noted here that, according to an embodiment of the present disclosure, the “available” information of samples or units is stored in memory for each component (all three components): for the Y component, for the Cb component, and for the Cr component.

[0205] It is noted here that even when a block is coded as an inter prediction mode, the "available" marking method can also be applied after the block is reconstructed.

[0206] EMBODIMENT 2 In this embodiment, the availability check process is performed by examining a sample of the corresponding component.

[0207] The difference between embodiment 2 and embodiment 1 is that the Cb and Cr components are not distinguished in the availability check process. For a block, the reference sample availability check process is performed by checking the samples of its corresponding component, where the component can be a luma component or a chroma component. The Y component is called the luma component. Both the Cb and Cr components are called the chroma components.

[0208] The difference between embodiment 2 and embodiment 1 occurs only in step 2, and in other steps, they are the same as those in embodiment 1. Step 2 of embodiment 2 is described in detail below.

[0209] Step 2: Execute a reference sample availability check process, which includes the step of checking the availability of a reference sample for the current block (1002 in FIG. 10).

[0210] In an embodiment, when the block is a luma block, the reference sample availability is derived by looking at the reference luma samples.

[0211] In an embodiment, when the block is a chroma block, the reference sample availability is derived by looking at the reference chroma samples.

[0212] 10, after checking the availability of a reference sample at 1002, the method includes determining whether there is an unavailable reference sample at 1003. If the method determines that there is an unavailable reference sample (yes at 1003), the method proceeds to 1004, otherwise the method proceeds to 1005.

[0213] It is noted here that the marking method discussed in embodiment 1 can be directly extended to embodiment 2, with the only difference being that for chroma components, chroma samples in a block area can be marked as “available” only after the Cb component block and the Cr component block have been reconstructed.

[0214] For example, for a luma block, once the block is reconstructed, the samples within the block region can be marked as "available". For a chroma block, once both the Cb and Cr blocks are reconstructed, the samples within the block region can be marked as "available". This means that when the current block is a luma block, all the luma samples in the positions covered by the block region are marked as available. When the current block is a chroma block, once both the Cb and Cr blocks are reconstructed, all the chroma samples in the positions covered by the block region are marked as available.

[0215] Other marking methods in embodiment 1 can be similarly extended.

[0216] Note that in the embodiments of the present disclosure, the "availability" information of a sample or unit is stored in memory for each component (two components in total), such as for the luma component and for the chroma component. This means that the Cb and Cr components share the same "availability" or "is available" information.

[0217] It should be noted here that even if the block is coded as an inter prediction mode, the "available" marking method can also be applied after the block is reconstructed.

[0218] The following is a description of the application of the encoding and decoding methods as illustrated in the above embodiments, and the systems that use them.

[0219] 17 is a block diagram showing a content supply system 3100 for implementing a content distribution service. The content supply system 3100 includes a capture device 3102, a terminal device 3106, and optionally a display 3126. The capture device 3102 communicates with the terminal device 3106 via a communication link 3104. The communication link may include the communication channel 13 described above. The communication link 3104 includes, but is not limited to, WIFI, Ethernet, cable, wireless (3G / 4G / 5G), USB, or any type of combination thereof, etc.

[0220] The capture device 3102 may generate data and encode the data by the encoding method as shown in the above embodiment. Alternatively, the capture device 3102 may deliver the data to a streaming server (not shown in the figure), which encodes the data and transmits the encoded data to the terminal device 3106. The capture device 3102 includes, but is not limited to, a camera, a smartphone or Pad, a computer or laptop, a video conferencing system, a PDA, an in-vehicle device, or any combination thereof. For example, the capture device 3102 may include the source device 102 as described above. When the data includes video, the video encoder 20 included in the capture device 3102 may actually perform the video encoding process. When the data includes audio (i.e., voice), the audio encoder included in the capture device 3102 may actually perform the audio encoding process. In some practical scenarios, the capture device 3102 delivers the encoded video and audio data by multiplexing them together. In other practical scenarios, for example, in a video conferencing system, the encoded audio data and the encoded video data are not multiplexed. The capture device 3102 delivers the encoded audio data and the encoded video data separately to the terminal device 3106 .

[0221] In the content delivery system 3100, the terminal device 3106 receives and plays the encoded data. The terminal device 3106 may be a device with data receiving and recovering capabilities, such as a smartphone or Pad 3108, a computer or laptop 3110, a network video recorder (NVR) / digital video recorder (DVR) 3112, a TV 3114, a set top box (STB) 3116, a video conferencing system 3118, a video surveillance system 3120, a personal digital assistant (PDA) 3122, an in-vehicle device 3124, or any combination thereof, that can decode the encoded data described above. For example, the terminal device 3106 may include the destination device 14 described above. When the encoded data includes video, the video decoder 30 included in the terminal device is prioritized to perform the video decoding. When the encoded data includes audio, the audio decoder included in the terminal device is prioritized to perform the audio decoding process.

[0222] In terminal devices with a display, such as a smartphone or Pad 3108, a computer or laptop 3110, a network video recorder (NVR) / digital video recorder (DVR) 3112, a TV 3114, a personal digital assistant (PDA) 3122, or an in-vehicle device 3124, the terminal device can provide the decoded data to its own display. In terminal devices without a display, such as a STB 3116, a video conferencing system 3118, or a video surveillance system 3120, an external display 3126 is connected to receive and display the decoded data.

[0223] When each device in the system performs encoding or decoding, it can use a picture encoding device or a picture decoding device as shown in the above embodiments.

[0224] 18 is a diagram showing an example structure of the terminal device 3106. After the terminal device 3106 receives a stream from the capture device 3102, the protocol progression unit 3202 analyzes the transmission protocol of the stream. The protocol includes, but is not limited to, Real Time Streaming Protocol (RTSP), Hyper Text Transfer Protocol (HTTP), HTTP Live streaming protocol (HLS), MPEG-DASH, Real-time Transport protocol (RTP), Real Time Messaging Protocol (RTMP), or any kind of combination thereof, etc.

[0225] After the protocol progression unit 3202 processes the stream, a stream file is generated. The file is output to the demultiplexing unit 3204. The demultiplexing unit 3204 can separate the multiplexed data into encoded audio data and encoded video data. As mentioned above, in some practical scenarios, for example in a video conferencing system, the encoded audio data and encoded video data are not multiplexed. In this situation, the encoded data is sent to the video decoder 3206 and the audio decoder 3208 without passing through the demultiplexing unit 3204.

[0226] The demultiplexing process generates a video elementary stream (ES), an audio ES, and any subtitles. The video decoder 3206, which includes the video decoder 30 as described in the above embodiment, decodes the video ES by the decoding method as shown in the above embodiment to generate video frames, and supplies this data to the synchronization unit 3212. The audio decoder 3208 decodes the audio ES to generate audio frames, and supplies this data to the synchronization unit 3212. Alternatively, the video frames may be stored in a buffer (not shown in FIG. 18) before being supplied to the synchronization unit 3212. Similarly, the audio frames may be stored in a buffer (not shown in FIG. 18) before being supplied to the synchronization unit 3212.

[0227] The synchronization unit 3212 synchronizes the video and audio frames and provides the video / audio to a video / audio display 3214. For example, the synchronization unit 3212 synchronizes the presentation of video and audio information, which may be encoded in the syntax with timestamps for the presentation of the encoded audio and visual data, and for the delivery of the data stream itself.

[0228] If subtitles are included in the stream, the subtitle decoder 3210 decodes the subtitles, synchronizes it with the video and audio frames, and provides the video / audio / subtitles to the video / audio / subtitle display 3216.

[0229] The present invention is not limited to the above-mentioned system, and either the picture encoding device or the picture decoding device in the above-mentioned embodiments can be incorporated into other systems, for example, a vehicle system.

[0230] According to the present disclosure, the methods and processes described may be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, the methods or processes may be performed by instructions or program code stored on a computer readable medium and executed by a hardware processing unit.

[0231] The proposed method stores availability information for each sample in each component, which can provide more accurate availability information in the intra prediction process.

[0232] By way of example, and not limitation, computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM, or other optical disk storage, magnetic disk storage, or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transitory media, but instead cover non-transitory tangible storage media. Disk and disc, as used herein, include compact disc (CD), laser disc, optical disk, digital versatile disc (DVD), floppy disk, and Blu-ray disc, where disks typically reproduce data magnetically and discs reproduce data optically with a laser. Combinations of the above should also be included within the scope of computer readable media.

[0233] The instructions or program code may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Accordingly, the term "processor", as used herein, may refer to any of the foregoing structures, or any other structure suitable for implementing the techniques described herein. Furthermore, in some aspects, the functionality described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding or incorporated into a combined codec. Furthermore, the techniques may be implemented entirely in one or more circuits or logic elements.

[0234] The techniques of this disclosure may be implemented in a wide variety of devices or equipment, including a wireless handset, an integrated circuit (IC), or a set of ICs (e.g., a chipset). Various components, modules, or units have been described in this disclosure to highlight functional aspects of devices configured to perform the techniques of the disclosure, but do not necessarily require accomplishment by different hardware units. Rather, as described above, the various units may be combined in a codec hardware unit, in combination with appropriate software and / or firmware, or provided by a collection of interoperable hardware units including one or more processors as described above.

[0235] Although several embodiments have been provided in this disclosure, it should be understood that the disclosed systems and methods may be embodied in many other specific forms without departing from the spirit or scope of the disclosure. The examples of the present invention should be considered illustrative and not restrictive, and are not intended to be limited to the details given herein. For example, various elements or components may be combined or integrated into another system, or certain features may be omitted or not implemented.

[0236] Additionally, the techniques, systems, subsystems, and methods described and shown in various individual or separate embodiments may be combined or integrated with other systems, modules, techniques, or methods without departing from the scope of the present disclosure. Other items shown or discussed as connected or directly connected or in communication with each other may be indirectly connected or communicate through some interface, device, or intermediate component, whether electrical, mechanical, or otherwise. Other examples of modifications, substitutions, and alterations will be ascertained by those skilled in the art and may be made without departing from the spirit and scope of the present disclosure. [Explanation of symbols]

[0237] 12 Source Device 14 Destination Device 18 Video Sources 20 Video Encoder 22 Output Interface 28 Input Interface 30 Video Decoder 32 Display device

Claims

1. 1. An apparatus for intra prediction, comprising: a memory containing instructions; one or more processors in communication with the memory, the one or more processors executing the instructions to Obtain the intra prediction mode of the current block; Derive the availability of a neighboring block of the current block, and specify whether the availability depends on a prediction mode when a variable checkPredModeY is equal to true, the neighboring block is unavailable, and the value of the prediction mode of the neighboring block is not equal to the value of the prediction mode of the current block; The device derives a prediction for the current block based on the intra-prediction mode of the current block and availability of the neighboring blocks.

2. The apparatus of claim 1 , wherein the availability of the neighboring blocks is derived for chroma components of the current block.

3. The apparatus of claim 2 , wherein the chroma components include a Cb component and a Cr component.

4. 4. The device of claim 1, wherein CuPredMode[0][xNbY][yNbY] specifies a value of a prediction mode of the neighboring block, and CuPredMode[0][xTbY][yTbY] specifies a value of a prediction mode of the current block.

5. The device according to any one of claims 1 to 4, wherein the value of the prediction mode of the neighboring blocks is equal to inter or intra.

6. 1. A method for image intra prediction performed by an encoder or decoder, comprising: obtaining an intra-prediction mode of a current block; deriving availability of a neighboring block of the current block, where the neighboring block is unavailable and the value of the prediction mode of the neighboring block is not equal to the value of the prediction mode of the current block when a variable checkPredModeY that specifies whether availability depends on a prediction mode is equal to true; deriving a prediction for the current block based on the intra-prediction mode of the current block and availability of the neighboring blocks; The method includes:

7. The method of claim 6 , wherein the availability of the neighboring blocks is derived for chroma components of the current block.

8. The method of claim 7 , wherein the chroma components include a Cb component and a Cr component.

9. A method according to any one of claims 6 to 8, wherein CuPredMode[0][xNbY][yNbY] specifies the value of the prediction mode of the neighboring blocks and CuPredMode[0][xTbY][yTbY] specifies the value of the prediction mode of the current block.

10. Method according to any one of claims 6 to 9, wherein the value of the prediction mode of the neighbouring blocks is equal to inter or intra.

11. A computer program comprising a program code for carrying out the method according to any one of claims 6 to 10.

12. A decoding device, comprising: a receiver configured to receive an encoded video bitstream; a memory coupled to the receiver and storing instructions; a processor coupled to said memory, said processor executing said instructions to cause said decoding device to perform the method of any one of claims 6 to 10; A decoding device comprising:

13. 1. An encoding device, comprising: A memory storing instructions; a processor coupled to said memory, said processor executing said instructions to cause said encoding device to perform the method of any one of claims 6 to 10; An encoding device comprising:

Images