Method and device for reference sample interpolation filtering for directional intra-frame prediction

Abstract

The present invention provides a method for intra-frame prediction processing of a coding block. The method comprises: obtaining a code stream of the coding block; obtaining a value of an intra-frame prediction mode of the coding block based on the code stream of the coding block; obtaining a value of an index of the coding block based on the code stream; obtaining a reference sample set of the coding block, wherein if the value of the index of the coding block is equal to a predefined value, the reference sample set includes other samples and samples in a reconstructed block; if the value of the index of the coding block is not equal to the predefined value, the reference sample set includes samples in the reconstructed block but does not include other samples; and obtaining predicted values ​​of samples included in the coding block based on the value of the intra-frame prediction mode of the coding block and the reference sample set.

Description

[0001] Cross-reference to related applications

[0002] This patent application claims priority from international patent application No. PCT / RU2019 / 000949 filed on December 16, 2019. The entire disclosure of the above patent application is incorporated herein by reference. Technical Field

[0003] Embodiments of the present invention generally relate to the field of image processing, and more particularly to intra-frame prediction processing. Background Art

[0004] Video coding (video encoding and decoding) is widely used in digital video applications such as broadcast digital television, video transmission over the Internet and mobile networks, real-time conversation applications such as video chat and video conferencing, DVD and Blu-ray Disc, video content acquisition and editing systems, and camcorders for security applications.

[0005] Even relatively short videos require a large amount of video data to describe them, which can create difficulties when the data is to be streamed or otherwise transmitted across communication networks with limited bandwidth capacity. Therefore, video data is typically compressed before being transmitted over modern telecommunications networks. Since memory resources may be limited, the size of the video can also be an issue when storing the video on a storage device. Video compression devices typically use software and / or hardware to encode the video data at the source side before transmitting or storing the video data, thereby reducing the amount of data required to represent the digital video image. A video decompression device then decodes the video data and receives the compressed data at the destination side. With limited network resources and the growing demand for higher video quality, there is a need for improved compression and decompression techniques that can increase the compression ratio with little impact on image quality. Summary of the Invention

[0006] Embodiments of the present application provide various devices and methods for encoding and decoding according to the independent claims.

[0007] The above and other objects are achieved by the subject matter of the independent claims. Other implementations are apparent from the dependent claims, the description and the drawings.

[0008] Particular embodiments are outlined in the accompanying independent claims, further embodiments are outlined in the dependent claims.

[0009] The present invention provides a method for intra-frame prediction processing of a coding block. The method comprises:

[0010] Obtaining a code stream of the coding block;

[0011] Obtaining a value of an intra-frame prediction mode of the coding block according to the code stream of the coding block;

[0012] Obtaining, according to the code stream, a value of the index of the coding block;

[0013] If the value of the index of the coding block is equal to a predefined value, obtaining a reference sample set of the coding block, wherein the reference sample set includes other samples and samples in the reconstructed block;

[0014] If the value of the index of the coding block is not equal to a predefined value, obtaining a reference sample set of the coding block, wherein the reference sample set includes samples in the reconstructed block but does not include other samples;

[0015] According to the value of the intra prediction mode of the coding block and the reference sample set, a prediction value of the sample included in the coding block is obtained.

[0016] In the method described above, the index of the coding block may be cIdx, and the predefined value may be 0.

[0017] In the method described above, the value of the index may represent a color component of the coding block.

[0018] In the method described above, when the value of the index of the coding block is equal to the predefined value, the value of the index may correspond to the luminance component of the coding block.

[0019] In the method described above, when the value of the index of the coding block is not equal to the predefined value, the value of the index may correspond to the chrominance component of the coding block.

[0020] In the method described above, the value of the intra prediction mode of the coding block may indicate that the coding block uses a directional intra prediction mode.

[0021] In the method described above, the number of the other samples may be obtained according to whether the selected directional mode is an integer slope mode.

[0022] In the method described above, the value of refFilterFlag may indicate whether the selected directionality mode is an integer slope mode.

[0023] In the above method, the number of other samples may be obtained according to the following formula:

[0024] When the value of the intra prediction mode, predModeIntra, is greater than or equal to 34, (refW+refIdx+Max(1,nTbW / nTbH)*refIdx+(refFilterFlag?0:1)); or

[0025] When the value of the intra prediction mode predModeIntra is less than 34, (refH+refIdx+Max(1,nTbH / nTbW)*refIdx+(refFilterFlag?0:1)).

[0026] The present invention further provides an encoder, wherein the encoder comprises a processing circuit, and the processing circuit is configured to execute the method described above.

[0027] The present invention further provides a decoder, wherein the decoder comprises a processing circuit, and the processing circuit is configured to execute the method described above.

[0028] The present invention also provides a computer program product comprising program code. When the program code is executed on a computer or a processor, the program code is used to execute the method described above.

[0029] The present invention also provides a decoder. The decoder comprises:

[0030] one or more processors;

[0031] A non-transitory computer-readable storage medium is coupled to the one or more processors and stores a program executed by the one or more processors, wherein when the program is executed by the one or more processors, the decoder is configured to perform the method described above.

[0032] The present invention also provides an encoder. The encoder includes:

[0033] one or more processors;

[0034] A non-transitory computer-readable storage medium is coupled to the one or more processors and stores a program executed by the one or more processors, wherein when the program is executed by the one or more processors, the encoder is configured to perform the method described above.

[0035] The present invention also provides a non-transitory computer-readable medium comprising program code. When the program code is executed by a computer device, the program code causes the computer device to perform the method described above.

[0036] The present invention also provides an encoder. The encoder includes:

[0037] A first acquiring unit, configured to acquire a code stream of a coding block;

[0038] A second acquiring unit, configured to acquire a value of an intra-frame prediction mode of the coding block according to the code stream of the coding block;

[0039] A third obtaining unit, configured to obtain a value of the index of the coding block according to the code stream;

[0040] The fourth acquisition unit is configured to acquire a reference sample set of the coding block, wherein:

[0041] If the value of the index of the coding block is equal to a predefined value, the reference sample set includes other samples and samples in the reconstructed block;

[0042] If the value of the index of the coding block is not equal to a predefined value, the reference sample set includes samples in the reconstructed block but does not include other samples;

[0043] A fifth obtaining unit is configured to obtain a prediction value of the sample included in the coding block according to the value of the intra prediction mode of the coding block and the reference sample set.

[0044] The present invention also provides a decoder. The decoder comprises:

[0045] A first acquiring unit, configured to acquire a code stream of a coding block;

[0046] A second acquiring unit, configured to acquire a value of an intra-frame prediction mode of the coding block according to the code stream of the coding block;

[0047] A third obtaining unit, configured to obtain a value of the index of the coding block according to the code stream;

[0048] The fourth acquisition unit is configured to acquire a reference sample set of the coding block, wherein:

[0049] If the value of the index of the coding block is equal to a predefined value, the reference sample set includes other samples and samples in the reconstructed block;

[0050] If the value of the index of the coding block is not equal to a predefined value, the reference sample set includes samples in the reconstructed block but does not include other samples;

[0051] A fifth obtaining unit is configured to obtain a prediction value of the sample included in the coding block according to the value of the intra prediction mode of the coding block and the reference sample set.

[0052] In other words, according to a first aspect, the present invention relates to a method for intra-frame prediction processing of a coding block (or unit). The method comprises: obtaining a code stream of the coding block; obtaining a value of an intra-frame prediction mode of the coding block according to the code stream of the coding block; obtaining a value of an index (e.g., cIdx) of the coding block (e.g., following the code stream of the coding block); obtaining a reference sample set of the coding block, wherein the value of the index of the coding block is equal to a predefined value (e.g., the predefined value is 0), and the reference sample set includes samples adjacent to the coding block and padding samples; and obtaining a prediction value of a sample (or predicted sample) included in the coding block according to the value of the intra-frame prediction mode of the coding block and the reference sample set.

[0053] In a possible implementation, when the value of the index of the coding block is not equal to the predefined value (for example, the predefined value is 0), the reference sample set includes samples adjacent to the coding block but does not include padding samples.

[0054] In a possible implementation, the value of the index represents a color component of the coding block.

[0055] In a possible implementation, when the value of the index of the coding block is equal to the predefined value (for example, the predefined value is 0), the value of the index corresponds to the luminance component of the coding block.

[0056] In a possible implementation, when the value of the index of the coding block is not equal to the predefined value (for example, the predefined value is 0), the value of the index corresponds to the chrominance component of the coding block.

[0057] In a possible implementation manner, the value of the intra-frame prediction mode of the coding block indicates that the coding block uses a directional intra-frame prediction mode.

[0058] According to a second aspect, the present invention relates to an encoder (20). The encoder (20) comprises a processing circuit for executing the above method embodiment.

[0059] According to a third aspect, the present invention relates to a decoder (30). The decoder (20) comprises a processing circuit for executing the above method embodiment.

[0060] According to a fourth aspect, the present invention relates to a computer program product comprising program code, which, when executed on a computer or a processor, is configured to perform the above method embodiments.

[0061] According to a fifth aspect, the present invention relates to a decoder. The decoder comprises:

[0062] One or more processors; a non-transitory computer-readable storage medium coupled to the one or more processors and storing a program executed by the one or more processors, wherein when the program is executed by the one or more processors, the decoder is used to perform the above method embodiment.

[0063] According to a sixth aspect, the present invention relates to an encoder. The encoder comprises:

[0064] One or more processors; a non-transitory computer-readable storage medium coupled to the one or more processors and storing a program executed by the one or more processors, wherein when the program is executed by the one or more processors, the encoder is used to perform the above method embodiment.

[0065] According to a seventh aspect, the present invention relates to a non-transitory computer-readable medium comprising program code, which, when executed by a computer device, causes the computer device to perform the above-mentioned method embodiment.

[0066] The following drawings and description will describe one or more embodiments in detail. Other features, objects, and advantages will be apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] The embodiments of the present invention are described in more detail below with reference to the accompanying drawings.

[0068] Figure 1A FIG. 1 is a block diagram of an example of a video decoding system for implementing an embodiment of the present invention.

[0069] Figure 1B FIG. 4 is a block diagram of another example of a video decoding system for implementing an embodiment of the present invention.

[0070] Figure 2 A block diagram of an example of a video encoder for implementing an embodiment of the present invention.

[0071] Figure 3 The block diagram is an exemplary structure of a video decoder for implementing an embodiment of the present invention.

[0072] Figure 4 A block diagram of an example of an encoding device or a decoding device.

[0073] Figure 5 A block diagram of another example of an encoding device or a decoding device.

[0074] Figure 6 93 prediction directions are shown, where the dashed directions are associated with wide-angle mode applied to non-square blocks.

[0075] Figure 7A The padding process is shown when the upper adjacent row is used as the main reference side to predict the block.

[0076] Figure 7B The padding process is shown when the upper adjacent row is used as the main reference side to predict the block.

[0077] Figure 8A The padding process is shown when the left adjacent column is used as the main reference side to predict the block.

[0078] Figure 8B The padding process is shown when the left adjacent column is used as the main reference side to predict the block.

[0079] Figure 9 The reference sample interpolation process in intra prediction using a linear (2-tap) filter (the upper adjacent row is used as the main reference side to predict the block) is shown.

[0080] Figure 10 The reference sample interpolation process in intra prediction using a linear (2-tap) filter (the left adjacent column is used as the main reference side to predict the block) is shown.

[0081] Figure 11 A block diagram of an exemplary structure of a content provision system 3100 for implementing content distribution services.

[0082] Figure 12 A block diagram of an exemplary structure of a terminal device.

[0083] Figure 13 A flowchart of a method for intra-frame prediction processing of a coding block is provided according to an embodiment of the present invention.

[0084] Figure 14 An encoder provided by one embodiment of the present invention is shown.

[0085] Figure 15 A decoder provided by an embodiment of the present invention is shown.

[0086] In the following, identical reference numerals refer to identical features or at least functionally equivalent features, unless explicitly stated otherwise. DETAILED DESCRIPTION

[0087] In the following description, reference is made to the accompanying drawings which form a part of the present invention, which illustrate, by way of illustration, specific aspects of embodiments of the invention or specific aspects in which embodiments of the invention may be used. It should be understood that embodiments of the invention may be used in other aspects and may include structural or logical changes not depicted in the accompanying drawings. Therefore, the following detailed description is not to be construed in a limiting sense, and the scope of the invention is defined by the appended claims.

[0088] For example, it should be understood that the disclosure relating to describing a method may also apply to a corresponding device or system for performing the method, and vice versa. For example, if one or more specific method steps are described, the corresponding device may include one or more units (e.g., functional units) to perform the described one or more method steps (e.g., one unit performs one or more steps, or multiple units perform one or more of the multiple steps, respectively), even if such one or more units are not explicitly described or shown in the accompanying drawings. On the other hand, for example, if a specific device is described based on one or more units (e.g., functional units), the corresponding method may include a step to perform the function of one or more units (e.g., one step performs the function of one or more units, or multiple steps perform the function of one or more of the multiple units, respectively), even if such one or more steps are not explicitly described or shown in the accompanying drawings. In addition, it should be understood that, unless otherwise expressly stated, the features of the various exemplary embodiments and / or aspects described herein may be combined with each other.

[0089] Video decoding (coding) generally refers to the processing of a sequence of images to form a video or video sequence. In the field of video coding, the terms "frame" and "picture / image" can be used as synonyms. Video decoding (or generally referred to as decoding) includes two parts: video encoding and video decoding. Video encoding is performed on the source side and generally includes processing (for example, by compression) the original video image to reduce the amount of data required to represent the video image (thereby more efficiently storing and / or transmitting). Video decoding is performed on the destination side and generally includes inverse processing relative to the encoder to reconstruct the video image. The "decoding" of the video image (or generally referred to as the image) involved in the embodiment should be understood as the "encoding" or "decoding" of the video image or the corresponding video sequence. The encoding part and the decoding part are also collectively referred to as codec (encoding and decoding).

[0090] In the case of lossless video coding, the original video image can be reconstructed, that is, the reconstructed video image has the same quality as the original video image (assuming no transmission loss or other data loss during storage or transmission). In the case of lossy video coding, further compression is performed through quantization, etc. to reduce the amount of data representing the video image, but the decoder side cannot fully reconstruct the video image, that is, the quality of the reconstructed video image is lower or worse than the quality of the original video image.

[0091] Several video coding standards belong to the group of "lossy hybrid video codecs" (i.e., combining spatial and temporal prediction in the sample domain with 2D transform coding in the transform domain for applying quantization). Each image in a video sequence is typically partitioned into a set of non-overlapping blocks, and decoding is typically performed at the block level. In other words, the encoder typically processes the video at the block (video block) level, i.e., encoding it, for example, by generating a prediction block through spatial (intra-frame) and / or temporal (inter-frame) prediction, subtracting the prediction block from the current block (currently processed / to-be-processed) to obtain a residual block, transforming and quantizing the residual block in the transform domain to reduce the amount of data to be transmitted (compressed), while the decoder applies the inverse of the encoder's processing to the coded or compressed block to reconstruct the current block for representation. Furthermore, the encoder replicates the decoder processing loop, so that the encoder and decoder generate the same prediction (e.g., intra-frame and inter-frame prediction) and / or reconstruction for processing (i.e., decoding) subsequent blocks.

[0092] In the following embodiment of the video decoding system 10, the video encoder 20 and the video decoder 30 are based on Figures 1A to 3 Provide a description.

[0093] Figure 1A FIG1 is a schematic block diagram of an exemplary decoding system 10, such as a video decoding system 10 (or simply, decoding system 10) that may utilize the techniques of the present application. The video encoder 20 (or simply, encoder 20) and the video decoder 30 (or simply, decoder 30) in the video decoding system 10 are two examples of devices that may be used to perform various techniques according to various examples described in this application.

[0094] like Figure 1A As shown, the decoding system 10 includes a source device 12 for providing encoded image data 21 to a destination device 14 or the like for decoding the encoded image data 13 .

[0095] The source device 12 includes an encoder 20 and may additionally (ie, optionally) include an image source 16 , a pre-processor (or pre-processing unit) 18 (eg, image pre-processor 18 ), and a communication interface or communication unit 22 .

[0096] Image source 16 may include or may be any type of image capture device such as a camera for capturing real-world images, and / or any type of image generation device such as a computer graphics processor for generating computer-animated images, or any other device for acquiring and / or providing real-world images, computer-generated images (e.g., screen content, virtual reality (VR) images), and / or any combination thereof (e.g., augmented reality (AR) images). The image source may be any type of memory (memory / storage) for storing any of the above images.

[0097] In order to distinguish the pre-processor 18 and the processing performed by the pre-processing unit 18 , the image or image data 17 may also be referred to as a raw image or raw image data 17 .

[0098] The preprocessor 18 is configured to receive (raw) image data 17 and perform preprocessing on the image data 17 to obtain a preprocessed image 19 or preprocessed image data 19. The preprocessing performed by the preprocessor 18 may include trimming, color format conversion (e.g., from RGB to YCbCr), color grading, or denoising. It will be appreciated that the preprocessing unit 18 may be an optional component.

[0099] The video encoder 20 is configured to receive the pre-processed image data 19 and provide encoded image data 21 (hereinafter referred to as Figure 2 etc. provide more detailed description).

[0100] The communication interface 22 in the source device 12 can be used to receive the encoded image data 21 and send the encoded image data 21 (or data obtained after further processing of the encoded image data 21) to another device, such as the destination device 14 or any other device, through the communication channel 13 for storage or direct reconstruction.

[0101] Destination device 14 includes a decoder 30 (eg, video decoder 30 ), and may additionally (ie, optionally) include a communication interface or communication unit 28 , a post-processor 32 (or post-processing unit 32 ), and a display device 34 .

[0102] The communication interface 28 in the destination device 14 is used to receive the encoded image data 21 (or data obtained by further processing the encoded image data 21) directly from the source device 12 or from any other source such as a storage device (for example, an encoded image data storage device), and provide the encoded image data 21 to the decoder 30.

[0103] Communication interface 22 and communication interface 28 can be used to send or receive encoded image data 21 or encoded data 13 via a direct communication link between source device 12 and destination device 14 (e.g., a direct wired or wireless connection) or via any type of network (e.g., a wired network, a wireless network, or any combination thereof, or any type of private and public network, or any combination thereof).

[0104] For example, the communication interface 22 may be used to encapsulate the encoded image data 21 into a suitable format (eg, data packets) and / or process the encoded image data through any type of transmission encoding or processing for transmission over a communication link or network.

[0105] For example, the communication interface 28 corresponding to the communication interface 22 may be configured to receive transmission data and process the transmission data using any type of corresponding transmission decoding or processing manner and / or decapsulation manner to obtain the encoded image data 21 .

[0106] Both the communication interface 22 and the communication interface 28 can be configured as Figure 1A The unidirectional communication interface indicated by the arrow of the communication channel 13 pointing from the source device 12 to the destination device 14, or configured as a bidirectional communication interface, can be used to send and receive messages, etc. to establish a connection, confirm and exchange any other information related to the communication link and / or data transmission (such as encoded image data transmission), etc.

[0107] The decoder 30 is configured to receive the encoded image data 21 and provide decoded image data 31 or decoded image 31 (hereinafter referred to as Figure 3 or Figure 5 etc. provide more detailed description).

[0108] The post-processor 32 in the destination device 14 is configured to post-process the decoded image data 31 (also referred to as reconstructed image data) (e.g., the decoded image 31) to obtain post-processed image data 33 (e.g., the post-processed image 33). The post-processing performed by the post-processing unit 32 may include color format conversion (e.g., from YCbCr to RGB), color grading, cropping, resampling, or any other processing to provide the decoded image data 31 for display by the display device 34, etc.

[0109] A display device 34 in the destination device 14 is configured to receive the post-processed image data 33 for displaying the image to a user, viewer, or the like. The display device 34 may be or include any type of display for displaying the reconstructed image, such as an integrated or external display or screen. For example, the display may include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro-LED display, a liquid crystal on silicon (LCoS) display, a digital light processor (DLP), or any other type of display.

[0110] Although Figure 1A Source device 12 and destination device 14 are shown as separate devices, but device embodiments may also include both devices or the functionality of both devices, namely, source device 12 or the corresponding functionality and destination device 14 or the corresponding functionality. In these embodiments, source device 12 or the corresponding functionality and destination device 14 or the corresponding functionality may provide the same hardware and / or software or be implemented by separate hardware and / or software or any combination thereof.

[0111] It is obvious to those skilled in the art based on the description that Figure 1A The presence and (precise) functional division of different units or functions within the source device 12 and / or destination device 14 shown may vary depending on the actual device and application.

[0112] The encoder 20 (eg, video encoder 20) or the decoder 30 (eg, video decoder 30) or both the encoder 20 and the decoder 30 may be configured to generate a video signal. Figure 1B The processing circuitry may include one or more microprocessors, one or more digital signal processors (DSPs), one or more application-specific integrated circuits (ASICs), one or more field-programmable gate arrays (FPGAs), one or more discrete logics, one or more hardware, one or more video decoding dedicated processors, or any combination thereof. The encoder 20 may be implemented by the processing circuitry 46 to include reference to Figure 2 The various modules described in the encoder 20 and / or any other encoder system or subsystem described herein. The decoder 30 may be implemented by the processing circuit 46 to include reference to Figure 3 The various modules described in the decoder 30 in FIG. 1 and / or any other decoder system or subsystem described herein. The processing circuitry may be used to perform the various operations discussed below. Figure 5 As shown, if the above technology is partially implemented in software, a device can store the instructions of the software in a suitable non-transitory computer-readable storage medium, and can execute these instructions in hardware through one or more processors to perform the technology of the present invention. The video encoder 20 or the video decoder 30 can be integrated into a single device as part of a combined encoder / decoder (codec), for example, as shown in FIG. Figure 1B shown.

[0113] Source device 12 and destination device 14 may include any of a variety of devices, including any type of handheld or fixed device, such as a notebook or laptop computer, a mobile phone, a smartphone, a tablet or tablet computer, a camera, a desktop computer, a set-top box, a television, a display device, a digital media player, a video game console, a video streaming device (such as a content service server or content distribution server), a broadcast receiver device, a broadcast transmitter device, etc., and may not use or use any type of operating system. In some cases, source device 12 and destination device 14 can be configured for wireless communication. Therefore, source device 12 and destination device 14 can be wireless communication devices.

[0114] In some cases, Figure 1A The video decoding system 10 shown is merely an example, and the techniques of this application can be applied to video decoding arrangements (e.g., video encoding or video decoding) that do not necessarily include any data communication between the encoding device and the decoding device. In other examples, data is retrieved from local memory, streamed over a network, etc. The video encoding device can encode the data and store the data in memory, and / or the video decoding device can retrieve the data from memory and decode the data. In some examples, encoding and decoding are performed by devices that do not communicate with each other but only encode data to memory and / or retrieve data from memory and decode the data.

[0115] For ease of description, this document (for example) refers to High-Efficiency Video Coding (HEVC) or the next-generation video coding standard, Versatile Video Coding (VVC), developed by the Joint Collaboration Team on Video Coding (JCT-VC) of the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Motion Picture Experts Group (MPEG) to describe embodiments of the present invention. Those skilled in the art will appreciate that embodiments of the present invention are not limited to HEVC or VVC.

[0116] Encoders and encoding methods

[0117] Figure 2 FIG. 2 is a schematic block diagram of an exemplary video encoder 20 for implementing the techniques of the present application. Figure 2 In the example of FIG, the video encoder 20 includes an input terminal 201 (or input interface 201), a residual calculation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, a reconstruction unit 214, a loop filter unit 220, a decoded picture buffer (DPB) 230, a mode selection unit 260, an entropy coding unit 270, and an output terminal 272 (or output interface 272). The mode selection unit 260 may include an inter-frame prediction unit 244, an intra-frame prediction unit 254, and a segmentation unit 262. The inter-frame prediction unit 244 may include a motion estimation unit and a motion compensation unit (not shown). Figure 2 The illustrated video encoder 20 may also be referred to as a hybrid video encoder or a hybrid video codec-based video encoder.

[0118] The residual calculation unit 204, the transform processing unit 206, the quantization unit 208, and the mode selection unit 260 may constitute a forward signal path of the encoder 20, while the inverse quantization unit 210, the inverse transform processing unit 212, the reconstruction unit 214, the buffer 216, the loop filter 220, the decoded picture buffer (DPB) 230, the inter-frame prediction unit 244, and the intra-frame prediction unit 254 may constitute a backward signal path of the video encoder 20, wherein the backward signal path of the video encoder 20 corresponds to the decoder (see Figure 3The inverse quantization unit 210, the inverse transform processing unit 212, the reconstruction unit 214, the loop filter 220, the decoded picture buffer (DPB) 230, the inter-frame prediction unit 244, and the intra-frame prediction unit 254 also constitute the "built-in decoder" of the video encoder 20.

[0119] Images and image segmentation (images and blocks)

[0120] Encoder 20 can be configured to receive an image 17 (or image data 17) via input 201 or the like. Image 17 can be an image from a sequence of images comprising a video or video sequence. The received image or image data can also be a pre-processed image 19 (or pre-processed image data 19). For simplicity, the following description uses image 17. Image 17 can also be referred to as a current image or a picture to be decoded (particularly in video decoding to distinguish the current image from other images (e.g., previously encoded and / or decoded images) in the same video sequence (i.e., a video sequence that also includes the current image)).

[0121] A (digital) image is, or can be considered to be, a two-dimensional array or matrix consisting of samples with intensity values. The samples in the array can also be called pixels (or pels), short for picture elements. The number of samples in the array or image in the horizontal and vertical directions (or axes) defines the image size and / or resolution. To represent color, three color components are typically used; that is, an image can be represented as or include three arrays of samples. In the RGB format or color space, an image consists of corresponding arrays of red, green, and blue samples. However, in video decoding, each pixel is typically represented in a luminance and chrominance format or color space, such as YCbCr, consisting of a luminance component represented by Y (sometimes also represented by L) and two chrominance components represented by Cb and Cr. The luminance (luma) component Y represents brightness or grayscale intensity (for example, in grayscale images, both are the same), while the two chrominance (chroma) components Cb and Cr represent chrominance or color information components. Thus, an image in YCbCr format includes a luma sample array consisting of luma sample values ​​(Y) and two chroma sample arrays consisting of chroma values ​​(Cb and Cr). An image in RGB format can be converted or transformed into YCbCr format, and vice versa. This process is also called color conversion or transformation. If the image is black and white, the image can include only the luma sample array. Accordingly, for example, the image can be a luma sample array in black and white format or a luma sample array and two corresponding chroma sample arrays in 4:2:0, 4:2:2, and 4:4:4 color formats.

[0122] An embodiment of the video encoder 20 may include an image segmentation unit ( Figure 2 ), is used to partition the picture 17 into a plurality of (typically non-overlapping) picture blocks 203. These blocks may also be referred to as root blocks, macroblocks (in H.264 / AVC), or coding tree blocks (CTBs) or coding tree units (CTUs) (in H.265 / HEVC and VVC). The picture partitioning unit may be used to use the same block size for all pictures in a video sequence and a corresponding grid of defined block sizes, or to vary the block size between pictures or a subset or group of pictures and to partition each picture into a plurality of corresponding blocks.

[0123] In other embodiments, the video encoder may be configured to directly receive the block 203 in the image 17, for example, one, several or all blocks constituting the image 17. The image block 203 may also be referred to as a current image block or an image block to be decoded.

[0124] Similar to image 17, image block 203 is also, or can be considered to be, a two-dimensional array or matrix of samples having intensity values ​​(sample values), but the size of image block 203 is smaller than that of image 17. In other words, depending on the color format used, block 203 may include, for example, one sample array (e.g., a luma array in the case of black and white image 17, or a luma array or chroma array in the case of a color image), three sample arrays (e.g., one luma array and two chroma arrays in the case of a color image 17), or any other number and / or type of arrays. The number of samples in block 203 in the horizontal and vertical directions (or axes) defines the size of block 203. Accordingly, a block may be an M×N (M columns×N rows) sample array, an M×N array of transform coefficients, and so on.

[0125] Figure 2 The illustrated embodiment of the video encoder 20 may be used to encode the image 17 block by block, eg, encoding and prediction are performed per block 203 .

[0126] Figure 2 The illustrated embodiment of the video encoder 20 may also be used to partition and / or encode a picture by slices (also referred to as video slices). A picture may be partitioned into or encoded by one or more (usually non-overlapping) slices, each of which may include one or more blocks (e.g., CTUs).

[0127] Figure 2The illustrated embodiment of the video encoder 20 may also be used to segment and / or encode an image by tile groups (also referred to as video tile groups) and / or blocks (also referred to as video blocks). An image may be segmented into or encoded by one or more (usually non-overlapping) tile groups; each tile group may include one or more blocks (e.g., CTUs) or one or more tiles, etc.; each tile may be rectangular, etc., and may include one or more complete or partial blocks (e.g., CTUs), etc.

[0128] Residual calculation

[0129] The residual calculation unit 204 can be used to calculate the residual block 205 (also referred to as residual 205) based on the image block 203 and the prediction block 265 (a more detailed description of the prediction block 265 is provided later) in the following manner to obtain the residual block 205 in the sample domain: for example, subtracting the sample value of the prediction block 265 from the sample value of the image block 203 on a sample-by-sample (pixel-by-pixel) basis.

[0130] Transform

[0131] The transform processing unit 206 may be configured to apply a transform such as a discrete cosine transform (DCT) or a discrete sine transform (DST) to the sample values ​​of the residual block 205 to obtain transform coefficients 207 in the transform domain. The transform coefficients 207 may also be referred to as transform residual coefficients and represent the residual block 205 in the transform domain.

[0132] The transform processing unit 206 can be used to apply an integer approximation of DCT / DST (e.g., a transform specified for H.265 / HEVC). Compared to an orthogonal DCT transform, this integer approximation is typically scaled by a certain factor. In order to maintain the norm of the residual block processed by the forward transform and the inverse transform, other scaling factors are applied as part of the transform process. The scaling factor is typically selected based on certain constraints, such as whether the scaling factor is a power of 2 for the shift operation, the bit depth of the transform coefficients, a trade-off between accuracy and implementation cost, etc. For example, a specific scaling factor is specified for the inverse transform through the inverse transform processing unit 212, etc. (and the video decoder 30 specifies the corresponding inverse transform through the inverse transform processing unit 312, etc.); accordingly, the encoder 20 can specify a corresponding scaling factor for the forward transform through the transform processing unit 206, etc.

[0133] Embodiments of the video encoder 20 (and accordingly, the transform processing unit 206) may be configured to output transform parameters (e.g., one or more transform types) directly or after encoding or compression through the entropy coding unit 270, such that the video decoder 30 may receive and use the transform parameters for decoding, and so on.

[0134] Quantification

[0135] The quantization unit 208 may be used to quantize the transform coefficients 207 by applying scalar quantization or vector quantization, etc., to obtain quantized coefficients 209 . The quantized coefficients 209 may also be referred to as quantized transform coefficients 209 or quantized residual coefficients 209 .

[0136] The quantization process can reduce the bit depth associated with some or all of the transform coefficients 207. For example, an n-bit transform coefficient can be rounded down to an m-bit transform coefficient during quantization, where n is greater than m. The degree of quantization can be modified by adjusting a quantization parameter (QP). For example, for scalar quantization, varying degrees of scaling can be performed to achieve finer or coarser quantization. A smaller quantization step size corresponds to finer quantization, while a larger quantization step size corresponds to coarser quantization. A suitable quantization step size can be represented by a quantization parameter (QP). For example, a quantization parameter can be an index into a predefined set of suitable quantization step sizes. For example, a smaller quantization parameter can correspond to fine quantization (a smaller quantization step size), while a larger quantization parameter can correspond to coarse quantization (a larger quantization step size), and vice versa. Quantization can include division by the quantization step size, while corresponding and / or inverse dequantization performed by the inverse quantization unit 210, etc., can include multiplication by the quantization step size. Embodiments according to some standards, such as HEVC, can be configured to use a quantization parameter to determine the quantization step size. Generally, the quantization step size can be calculated based on the quantization parameter using a fixed-point approximation of an equation involving division. Additional scaling factors can be introduced for quantization and inverse quantization to restore the norm of the residual block that may have been modified by the scaling used in the fixed-point approximation of the equations for the quantization step size and the quantization parameter. In one exemplary implementation, the scaling of the inverse transform and the inverse quantization can be combined. Alternatively, a custom quantization table can be used and signaled from the encoder to the decoder in the bitstream, etc. Quantization is a lossy operation, where the larger the quantization step size, the greater the loss.

[0137] Embodiments of the video encoder 20 (correspondingly, the quantization unit 208 ) may be configured to output a quantization parameter (QP) directly or after encoding via the entropy coding unit 270 , such that the video decoder 30 may receive and use the quantization parameter for decoding, and so on.

[0138] Dequantization

[0139] The inverse quantization unit 210 is configured to apply the inverse quantization of the quantization unit 208 to the quantized coefficients to obtain dequantized coefficients 211 by, for example, applying an inverse quantization scheme that is the opposite of the quantization scheme applied by the quantization unit 208, based on or using the same quantization step size as the quantization unit 208. The dequantized coefficients 211 may also be referred to as dequantized residual coefficients 211, which correspond to the transform coefficients 207 but are typically different from the transform coefficients due to losses caused by quantization.

[0140] Inverse transform

[0141] The inverse transform processing unit 212 is configured to apply an inverse transform, such as an inverse discrete cosine transform (DCT) or an inverse discrete sine transform (DST), or another inverse transform, to the transform processing unit 206 to obtain a reconstructed residual block 213 (or corresponding dequantized coefficients 213) in the sample domain. The reconstructed residual block 213 may also be referred to as a transform block 213.

[0142] reconstruction

[0143] The reconstruction unit 214 (e.g., an adder or summer 214) is used to add the transform block 213 (i.e., the reconstructed residual block 213) to the prediction block 265 in the following manner to obtain the reconstructed block 215 in the sample domain: for example, the sample values ​​of the reconstructed residual block 213 are added to the sample values ​​of the prediction block 265 sample by sample.

[0144] Filtering

[0145] The loop filter unit 220 (or simply "loop filter" 220) is used to filter the reconstructed block 215 to obtain a filtered block 221, or generally to filter the reconstructed samples to obtain filtered samples. For example, the loop filter unit is used to smoothly perform pixel transitions or otherwise improve video quality. The loop filter unit 220 may include one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or one or more other filters, such as a bilateral filter, an adaptive loop filter (ALF), a sharpening or smoothing filter, a collaborative filter, or any combination thereof. Although the loop filter unit 220 is in Figure 2 2. The loop filter unit 220 is shown as an in-loop filter, but in other configurations, the loop filter unit 220 may be implemented as a post-loop filter. The filtered block 221 may also be referred to as a filtered reconstructed block 221.

[0146] An embodiment of the video encoder 20 (correspondingly, the loop filter unit 220) can be used to directly output or output loop filter parameters (e.g., sample adaptive offset information) after encoding through the entropy coding unit 270, so that the decoder 30 can receive and use the same loop filter parameters or the corresponding loop filter for decoding, etc.

[0147] Decoded image buffer

[0148] The decoded picture buffer (DPB) 230 may be a memory for storing reference images, or generally for storing reference image data, used by the video encoder 20 to encode video data. The DPB 230 may be formed from any of a variety of memory devices, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. The decoded picture buffer (DPB) 230 may be used to store one or more filtered blocks 221. The decoded picture buffer 230 may also be used to store other previously filtered blocks (e.g., previously filtered reconstructed blocks 221) from the same current picture or a different picture (e.g., a previously reconstructed picture), and may provide previously fully reconstructed (i.e., decoded) pictures (and corresponding reference blocks and samples) and / or partially reconstructed current pictures (and corresponding reference blocks and samples) for inter-frame prediction, etc. The decoded picture buffer (DPB) 230 may also be used to store one or more unfiltered reconstructed blocks 215 or generally unfiltered reconstructed samples if the reconstructed block 215 has not been filtered by the loop filter unit 220, etc., or to store any other blocks or samples resulting from further processing of the reconstructed blocks or samples.

[0149] Mode selection (segmentation and prediction)

[0150] The mode selection unit 260 includes a segmentation unit 262, an inter-frame prediction unit 244, and an intra-frame prediction unit 254, and is configured to receive or obtain original image data such as the original block 203 (the current block 203 in the current image 17) and reconstructed image data (e.g., filtered and / or unfiltered reconstructed samples or blocks in the same (current) image and / or one or more previously decoded images) from the decoded image buffer 230 or other buffers (e.g., line buffers, not shown). The reconstructed image data is used as reference image data for prediction such as inter-frame prediction or intra-frame prediction to obtain a prediction block 265 or a prediction value 265.

[0151] The mode selection unit 260 can be used to determine or select a partitioning method for the current block prediction mode (including non-partitioning) and determine or select a prediction mode (such as intra-frame or inter-frame prediction mode), and generate a corresponding prediction block 265, which is used to calculate the residual block 205 and reconstruct the reconstructed block 215.

[0152] Embodiments of the mode selection unit 260 may be configured to select a partitioning scheme and a prediction mode (e.g., from those supported or available by the mode selection unit 260). The partitioning scheme and the prediction mode provide the best match or minimum residual (minimum residual means better compression in transmission or storage), or provide minimum signaling overhead (minimum signaling overhead means better compression in transmission or storage), or consider or balance both of the above. The mode selection unit 260 may be configured to determine the partitioning scheme and the prediction mode based on rate distortion optimization (RDO), i.e., select the prediction mode that provides the minimum rate distortion. Terms such as "best," "minimum," and "optimal" herein do not necessarily mean "best," "minimum," "optimal," etc. in general, but may also refer to situations where termination or selection criteria are met, e.g., values ​​exceeding or falling below a threshold or other constraints may result in a "suboptimal selection," but may reduce complexity and processing time.

[0153] In other words, the segmentation unit 262 can be used to segment the block 203 into smaller block partitions or sub-blocks (forming blocks again) in the following manner: for example, iteratively using quadtree (quad-tree, QT) segmentation, binary tree (binary-tree, BT) segmentation or triple-tree (triple-tree, TT) segmentation or any combination thereof; and to perform prediction on each block partition or sub-block, etc., wherein the mode selection includes selecting a tree structure of the partitioned block 203, and the prediction mode is applied to each block partition or sub-block.

[0154] The segmentation (eg, performed by segmentation unit 260) and prediction processes (performed by inter-prediction unit 244 and intra-prediction unit 254) performed by exemplary video encoder 20 are described in detail below.

[0155] segmentation

[0156] The segmentation unit 262 can segment (or divide) the current block 203 into smaller partitions, such as square or rectangular blocks. These blocks (also referred to as sub-blocks) can be further segmented into even smaller partitions. This is also called tree segmentation or hierarchical tree segmentation. A root block at root tree level 0 (hierarchical level 0, depth 0) can be recursively segmented into two or more blocks at the next lower tree level, such as nodes at tree level 1 (hierarchical level 1, depth 1). These blocks can be further segmented into two or more blocks at the next lower level, such as tree level 2 (hierarchical level 2, depth 2), and so on, until the segmentation is completed (because the end criteria are met, such as reaching the maximum tree depth or minimum block size). Blocks that are not further segmented are also called leaf blocks or leaf nodes of the tree. A tree segmented into two partitions is called a binary tree (BT), a tree segmented into three partitions is called a ternary tree (TT), and a tree segmented into four partitions is called a quadtree (QT).

[0157] As described above, the term "block" used herein may be a portion of an image, in particular a square or rectangular portion. With reference to HEVC, VVC, etc., a block may be or may correspond to a coding tree unit (CTU), a coding unit (CU), a prediction unit (PU), and a transform unit (TU), and / or may correspond to multiple corresponding blocks, for example, a coding tree block (CTB), a coding block (CB), a transform block (TB), or a prediction block (PB).

[0158] For example, a coding tree unit (CTU) may be or may include one CTB consisting of luma samples from an image with three sample arrays and two corresponding CTBs consisting of chroma samples from the image, or may be or may include one CTB consisting of samples from a monochrome image or an image coded using three separate color planes and syntax structures. These syntax structures are used to decode the above samples. Correspondingly, a coding tree block (CTB) may be an N×N block of samples, where N may be set to a value such that a component is divided into multiple CTBs, which is a partitioning method. A coding unit (CU) may be or may include one coding block consisting of luma samples from an image with three sample arrays and two corresponding coding blocks consisting of chroma samples from the image, or may be or may include one coding block consisting of samples from a monochrome image or an image coded using three separate color planes and syntax structures. These syntax structures are used to decode the above samples. Correspondingly, a coding block (CB) may be an M×N sample block, where M and N may be set to certain values ​​such that one CTB is divided into multiple coding blocks. This is a partitioning method.

[0159] In an embodiment, for example according to HEVC, a coding tree unit (CTU) may be divided into multiple CUs by a quadtree structure represented as a coding tree. Whether to use inter-frame (temporal) prediction or intra-frame (spatial) prediction to decode an image area is determined at the CU level. Each CU may be further divided into 1, 2, or 4 PUs according to the PU partition type. The same prediction process is performed within a PU, and relevant information is sent to the decoder in units of PUs. After obtaining the residual block by performing the prediction process according to the PU partition type, the CU may be divided into transform units (TUs) according to other quadtree structures similar to the coding tree of the CU.

[0160] In an embodiment, for example, according to the latest video coding standard currently under development called Versatile Video Coding (VVC), a combined quad-tree and binary tree (QTBT) segmentation is used to segment coding blocks, etc. In the QTBT block structure, a CU can be square or rectangular. For example, a coding tree unit (CTU) is first segmented by a quadtree structure. The quadtree leaf nodes are further segmented by a binary tree or a ternary / triple tree structure. The segmented leaf nodes are called coding units (CUs), and this segmentation is used for prediction and transform processing without any further segmentation. This means that CU, PU, ​​and TU have the same block size in the QTBT coding block structure. At the same time, multiple segmentations such as ternary tree segmentation can be used with the QTBT block structure.

[0161] In one example, mode select unit 260 in video encoder 20 may be used to perform any combination of the segmentation techniques described herein.

[0162] As described above, the video encoder 20 is configured to determine or select the best or optimal prediction mode from a (eg, predetermined) prediction mode set, which may include intra-frame prediction mode and / or inter-frame prediction mode.

[0163] Intra-frame prediction

[0164] The intra-frame prediction mode set may include 35 different intra-frame prediction modes, for example, non-directional modes or directional modes such as the DC (or mean) mode and the planar mode defined in HEVC, etc., or may include 67 different intra-frame prediction modes, for example, non-directional modes or directional modes such as the DC (or mean) mode and the planar mode defined in VVC, etc.

[0165] The intra prediction unit 254 is configured to generate an intra prediction block 265 using reconstructed samples of neighboring blocks in the same current image according to an intra prediction mode in the intra prediction mode set.

[0166] The intra-frame prediction unit 254 (or generally referred to as the mode selection unit 260) is also used to output the intra-frame prediction parameters (or generally referred to as information representing the selected intra-frame prediction mode of the block) to the entropy coding unit 270 in the form of syntax elements 266 to include the intra-frame prediction parameters in the encoded image data 21, so that the video decoder 30 can receive and use the prediction parameters for decoding, etc.

[0167] Inter-frame prediction

[0168] The set of (possible) inter-frame prediction modes depends on the available reference image (i.e. at least part of the decoded image stored in the DPB 230 or the like as mentioned above) and other inter-frame prediction parameters, e.g. on whether the entire reference image or only a part of the reference image (e.g. a search window area around the area of ​​the current block) is used to search for the best matching reference block, and / or e.g. on whether pixel interpolation is performed, e.g. half-pixel / half-pixel interpolation and / or quarter-pixel interpolation.

[0169] In addition to the above-mentioned prediction modes, skip mode and / or direct mode may also be used.

[0170] The inter-frame prediction unit 244 may include a motion estimation (ME) unit and a motion compensation (MC) unit (both in Figure 2 (not shown in the figure). The motion estimation unit can be used to receive or obtain the image block 203 (the current image block 203 in the current image 17) and the decoded image 231, or at least one or more previously reconstructed blocks (e.g., reconstructed blocks in one or more other / different previously decoded images 231) for motion estimation. For example, a video sequence may include the current image and the previously decoded image 231, or in other words, the current image and the previously decoded image 231 may be part of or constitute an image sequence, which image sequence constitutes the video sequence.

[0171] For example, the encoder 20 may be configured to select a reference block from a plurality of reference blocks of the same or different images in a plurality of other images, and provide the reference image (or reference image index) and / or the offset (spatial offset) between the position (x-coordinate, y-coordinate) of the reference block and the position of the current block as an inter-frame prediction parameter to the motion estimation unit. This offset is also referred to as a motion vector (MV).

[0172] The motion compensation unit is configured to obtain (e.g., receive) inter-frame prediction parameters and perform inter-frame prediction based on or using the inter-frame prediction parameters to obtain an inter-frame prediction block 265. The motion compensation performed by the motion compensation unit may include extracting or generating a prediction block based on a motion / block vector determined by motion estimation, and may also include performing interpolation to achieve sub-pixel accuracy. Interpolation filtering can generate additional pixel samples based on known pixel samples, thereby potentially increasing the number of candidate prediction blocks that can be used to decode the image block. Upon receiving a motion vector corresponding to a PU of the current image block, the motion compensation unit may locate the prediction block pointed to by the motion vector in one of the reference picture lists.

[0173] The motion compensation unit may also generate syntax elements associated with blocks and video slices for use by video decoder 30 when decoding image blocks in the video slices. In addition to or as an alternative to slices and corresponding syntax elements, partition groups and / or partitions and corresponding syntax elements may be generated or used.

[0174] Entropy Coding

[0175] The entropy coding unit 270 is configured to apply or not apply an entropy coding algorithm or scheme (e.g., a variable length coding (VLC) scheme, a context adaptive VLC (CAVLC) scheme, an arithmetic coding scheme, binarization, context adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or other entropy coding methods or techniques) to the quantized coefficients 209, inter-frame prediction parameters, intra-frame prediction parameters, loop filter parameters, and / or other syntax elements to obtain coded image data 21 that can be output via an output terminal 272 in the form of a coded bitstream 21, etc., so that the video decoder 30 can receive and use these parameters for decoding, etc. The coded bitstream 21 can be sent to the video decoder 30 or stored in a memory for later transmission or retrieval by the video decoder 30.

[0176] Other structural variations of the video encoder 20 may be used to encode the video stream. For example, a non-transform based encoder 20 may directly quantize the residual signal for certain blocks or frames without the transform processing unit 206. In another implementation, the encoder 20 may include the quantization unit 208 and the inverse quantization unit 210 combined into a single unit.

[0177] Decoder and decoding method

[0178] Figure 33 is an example of a video decoder 30 for implementing the techniques of the present application. The video decoder 30 is configured to receive coded image data 21 (e.g., coded codestream 21) coded by, for example, an encoder 20, to obtain a decoded image 331. The coded image data or codestream includes information for processing the coded image data, such as data representing image blocks of coded video slices (and / or partition groups or partitions) and related syntax elements.

[0179] exist Figure 3 In the example of FIG. 3 , decoder 30 includes an entropy decoding unit 304, an inverse quantization unit 310, an inverse transform processing unit 312, a reconstruction unit 314 (e.g., a summer 314), a loop filter 320, a decoded picture buffer (DBP) 330, a mode application unit 360, an inter-frame prediction unit 344, and an intra-frame prediction unit 354. Inter-frame prediction unit 344 may be or may include a motion compensation unit. In some examples, video decoder 30 may perform substantially the same motion compensation as reference video. Figure 2 The encoding process described in the video encoder 100 is the inverse of the decoding process.

[0180] As described with reference to encoder 20, inverse quantization unit 210, inverse transform processing unit 212, reconstruction unit 214, loop filter 220, decoded picture buffer (DPB) 230, inter-frame prediction unit 344, and intra-frame prediction unit 354 also constitute the "built-in decoder" of video encoder 20. Accordingly, inverse quantization unit 310 may be functionally identical to inverse quantization unit 110, inverse transform processing unit 312 may be functionally identical to inverse transform processing unit 212, reconstruction unit 314 may be functionally identical to reconstruction unit 214, loop filter 320 may be functionally identical to loop filter 220, and decoded picture buffer 330 may be functionally identical to decoded picture buffer 230. Therefore, the explanations given with respect to the corresponding units and functions of video encoder 20 apply correspondingly to the corresponding units and functions of video decoder 30.

[0181] Entropy decoding

[0182] The entropy decoding unit 304 is used to parse the code stream 21 (or generally referred to as the encoded image data 21) and perform entropy decoding on the encoded image data 21 to obtain quantized coefficients 309 and / or decoded encoding parameters ( Figure 3, for example, any or all of inter-frame prediction parameters (e.g., reference picture index and motion vector), intra-frame prediction parameters (e.g., intra-frame prediction mode or index), transform parameters, quantization parameters, loop filter parameters, and / or other syntax elements. The entropy decoding unit 304 may be configured to apply a decoding algorithm or scheme corresponding to the encoding scheme described with reference to the entropy encoding unit 270 in the encoder 20. The entropy decoding unit 304 may also be configured to provide the inter-frame prediction parameters, intra-frame prediction parameters, and / or other syntax elements to the mode application unit 360 and to provide other parameters to other units in the decoder 30. The video decoder 30 may receive syntax elements at the video slice level and / or the video block level. In addition to or as an alternative to slices and corresponding syntax elements, partition groups and / or partitions and corresponding syntax elements may be received and / or used.

[0183] Dequantization

[0184] The inverse quantization unit 310 may be configured to receive a quantization parameter (QP) (or generally referred to as information related to inverse quantization) and quantized coefficients from the encoded image data 21 (e.g., parsed and / or decoded by the entropy decoding unit 304 or the like), and inverse quantize the decoded quantized coefficients 309 according to the quantization parameters to obtain dequantized coefficients 311. The dequantized coefficients 311 may also be referred to as transform coefficients 311. The inverse quantization process may include using the quantization parameters determined by the video encoder 20 for each video block in a video slice (or partition or group of partitions) to determine a degree of quantization, and thus a degree of inverse quantization to be applied.

[0185] Inverse transform

[0186] The inverse transform processing unit 312 may be configured to receive the dequantized coefficients 311 (also referred to as transform coefficients 311) and perform a transform on the dequantized coefficients 311 to obtain a reconstructed residual block 213 in the sample domain. The reconstructed residual block 213 may also be referred to as a transform block 313. The transform may be an inverse transform, such as an inverse DCT, an inverse DST, an inverse integer transform, or a conceptually similar inverse transform process. The inverse transform processing unit 312 may also be configured to receive transform parameters or corresponding information from the encoded image data 21 (e.g., parsed and / or decoded by the entropy decoding unit 304 or the like) to determine the transform to be applied to the dequantized coefficients 311.

[0187] reconstruction

[0188] The reconstruction unit 314 (e.g., adder or summer 314) can be used to add the reconstructed residual block 313 to the prediction block 365 to obtain the reconstructed block 315 in the sample domain by, for example, adding the sample values ​​of the reconstructed residual block 313 to the sample values ​​of the prediction block 365.

[0189] Filtering

[0190] The loop filter unit 320 (in the decoding loop or after) is used to filter the reconstructed block 315 to obtain a filtered block 321 to smooth pixel transitions or otherwise improve video quality, etc. The loop filter unit 320 may include one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or one or more other filters, such as a bilateral filter, an adaptive loop filter (ALF), a sharpening or smoothing filter, a collaborative filter, or any combination thereof. Although the loop filter unit 320 is in Figure 3 3. The loop filter unit 320 is shown as an in-loop filter, but in other configurations, the loop filter unit 320 may be implemented as a post-loop filter.

[0191] Decoded image buffer

[0192] The decoded video block 321 of one picture is then stored in a decoded picture buffer 330 , which stores the decoded picture 331 as a reference picture for subsequent motion compensation and / or output or display of other pictures.

[0193] The decoder 30 is configured to output a decoded image 311 through an output terminal 312 or the like, so as to be presented to a user or viewed by the user.

[0194] predict

[0195] The inter-frame prediction unit 344 may be functionally identical to the inter-frame prediction unit 244 (particularly the motion compensation unit), and the intra-frame prediction unit 354 may be functionally identical to the inter-frame prediction unit 254, and may perform partitioning or segmentation decisions and prediction based on the partitioning mode and / or prediction parameters or corresponding information received from the coded image data 21 (e.g., parsed and / or decoded by the entropy decoding unit 304, etc.). The mode application unit 360 may be configured to perform prediction (intra-frame or inter-frame prediction) on a block-by-block basis based on the reconstructed image, block, or corresponding samples (filtered or unfiltered) to obtain a prediction block 365.

[0196] When the video slice is coded as an intra-coded (I) slice, the intra-prediction unit 354 in the mode application unit 360 is configured to generate a prediction block 365 for a picture block in the current video slice based on the indicated intra-prediction mode and data from previously decoded blocks in the current picture. When the video picture is coded as an inter-coded (i.e., B or P) slice, the inter-prediction unit 344 (e.g., a motion compensation unit) in the mode application unit 360 is configured to generate a prediction block 365 for a video block in the current video slice based on the motion vector and other syntax elements received from the entropy decoding unit 304. For inter-prediction, these prediction blocks can be generated from one of the reference pictures in one of the reference picture lists. Video decoder 30 can construct reference frame list 0 and list 1 using a default construction technique based on the reference pictures stored in DPB 330. The same or similar processes may be applied to or by embodiments that use partition groups (e.g., video partition groups) and / or partitions (e.g., video partitions) in addition to or instead of slices (e.g., video slices), e.g., video may be coded using I, P, or B partition groups and / or partitions.

[0197] Mode application unit 360 is configured to determine prediction information for a video block in a current video slice by parsing motion vectors or related information and other syntax elements, and use the prediction information to generate a prediction block for the current video block being decoded. For example, mode application unit 360 uses some of the received syntax elements to determine a prediction mode (e.g., intra or inter prediction) for coding the video block in the video slice, the inter prediction slice type (e.g., B slice, P slice, or GPB slice), construction information for one or more reference picture lists for the slice, motion vectors for each inter-coded video block in the slice, inter prediction status for each inter-coded video block in the slice, and other information to decode the video block in the current video slice. The same or similar processes may be applied to or by embodiments that use partition groups (e.g., video partition groups) and / or partitions (e.g., video partitions) in addition to or as an alternative to slices (e.g., video slices). For example, video may be coded using I, P, or B partition groups and / or partitions.

[0198] Figure 3 The illustrated embodiment of the video decoder 30 may be used to partition and / or decode a picture by slices (also referred to as video slices). A picture may be partitioned into or decoded by one or more (usually non-overlapping) slices, each of which may include one or more blocks (e.g., CTUs).

[0199] Figure 3The illustrated embodiment of the video decoder 30 may be used to segment and / or decode an image by partition groups (also referred to as video partition groups) and / or blocks (also referred to as video blocks). A picture may be segmented into or decoded by one or more (usually non-overlapping) partition groups; each partition group may include one or more blocks (e.g., CTUs) or one or more partitions, etc.; each partition may be rectangular, etc., and may include one or more complete or partial blocks (e.g., CTUs), etc.

[0200] Other variations of the video decoder 30 may be used to decode the encoded image data 21. For example, the decoder 30 may generate an output video stream without the loop filter unit 320. For example, a non-transform-based decoder 30 may directly dequantize the residual signal for certain blocks or frames without the inverse transform processing unit 312. In another implementation, the video decoder 30 may include the dequantization unit 310 and the inverse transform processing unit 312 combined into a single unit.

[0201] It should be understood that the processing results of the current step can be further processed in the encoder 20 and decoder 30 before being output to the next step. For example, after interpolation filtering, motion vector derivation, or loop filtering, further operations such as clipping or shifting can be performed on the processing results of interpolation filtering, motion vector derivation, or loop filtering.

[0202] It should be noted that more operations can be applied to the derived motion vector of the current block (including but not limited to control point motion vectors in affine mode, sub-block motion vectors in affine mode, planar mode, and ATMVP mode, temporal motion vectors, etc.). For example, the value of the motion vector is constrained to a predefined range based on the bit depth of the motion vector. If the bit depth of the motion vector is bit depth, the range is –2^(bit depth – 1) to 2^(bit depth – 1) – 1, where “^” represents a power. For example, if bit depth is set to 16, the range is –32768 to 32767; if bit depth is set to 18, the range is –131072 to 131071. For example, the value of the derived motion vector (e.g., the MVs of four 4×4 sub-blocks in an 8×8 block) is constrained so that the maximum difference between the integer parts of the MVs of these four 4×4 sub-blocks does not exceed N pixels, for example, not more than 1 pixel. Two methods for constraining motion vectors based on bit depth are provided herein.

[0203] Method 1: Remove the most significant bit (MSB) of the overflow by smoothing

[0204] ux=(mvx+2 bitDepth )%2 bitDepth (1)

[0205] mvx=(ux>=2 bitDepth–1 )? (ux–2 bitDepth ):ux (2)

[0206] uy=(mvy+2 bitDepth )%2 bitDepth (3)

[0207] mvy=(uy>=2 bitDepth–1 )? (uy–2 bitDepth ):uy (4)

[0208] Among them, mvx is the horizontal component of the motion vector of an image block or sub-block, mvy is the vertical component of the motion vector of an image block or sub-block, and ux and uy represent median values.

[0209] For example, if the value of mvx is –32769, the value obtained by using formulas (1) and (2) is 32767. In computer systems, decimal numbers are stored in two's complement form. The two's complement of –32769 is 1,0111,1111,1111,1111 (17 bits), and then the MSB is discarded, so the two's complement obtained is 0111,1111,1111,1111 (32767 in decimal), which is the same as the output result obtained by using formulas (1) and (2).

[0210] ux=(mvpx+mvdx+2 bitDepth )%2 bitDepth (5)

[0211] mvx=(ux>=2 bitDepth–1 )? (ux–2 bitDepth ):ux (6)

[0212] uy=(mvpy+mvdy+2 bitDepth )%2 bitDepth (7)

[0213] mvy=(uy>=2 bitDepth–1 )? (uy–2 bitDepth ):uy (8)

[0214] These operations can be performed in the process of summing mvp and mvd, as shown in equations (5) to (8).

[0215] Method 2: Remove the overflowed MSB by clipping the value

[0216] vx=Clip3(–2 bitDepth–1 ,2 bitDepth–1 –1,vx)

[0217] vy=Clip3(–2 bitDepth–1 ,2 bitDepth–1 –1,vy)

[0218] Where vx is the horizontal component of the motion vector of an image block or sub-block, vy is the vertical component of the motion vector of an image block or sub-block, and x, y, and z correspond to the three input values ​​of the MV clipping process. The function Clip3 is defined as follows:

[0219]

[0220] Figure 4 Schematic diagram of a video decoding device 400 provided in accordance with an embodiment of the present invention. The video decoding device 400 is suitable for implementing the disclosed embodiments described herein. In one embodiment, the video decoding device 400 may be a decoder (e.g., Figure 1A ) or an encoder (e.g. Figure 1A Video encoder 20 in ).

[0221] The video decoding device 400 includes an input port 410 (or input port 410) and a receiving unit (Rx) 420 for receiving data, a processor, a logic unit, or a central processing unit (CPU) 430 for processing the data, a transmitting unit (Tx) 440 and an output port 450 (or output port 450) for transmitting the data, and a memory 460 for storing the data. The video decoding device 400 may also include optical-to-electrical (OE) components and electrical-to-optical (EO) components coupled to the input port 410, the receiving unit 420, the transmitting unit 440, and the output port 450, serving as an outlet or inlet for optical or electrical signals.

[0222] Processor 430 is implemented using hardware and software. Processor 430 can be implemented as one or more CPU chips, one or more cores (e.g., a multi-core processor), one or more FPGAs, one or more ASICs, and one or more DSPs. Processor 430 communicates with input port 410, receiving unit 420, transmitting unit 440, output port 450, and memory 460. Processor 430 includes a decoding module 470. Decoding module 470 implements the disclosed embodiments described above. For example, decoding module 470 performs, processes, prepares, or provides various decoding operations. Therefore, the inclusion of decoding module 470 provides substantial improvements to the functionality of video decoding device 400 and affects the transition of video decoding device 400 to different states. Optionally, decoding module 470 is implemented using instructions stored in memory 460 and executed by processor 430.

[0223] The memory 460 may include one or more magnetic disks, one or more tape drives, and one or more solid-state drives, and may be used as an overflow data storage device to store programs when they are selected for execution, as well as to store instructions and data read during the execution of the programs. For example, the memory 460 may be volatile and / or non-volatile, and may be a read-only memory (ROM), a random access memory (RAM), a ternary content-addressable memory (TCAM), and / or a static random-access memory (SRAM).

[0224] Figure 5 A simplified block diagram of an apparatus 500 is provided for an exemplary embodiment. The apparatus 500 may be used as Figure 1A The source device 12 and / or destination device 14 in.

[0225] The processor 502 in the apparatus 500 may be a central processing unit. Alternatively, the processor 502 may be any other type of device or devices, now available or to be developed in the future, that can operate or process information. Although the disclosed implementations may be implemented using a single processor, such as the processor 502 shown, using multiple processors may improve speed and efficiency.

[0226] In one implementation, the memory 504 in the apparatus 500 may be a read-only memory (ROM) device or a random access memory (RAM) device. Any other suitable type of storage device may be used as the memory 504. The memory 504 may include code and data 506 that are accessed by the processor 502 via a bus 512. The memory 504 may also include an operating system 508 and application programs 510. The application programs 510 include at least one program that causes the processor 502 to perform the methods described herein. For example, the application programs 510 may include application 1 through application N, and may also include a video decoding application that performs the methods described herein.

[0227] The apparatus 500 may also include one or more output devices, such as a display 518. In one example, the display 518 may be a touch-sensitive display that combines a display with a touch-sensitive element capable of sensing touch input. The display 518 may be coupled to the processor 502 via the bus 512.

[0228] Although bus 512 in device 500 is described herein as a single bus, bus 512 may include multiple buses. In addition, auxiliary memory 514 may be directly coupled to other components in device 500 or may be accessed via a network and may include a single integrated unit (e.g., a memory card) or multiple units (e.g., multiple memory cards). Thus, device 500 may be implemented in a variety of configurations.

[0229] The current version of the VVC draft specification (document JVET-P2001-vE: B. Bross, J. Chen, S. Liu, Y.-K. Wang, "Generic Video Coding (Draft 7)", output document JVET-P2001 from the 16th JVET meeting in Geneva, Switzerland; this document is contained in the file JVET-P2001-v14: http: / / phenix.it-sudparis.eu / jvet / doc_end_user / documents / 16_Geneva / wg11 / JVET-P2001-v14.zip) specifies the following series of steps to generate intra-frame prediction values ​​for the directional modes available in VVC.

[0230] Detailed description of INTRA_ANGULAR2..INTRA_ANGULAR66 intra prediction modes

[0231] Inputs to this process include:

[0232] – Intra prediction mode predModeIntra;

[0233] – variable refIdx, indicating the intra prediction reference line index;

[0234] – variable nTbW, representing the transform block width;

[0235] – variable nTbH, representing the transform block height;

[0236] – variable refW, indicating the reference sample width;

[0237] – variable refH, indicating the reference sample height;

[0238] – variable nCbW, which represents the coding block width;

[0239] – variable nCbH, which represents the coding block height;

[0240] – variable refFilterFlag, indicating the value of the reference filter flag;

[0241] –Variable cIdx, which represents the color component of the current block.

[0242] – Neighboring samples p[x][y], where x = –1–refIdx, y = –1–refIdx..refH–1, and x = –refIdx..refW–1, y = –1–refIdx.

[0243] The output of this process is the predicted samples predSamples[x][y], where x = 0..nTbW–1 and y = 0..nTbH–1.

[0244] The variable nTbS is set to (Log2(nTbW)+Log2(nTbH))>>1.

[0245] The variable filterFlag is derived as follows:

[0246] – filterFlag is set to 0 if one or more of the following conditions are true:

[0247] –refFilterFlag is equal to 1;

[0248] –refIdx is not equal to 0;

[0249] –IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT.

[0250] – Otherwise, the following applies:

[0251] –The variable minDistVerHor is set to Min(Abs(predModeIntra–50),Abs(predModeIntra–18)).

[0252] –The variable intraHorVerDistThres[nTbS] is shown in Table 1.

[0253] –The variable filterFlag is derived as follows:

[0254] – If minDistVerHor is greater than intraHorVerDistThres[nTbS], filterFlag is set to 1.

[0255] – Otherwise, filterFlag is set to 0.

[0256] Table 1: Detailed description of intraHorVerDistThres[nTbS] for various transform block sizes nTbS

[0257] nTbS=2 nTbS=3 nTbS=4 nTbS=5 nTbS=6 nTbS=7 intraHorVerDistThres[nTbS] 24 14 2 0 0 0

[0258] Figure 6 93 prediction directions are shown, where the dashed directions are associated with wide-angle modes applied to non-square blocks. Table 2 shows a mapping table between predModeIntra and the angle parameter intraPredAngle.

[0259] Table 2: Detailed description of intraPredAngle

[0260]

[0261]

[0262] The inverse angle parameter invAngle is derived from intraPredAngle as follows:

[0263]

[0264] The interpolation filter coefficients fC[phase][j] and fG[phase][j] (where phase = 0..31, j = 0..3) are shown in Table 3.

[0265] Table 3: Detailed description of interpolation filter coefficients fC and fG

[0266]

[0267]

[0268] The values ​​of the predicted samples predSamples[x][y] (where x=0..nTbW-1, y=0..nTbH-1) are derived as follows:

[0269] – If predModeIntra is greater than or equal to 34, the following steps, in order, apply:

[0270] The reference sample array ref[x] is specified as follows:

[0271] – The following apply:

[0272] ref[x]=p[-1-refIdx+x][-1-refIdx], where x=0..nTbW+refIdx+1.

[0273] – If intraPredAngle is less than 0, the main reference sample array is extended as follows:

[0274] ref[x]=p[-1-refIdx][-1-refIdx+Min((x*invAngle+256)>>9,nTbH)], where x=-nTbH..-1.

[0275] -otherwise,

[0276] ref[x]=p[-1-refIdx+x][-1-refIdx], where x=nTbW+2+refIdx..refW+refIdx.

[0277] – The other samples ref[refW+refIdx+x] (where x=1..(Max(1,nTbW / nTbH)*refIdx+1) are derived as follows:

[0278] ref[refW+refIdx+x]=p[-1+refW][-1-refIdx].

[0279] The values ​​of the predicted samples predSamples[x][y] (where x=0..nTbW-1, y=0..nTbH-1) are derived as follows:

[0280] –The index variable iIdx and the multiplication factor iFact are derived as follows:

[0281] iIdx=(((y+1+refIdx)*intraPredAngle)>>5)+refIdx,

[0282] iFact=((y+1+refIdx)*intraPredAngle)&31.

[0283] – If cIdx is equal to 0, the following applies:

[0284] – The interpolation filter coefficients fT[j] (where j = 0..3) are derived as follows:

[0285] fT[j]=filterFlag? fG[iFact][j]:fC[iFact][j].

[0286] –The value of the predicted sample predSamples[x][y] is derived as follows:

[0287]

[0288] – Otherwise (cIdx is not equal to 0), depending on the value of iFact, the following applies:

[0289] –If iFact is not equal to 0, the value of the predicted sample predSamples[x][y] is derived as follows:

[0290] predSamples[x][y]=((32-iFact)*ref[x+iIdx+1]+iFact*ref[x+iIdx+2]+16)>>5.

[0291] – Otherwise, the value of the predicted sample predSamples[x][y] is derived as follows:

[0292] predSamples[x][y]=ref[x+iIdx+1].

[0293] Otherwise (predModeIntra is less than 34), the following steps, in order, apply:

[0294] The reference sample array ref[x] is specified as follows:

[0295] – The following apply:

[0296] ref[x]=p[-1-refIdx][-1-refIdx+x], where x=0..nTbH+refIdx+1.

[0297] – If intraPredAngle is less than 0, the main reference sample array is extended as follows:

[0298] ref[x]=p[-1-refIdx+Min((x*invAngle+256)>>9,nTbW)][-1-refIdx], where x=-nTbW..-1.

[0299] -otherwise,

[0300] ref[x]=p[-1-refIdx][-1-refIdx+x], where x=nTbH+2+refIdx..refH+refIdx.

[0301] – Other samples ref[refH+refIdx+x] (where x=1..(Max(1,nTbH / nTbW)*refIdx+1)

[0302] The derivation is as follows:

[0303] ref[refH+refIdx+x]=p[-1+refH][-1-refIdx].

[0304] The values ​​of the predicted samples predSamples[x][y] (where x=0..nTbW-1, y=0..nTbH-1) are derived as follows:

[0305] –The index variable iIdx and the multiplication factor iFact are derived as follows:

[0306] iIdx=(((x+1+refIdx)*intraPredAngle)>>5)+refIdx,

[0307] iFact=((x+1+refIdx)*intraPredAngle)&31.

[0308] – If cIdx is equal to 0, the following applies:

[0309] – The interpolation filter coefficients fT[j] (where j = 0..3) are derived as follows:

[0310] fT[j]=filterFlag? fG[iFact][j]:fC[iFact][j].

[0311] –The value of the predicted sample predSamples[x][y] is derived as follows:

[0312]

[0313] – Otherwise (cIdx is not equal to 0), depending on the value of iFact, the following applies:

[0314] –If iFact is not equal to 0, the value of the predicted sample predSamples[x][y] is derived as follows:

[0315] predSamples[x][y]=((32-iFact)*ref[y+iIdx+1]+iFact*ref[y+iIdx+2]+16)>>5.

[0316] – Otherwise, the value of the predicted sample predSamples[x][y] is derived as follows:

[0317] predSamples[x][y]=ref[y+iIdx+1].

[0318] As described in the VVC specification, the interpolation filters applied to the reference samples in luma blocks and chroma blocks have different lengths. One of two 4-tap interpolation filters can be selected depending on the direction and block size (by setting refFilterFlag and filterFlag to some specific values) to construct the intra-predicted values ​​for the directional mode in the case of luma blocks, while a linear (e.g. 2-tap) filter is used for the directional mode in the case of chroma blocks. Interpolation filtering can be used to predict samples in a block when their projection falls between the positions of reference samples that belong to a previously reconstructed block and can be (optionally or conditionally) filtered. The fractional sample position p is selected based on the directional intra-prediction mode index and the position of the predicted sample within the block. The mode in which the interpolation filtering process needs to be performed is called "fractional-slope mode". As Figure 7A and Figure 8A as well as Figure 7B and Figure 8B As shown, the padding operation is performed by expanding the length of the reference samples belonging to the main reference side, depending on the length of the interpolation filter. For example, the padding reference sample 704 or 804 can be generated by copying the nearest available reference sample (e.g., 705 or 805) belonging to the main reference side 701 or 801. Here, Figure 7B and Figure 8B Corresponding to Figure 7A and Figure 8A .However, Figure 7B and Figure 8B Elements 705 and 805 are explicitly shown, while all other elements are Figure 7A and Figure 8A The fill reference sample 704 or 804 may also be filled with a predefined value. For example, this predefined value may be equal to the middle value s in the sample range. m , the intermediate value can be expressed as s m =(1< <BitDepth C )>>1, or equivalently expressed as s m =(1<<(BitDepth C –1), where BitDepthC is the bit depth of the samples of color component C. Another way to perform padding is to calculate the average value s of the entire set of available reference samples or a subset of them DC , and then use this average value s DC Assigned to padding reference samples. Other padding mechanisms are also possible. Normally, the longer the intra-frame interpolation filter used, the more padding reference samples should be used. Typically, prediction block 703 or 803 does not require padding side reference samples 702 or 802. In the VVC draft specification, padding reference samples are referred to as "other samples" above.

[0319] For integer-slope mode, the projection of the prediction sample always falls on an integer position of the reference sample. In this case, if reference sample smoothing is required for integer-slope mode, it can be performed by applying a 3-tap filter with coefficients [1,2,1] / 4 to the reference samples in the luma block. If the reference sample smoothing filter is applied to the reference samples, reference sample padding is usually not performed. To avoid referencing samples that exceed twice the length of a reference side, one or more outermost reference samples in each side (e.g., Figure 9 Reference samples 931, 932, 941 and 942 are shown and Figure 10 The reference samples 1031, 1032, 1041, and 1042 shown can remain unfiltered. Another approach is to reduce the length of the reference sample smoothing filter (e.g., from 5 taps to 3 taps). Specifically, if a 5-tap filter with coefficients of [1, 0, 2, 0, 1] / 4 is applied to the reference samples in integer slope mode, then the 2 outermost reference samples (e.g., Figure 9 Reference samples 931 and 932 are shown and Figure 10 The reference samples 1031 and 1032 shown here skip this filtering, which is mathematically equivalent to using a 1-tap bypass filter with a coefficient of [1] / 1. A shorter length filter can be applied to the outermost samples belonging to the left reference side (e.g., Figure 9 Reference samples 942 and Figure 10 The reference sample 1041 shown) and the outermost sample belonging to the upper reference side (e.g., Figure 9 Reference samples 941 and Figure 10 1] / 4 ). Note that in some examples (e.g., in the VVC draft specification), reference sample smoothing is disabled for chroma blocks.

[0320] Generally speaking, interpolation filters of different lengths can refer to different reference samples. Figure 9 As shown in the example, since the linear filter does not extend beyond primary reference side 901 of prediction block 903, no padding reference samples are required. Even for outermost directional patterns 911 and 912, which limit the range 910 of directional patterns available for block 903, no padding reference samples are required. Linear filter coefficients 921 applied to the outermost reference samples belonging to primary reference side 901 are used for the bottom-right sample in prediction block 903.

[0321] like Figure 10 As shown exemplarily, if left reference column 1001 is used as the primary reference side, range 1010 lies between the outermost directivity patterns 1011 and 1012 of block 1003. Typically, reference sides 902 and 1002 are not primary sides and therefore do not participate in the interpolation process of the outermost directivity patterns.

[0322] Therefore, if a linear filter is applied to the reference samples in the fractional slope mode case, the reference sample set does not need to exceed the number defined by the following formula:

[0323] – When predModeIntra is greater than or equal to 34, (refW + refIdx + Max(1,nTbW / nTbH)*refIdx); or

[0324] –When predModeIntra is less than 34, (refH+refIdx+Max(1,nTbH / nTbW)*refIdx).

[0325] Note that this formula covers the case where non-adjacent reference lines are used to construct the intra prediction values ​​for directional modes, i.e., the selected reference lines can be some distance away from the block to be predicted. However, multi-reference intra prediction is not applicable to chroma blocks, as described in section 8.4.5.2.5 "General intra sample prediction" of the VVC draft specification (document JVET-P2001-vE: B. Bross, J. Chen, S. Liu, Y.-K. Wang, "Generic Video Coding (Draft 7)", output document JVET-P2001 of the 16th JVET meeting, Geneva, Switzerland).

[0326] 8.4.5.2.5 General Intra Sample Prediction

[0327] Inputs to this process include:

[0328] – Sample position (xTbCmp, yTbCmp), which indicates the position of the upper left sample of the current transform block relative to the upper left sample of the current image;

[0329] – variable predModeIntra, indicating intra prediction mode;

[0330] – variable nTbW, representing the transform block width;

[0331] – variable nTbH, representing the transform block height;

[0332] – variable nCbW, which represents the coding block width;

[0333] – variable nCbH, which represents the coding block height;

[0334] –Variable cIdx, which represents the color component of the current block.

[0335] The output of this process is the predicted samples predSamples[x][y], where x = 0..nTbW–1 and y = 0..nTbH–1.

[0336] The variables refW and refH are derived as follows:

[0337] – If IntraSubPartitionsSplitType is equal to ISP_NO_SPLIT or cIdx is not equal to 0, the following applies:

[0338] refW=nTbW*2,

[0339] refH=nTbH*2.

[0340] – Otherwise (IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT and cIdx is equal to 0), the following applies:

[0341] refW=nCbW+nTbW,

[0342] refH=nCbH+nTbH.

[0343] The variable refIdx represents the intra prediction reference line index, which is derived as follows:

[0344] refIdx=(cIdx==0)? IntraLumaRefLineIdx[xTbCmp][yTbCmp]:0.

[0345] The Wide Intra prediction mode mapping process detailed in section 8.4.5.2.6 is called with predModeIntra, nTbW, nTbH and cIdx as inputs and the modified predModeIntra as output.

[0346] The variable refFilterFlag is derived as follows:

[0347] – If predModeIntra is equal to one of the following values: 0, -14, -12, -10, -6, 2, 34, 66, 72, 76, 78, 80, then refFilterFlag is set to 1.

[0348] – Otherwise, refFilterFlag is set to 0.

[0349] Therefore, if cIdx is not equal to 0, the value of the variable refIdx is equal to 0; this means that the color component of the block is not luma. For chroma blocks, refIdx cannot take a value different from 0. Therefore, substituting 0 instead of refIdx in the formula,

[0350] – When predModeIntra is greater than or equal to 34, (refW + refIdx + Max(1,nTbW / nTbH)*refIdx); or

[0351] –When predModeIntra is less than 34, (refH+refIdx+Max(1,nTbH / nTbW)*refIdx).

[0352] refW or refH is obtained. The refW or refH of the reference sample is available because it satisfies the expression used above before defining the other samples ref[refW+refIdx+x] and ref[refH+refIdx+x], as described in Section 8.4.5.2.12 "Detailed description of INTRA_ANGULAR2..INTRA_ANGULAR66 intra prediction modes" in the VVC draft specification (document JVET-P2001-vE: B. Bross, J. Chen, S. Liu, Y.-K. Wang, "Generic Video Coding (Draft 7)", output document JVET-P2001 of the 16th JVET meeting in Geneva, Switzerland). Therefore, when predicting chroma blocks, the above-mentioned specific requirements specified by the linear filter are applied to the chroma reference samples; two parts related to how to derive other samples are derived, and the various parts of Section 8.4.5.2.12 "Detailed description of INTRA_ANGULAR2..INTRA_ANGULAR66 intra-frame prediction modes" in the VVC draft specification (document JVET-P2001-vE: "Generic Video Coding (Draft 7)" by B. Bross, J. Chen, S. Liu, Y.-K. Wang, output document JVET-P2001 at the 16th JVET meeting in Geneva, Switzerland) can be rewritten as follows.

[0353] Example 1:

[0354] If predModeIntra is greater than or equal to 34, the following steps, in order, apply:

[0355] The reference sample array ref[x] is specified as follows:

[0356] – The following apply:

[0357] ref[x]=p[-1-refIdx+x][-1-refIdx], where x=0..nTbW+refIdx+1;

[0358] – If intraPredAngle is less than 0, the main reference sample array is extended as follows:

[0359] ref[x]=p[-1-refIdx][-1-refIdx+Min((x*invAngle+256)>>9,nTbH)], where x=-nTbH..-1.

[0360] -otherwise,

[0361] ref[x]=p[-1-refIdx+x][-1-refIdx], where x=nTbW+2+refIdx..refW+refIdx.

[0362] If cIdx is equal to 0 (in an embodiment, “cIdx==0” indicates that the color component of the block to be predicted is luma. The chroma block skips deriving other samples), the other samples ref[refW+refIdx+x] (where x=1..(Max(1,nTbW / nTbH)*refIdx+1)) are derived as follows:

[0363] ref[refW+refIdx+x]=p[-1+refW][-1-refIdx].

[0364] Otherwise (predModeIntra is less than 34), the following steps, in order, apply:

[0365] The reference sample array ref[x] is specified as follows:

[0366] – The following apply:

[0367] ref[x]=p[-1-refIdx][-1-refIdx+x], where x=0..nTbH+refIdx+1.

[0368] – If intraPredAngle is less than 0, the main reference sample array is extended as follows:

[0369] ref[x]=p[-1-refIdx+Min((x*invAngle+256)>>9,nTbW)][-1-refIdx], where x=-nTbW..-1.

[0370] -otherwise,

[0371] ref[x]=p[-1-refIdx][-1-refIdx+x], where x=nTbH+2+refIdx..refH+refIdx.

[0372] If cIdx is equal to 0, the other samples ref[refH+refIdx+x] (where x=1..(Max(1,nTbH / nTbW)*refIdx+1)) are derived as follows:

[0373] ref[refH+refIdx+x]=p[-1+refH][-1-refIdx].

[0374] Another embodiment of the present invention is described below.

[0375] Example 2:

[0376] If predModeIntra is greater than or equal to 34, the following steps, in order, apply:

[0377] The reference sample array ref[x] is specified as follows:

[0378] – The following apply:

[0379] ref[x]=p[-1-refIdx+x][-1-refIdx], where x=0..nTbW+refIdx+1.

[0380] – If intraPredAngle is less than 0, the main reference sample array is extended as follows:

[0381] ref[x]=p[-1-refIdx][-1-refIdx+Min((x*invAngle+256)>>9,nTbH)], where x=-nTbH..-1.

[0382] -otherwise,

[0383] ref[x]=p[-1-refIdx+x][-1-refIdx], where x=nTbW+2+refIdx..refW+refIdx.

[0384] If cIdx is equal to 0, the other samples ref[refW+refIdx+x] (where x=1..(Max(1,nTbW / nTbH)*refIdx+2)) are derived as follows:

[0385] ref[refW+refIdx+x]=p[-1+refW][-1-refIdx].

[0386] Otherwise (predModeIntra is less than 34), the following steps, in order, apply:

[0387] The reference sample array ref[x] is specified as follows:

[0388] – The following apply:

[0389] ref[x]=p[-1-refIdx][-1-refIdx+x], where x=0..nTbH+refIdx+1.

[0390] – If intraPredAngle is less than 0, the main reference sample array is extended as follows:

[0391] ref[x]=p[-1-refIdx+Min((x*invAngle+256)>>9,nTbW)][-1-refIdx], where x=-nTbW..-1.

[0392] -otherwise,

[0393] ref[x]=p[-1-refIdx][-1-refIdx+x], where x=nTbH+2+refIdx..refH+refIdx.

[0394] If cIdx is equal to 0, the other samples ref[refH+refIdx+x] (where x=1..(Max(1,nTbH / nTbW)*refIdx+2)) are derived as follows:

[0395] ref[refH+refIdx+x]=p[-1+refH][-1-refIdx].

[0396] Therefore, Section 8.4.5.2.12 “Detailed description of INTRA_ANGULAR2..INTRA_ANGULAR66 intra prediction modes” in the VVC draft specification (document JVET-P2001-vE: B. Bross, J. Chen, S. Liu, Y.-K. Wang, “Generic Video Coding (Draft 7)”, output document JVET-P2001 from the 16th JVET meeting in Geneva, Switzerland) can be rewritten as follows.

[0397] Example 1:

[0398] Detailed description of INTRA_ANGULAR2..INTRA_ANGULAR66 intra prediction modes

[0399] Inputs to this process include:

[0400] – Intra prediction mode predModeIntra;

[0401] – variable refIdx, indicating the intra prediction reference line index;

[0402] – variable nTbW, representing the transform block width;

[0403] – variable nTbH, representing the transform block height;

[0404] – variable refW, indicating the reference sample width;

[0405] – variable refH, indicating the reference sample height;

[0406] – variable nCbW, which represents the coding block width;

[0407] – variable nCbH, which represents the coding block height;

[0408] – variable refFilterFlag, indicating the value of the reference filter flag;

[0409] –Variable cIdx, which represents the color component of the current block.

[0410] – Neighboring samples p[x][y], where x = –1–refIdx, y = –1–refIdx..refH–1, and x = –refIdx..refW–1, y = –1–refIdx.

[0411] The output of this process is the predicted samples predSamples[x][y], where x = 0..nTbW–1 and y = 0..nTbH–1.

[0412] The variable nTbS is set to (Log2(nTbW)+Log2(nTbH))>>1.

[0413] The variable filterFlag is derived as follows:

[0414] – filterFlag is set to 0 if one or more of the following conditions are true:

[0415] –refFilterFlag is equal to 1;

[0416] –refIdx is not equal to 0;

[0417] –IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT.

[0418] – Otherwise, the following applies:

[0419] –The variable minDistVerHor is set to Min(Abs(predModeIntra–50),Abs(predModeIntra–18)).

[0420] –The variable intraHorVerDistThres[nTbS] is shown in Table 4.

[0421] –The variable filterFlag is derived as follows:

[0422] – If minDistVerHor is greater than intraHorVerDistThres[nTbS], filterFlag is set to 1.

[0423] – Otherwise, filterFlag is set to 0.

[0424] Table 4: Detailed description of intraHorVerDistThres[nTbS] for various transform block sizes nTbS

[0425] nTbS=2 nTbS=3 nTbS=4 nTbS=5 nTbS=6 nTbS=7 intraHorVerDistThres[nTbS] 24 14 2 0 0 0

[0426] Figure 6 93 prediction directions are shown, where the dashed directions are associated with wide-angle modes applied to non-square blocks. Table 5 shows a mapping table between predModeIntra and the angle parameter intraPredAngle.

[0427] Table 5: Detailed description of intraPredAngle

[0428]

[0429]

[0430] The inverse angle parameter invAngle is derived from intraPredAngle as follows:

[0431]

[0432] The interpolation filter coefficients fC[phase][j] and fG[phase][j] (where phase = 0..31, j = 0..3) are shown in Table 6.

[0433] Table 6: Detailed description of interpolation filter coefficients fC and fG

[0434]

[0435]

[0436] The values ​​of the predicted samples predSamples[x][y] (where x=0..nTbW-1, y=0..nTbH-1) are derived as follows:

[0437] – If predModeIntra is greater than or equal to 34, the following steps, in order, apply:

[0438] 1. The reference sample array ref[x] is specified as follows:

[0439] – The following apply:

[0440] ref[x]=p[-1-refIdx+x][-1-refIdx], where x=0..nTbW+refIdx+1.

[0441] – If intraPredAngle is less than 0, the main reference sample array is extended as follows:

[0442] ref[x]=p[-1-refIdx][-1-refIdx+Min((x*invAngle+256)>>9,nTbH)], where x=-nTbH..-1.

[0443] -otherwise,

[0444] ref[x]=p[-1-refIdx+x][-1-refIdx], where x=nTbW+2+refIdx..refW+refIdx.

[0445] If cIdx is equal to 0, the other samples ref[refW+refIdx+x] (where x=1..(Max(1,nTbW / nTbH)*refIdx+1)) are derived as follows:

[0446] ref[refW+refIdx+x]=p[-1+refW][-1-refIdx].

[0447] 2. The values ​​of the predicted samples predSamples[x][y] (where x=0..nTbW-1, y=0..nTbH-1) are derived as follows:

[0448] –The index variable iIdx and the multiplication factor iFact are derived as follows:

[0449] iIdx=(((y+1+refIdx)*intraPredAngle)>>5)+refIdx,

[0450] iFact=((y+1+refIdx)*intraPredAngle)&31.

[0451] – If cIdx is equal to 0, the following applies:

[0452] – The interpolation filter coefficients fT[j] (where j = 0..3) are derived as follows:

[0453] fT[j]=filterFlag? fG[iFact][j]:fC[iFact][j].

[0454] –The value of the predicted sample predSamples[x][y] is derived as follows:

[0455]

[0456] – Otherwise (cIdx is not equal to 0), depending on the value of iFact, the following applies:

[0457] –If iFact is not equal to 0, the value of the predicted sample predSamples[x][y] is derived as follows:

[0458] predSamples[x][y]=((32-iFact)*ref[x+iIdx+1]+iFact*ref[x+iIdx+2]+16)>>5.

[0459] – Otherwise, the value of the predicted sample predSamples[x][y] is derived as follows:

[0460] predSamples[x][y]=ref[x+iIdx+1].

[0461] Otherwise (predModeIntra is less than 34), the following steps, in order, apply:

[0462] 1. The reference sample array ref[x] is specified as follows:

[0463] – The following apply:

[0464] ref[x]=p[-1-refIdx][-1-refIdx+x], where x=0..nTbH+refIdx+1.

[0465] – If intraPredAngle is less than 0, the main reference sample array is extended as follows:

[0466] ref[x]=p[-1-refIdx+Min((x*invAngle+256)>>9,nTbW)][-1-refIdx], where x=-nTbW..-1.

[0467] -otherwise,

[0468] ref[x]=p[-1-refIdx][-1-refIdx+x], where x=nTbH+2+refIdx..refH+refIdx.

[0469] If cIdx is equal to 0, the other samples ref[refH+refIdx+x] (where x=1..(Max(1,nTbH / nTbW)*refIdx+1)) are derived as follows:

[0470] ref[refH+refIdx+x]=p[-1+refH][-1-refIdx].

[0471] 2. The values ​​of the predicted samples predSamples[x][y] (where x=0..nTbW-1, y=0..nTbH-1) are derived as follows:

[0472] –The index variable iIdx and the multiplication factor iFact are derived as follows:

[0473] iIdx=(((x+1+refIdx)*intraPredAngle)>>5)+refIdx,

[0474] iFact=((x+1+refIdx)*intraPredAngle)&31.

[0475] – If cIdx is equal to 0, the following applies:

[0476] – The interpolation filter coefficients fT[j] (where j = 0..3) are derived as follows:

[0477] fT[j]=filterFlag? fG[iFact][j]:fC[iFact][j].

[0478] –The value of the predicted sample predSamples[x][y] is derived as follows:

[0479]

[0480] – Otherwise (cIdx is not equal to 0), depending on the value of iFact, the following applies:

[0481] –If iFact is not equal to 0, the value of the predicted sample predSamples[x][y] is derived as follows:

[0482] predSamples[x][y]=((32-iFact)*ref[y+iIdx+1]+iFact*ref[y+iIdx+2]+16)>>5.

[0483] – Otherwise, the value of the predicted sample predSamples[x][y] is derived as follows:

[0484] predSamples[x][y]=ref[y+iIdx+1].

[0485] Example 2:

[0486] Detailed description of INTRA_ANGULAR2..INTRA_ANGULAR66 intra prediction modes

[0487] Inputs to this process include:

[0488] – Intra prediction mode predModeIntra;

[0489] – variable refIdx, indicating the intra prediction reference line index;

[0490] – variable nTbW, representing the transform block width;

[0491] – variable nTbH, representing the transform block height;

[0492] – variable refW, indicating the reference sample width;

[0493] – variable refH, indicating the reference sample height;

[0494] – variable nCbW, which represents the coding block width;

[0495] – variable nCbH, which represents the coding block height;

[0496] – variable refFilterFlag, indicating the value of the reference filter flag;

[0497] –Variable cIdx, which represents the color component of the current block.

[0498] – Neighboring samples p[x][y], where x = –1–refIdx, y = –1–refIdx..refH–1, and x = –refIdx..refW–1, y = –1–refIdx.

[0499] The output of this process is the predicted samples predSamples[x][y], where x = 0..nTbW–1 and y = 0..nTbH–1.

[0500] The variable nTbS is set to (Log2(nTbW)+Log2(nTbH))>>1.

[0501] The variable filterFlag is derived as follows:

[0502] – filterFlag is set to 0 if one or more of the following conditions are true:

[0503] –refFilterFlag is equal to 1;

[0504] –refIdx is not equal to 0;

[0505] –IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT.

[0506] – Otherwise, the following applies:

[0507] –The variable minDistVerHor is set to Min(Abs(predModeIntra–50),Abs(predModeIntra–18)).

[0508] –The variable intraHorVerDistThres[nTbS] is shown in Table 7.

[0509] –The variable filterFlag is derived as follows:

[0510] – If minDistVerHor is greater than intraHorVerDistThres[nTbS], filterFlag is set to 1.

[0511] – Otherwise, filterFlag is set to 0.

[0512] Table 7: Detailed description of intraHorVerDistThres[nTbS] for various transform block sizes nTbS

[0513] nTbS=2 nTbS=3 nTbS=4 nTbS=5 nTbS=6 nTbS=7 intraHorVerDistThres[nTbS] 24 14 2 0 0 0

[0514] Figure 6 93 prediction directions are shown, where the dashed directions are associated with wide-angle modes applied to non-square blocks. Table 8 shows a mapping table between predModeIntra and the angle parameter intraPredAngle.

[0515] Table 8: Detailed description of intraPredAngle

[0516]

[0517]

[0518] The inverse angle parameter invAngle is derived from intraPredAngle as follows:

[0519]

[0520] The interpolation filter coefficients fC[phase][j] and fG[phase][j] (where phase = 0..31, j = 0..3) are shown in Table 9.

[0521] Table 9: Detailed description of interpolation filter coefficients fC and fG

[0522]

[0523]

[0524] The values ​​of the predicted samples predSamples[x][y] (where x=0..nTbW-1, y=0..nTbH-1) are derived as follows:

[0525] – If predModeIntra is greater than or equal to 34, the following steps, in order, apply:

[0526] The reference sample array ref[x] is specified as follows:

[0527] – The following apply:

[0528] ref[x]=p[-1-refIdx+x][-1-refIdx], where x=0..nTbW+refIdx+1.

[0529] – If intraPredAngle is less than 0, the main reference sample array is extended as follows:

[0530] ref[x]=p[-1-refIdx][-1-refIdx+Min((x*invAngle+256)>>9,nTbH)], where x=-nTbH..-1.

[0531] -otherwise,

[0532] ref[x]=p[-1-refIdx+x][-1-refIdx], where x=nTbW+2+refIdx..refW+refIdx.

[0533] If cIdx is equal to 0, the other samples ref[refW+refIdx+x] (where x=1..(Max(1,nTbW / nTbH)*refIdx+2)) are derived as follows:

[0534] ref[refW+refIdx+x]=p[-1+refW][-1-refIdx].

[0535] The values ​​of the predicted samples predSamples[x][y] (where x=0..nTbW-1, y=0..nTbH-1) are derived as follows:

[0536] –The index variable iIdx and the multiplication factor iFact are derived as follows:

[0537] iIdx=(((y+1+refIdx)*intraPredAngle)>>5)+refIdx,

[0538] iFact=((y+1+refIdx)*intraPredAngle)&31.

[0539] – If cIdx is equal to 0, the following applies:

[0540] – The interpolation filter coefficients fT[j] (where j = 0..3) are derived as follows:

[0541] fT[j]=filterFlag? fG[iFact][j]:fC[iFact][j].

[0542] –The value of the predicted sample predSamples[x][y] is derived as follows:

[0543]

[0544] – Otherwise (cIdx is not equal to 0), depending on the value of iFact, the following applies:

[0545] –If iFact is not equal to 0, the value of the predicted sample predSamples[x][y] is derived as follows:

[0546] predSamples[x][y]=((32-iFact)*ref[x+iIdx+1]+iFact*ref[x+iIdx+2]+16)>>5.

[0547] – Otherwise, the value of the predicted sample predSamples[x][y] is derived as follows:

[0548] predSamples[x][y]=ref[x+iIdx+1].

[0549] Otherwise (predModeIntra is less than 34), the following steps, in order, apply:

[0550] The reference sample array ref[x] is specified as follows:

[0551] – The following apply:

[0552] ref[x]=p[-1-refIdx][-1-refIdx+x], where x=0..nTbH+refIdx+1.

[0553] – If intraPredAngle is less than 0, the main reference sample array is extended as follows:

[0554] ref[x]=p[-1-refIdx+Min((x*invAngle+256)>>9,nTbW)][-1-refIdx], where x=-nTbW..-1.

[0555] -otherwise,

[0556] ref[x]=p[-1-refIdx][-1-refIdx+x], where x=nTbH+2+refIdx..refH+refIdx.

[0557] If cIdx is equal to 0, the other samples ref[refH+refIdx+x] (where x=1..(Max(1,nTbH / nTbW)*refIdx+2)) are derived as follows:

[0558] ref[refH+refIdx+x]=p[-1+refH][-1-refIdx].

[0559] The values ​​of the predicted samples predSamples[x][y] (where x=0..nTbW-1, y=0..nTbH-1) are derived as follows:

[0560] –The index variable iIdx and the multiplication factor iFact are derived as follows:

[0561] iIdx=(((x+1+refIdx)*intraPredAngle)>>5)+refIdx,

[0562] iFact=((x+1+refIdx)*intraPredAngle)&31.

[0563] – If cIdx is equal to 0, the following applies:

[0564] – The interpolation filter coefficients fT[j] (where j = 0..3) are derived as follows:

[0565] fT[j]=filterFlag? fG[iFact][j]:fC[iFact][j].

[0566] –The value of the predicted sample predSamples[x][y] is derived as follows:

[0567]

[0568] – Otherwise (cIdx is not equal to 0), depending on the value of iFact, the following applies:

[0569] –If iFact is not equal to 0, the value of the predicted sample predSamples[x][y] is derived as follows:

[0570] predSamples[x][y]=((32-iFact)*ref[y+iIdx+1]+iFact*ref[y+iIdx+2]+16)>>5.

[0571] – Otherwise, the value of the predicted sample predSamples[x][y] is derived as follows:

[0572] predSamples[x][y]=ref[y+iIdx+1].

[0573] Yet another embodiment can be described as follows.

[0574] Example 3:

[0575] Detailed description of INTRA_ANGULAR2..INTRA_ANGULAR66 intra prediction modes

[0576] Inputs to this process include:

[0577] – Intra prediction mode predModeIntra;

[0578] – variable refIdx, indicating the intra prediction reference line index;

[0579] – variable nTbW, representing the transform block width;

[0580] – variable nTbH, representing the transform block height;

[0581] – variable refW, indicating the reference sample width;

[0582] – variable refH, indicating the reference sample height;

[0583] – variable nCbW, which represents the coding block width;

[0584] – variable nCbH, which represents the coding block height;

[0585] – variable refFilterFlag, indicating the value of the reference filter flag;

[0586] –Variable cIdx, which represents the color component of the current block.

[0587] – Neighboring samples p[x][y], where x = –1–refIdx, y = –1–refIdx..refH–1, and x = –refIdx..refW–1, y = –1–refIdx.

[0588] The output of this process is the predicted samples predSamples[x][y], where x = 0..nTbW–1 and y = 0..nTbH–1.

[0589] The variable nTbS is set to (Log2(nTbW)+Log2(nTbH))>>1.

[0590] The variable filterFlag is derived as follows:

[0591] – filterFlag is set to 0 if one or more of the following conditions are true:

[0592] –refFilterFlag is equal to 1;

[0593] –refIdx is not equal to 0;

[0594] –IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT.

[0595] – Otherwise, the following applies:

[0596] –The variable minDistVerHor is set to Min(Abs(predModeIntra–50),Abs(predModeIntra–18)).

[0597] –The variable intraHorVerDistThres[nTbS] is shown in Table 10.

[0598] –The variable filterFlag is derived as follows:

[0599] – If minDistVerHor is greater than intraHorVerDistThres[nTbS], filterFlag is set to 1.

[0600] – Otherwise, filterFlag is set to 0.

[0601] Table 10: Detailed description of intraHorVerDistThres[nTbS] for various transform block sizes nTbS

[0602] nTbS=2 nTbS=3 nTbS=4 nTbS=5 nTbS=6 nTbS=7 intraHorVerDistThres[nTbS] 24 14 2 0 0 0

[0603] Figure 6 93 prediction directions are shown, where the dashed directions are associated with wide-angle mode applied to non-square blocks. Table 11 shows a mapping table between predModeIntra and the angle parameter intraPredAngle.

[0604] Table 11: Detailed description of intraPredAngle

[0605]

[0606]

[0607] The inverse angle parameter invAngle is derived from intraPredAngle as follows:

[0608]

[0609] The interpolation filter coefficients fC[phase][j] and fG[phase][j] (where phase = 0..31, j = 0..3) are shown in Table 12.

[0610] Table 12: Detailed description of interpolation filter coefficients fC and fG

[0611]

[0612]

[0613] The values ​​of the predicted samples predSamples[x][y] (where x=0..nTbW-1, y=0..nTbH-1) are derived as follows:

[0614] – If predModeIntra is greater than or equal to 34, the following steps, in order, apply:

[0615] The reference sample array ref[x] is specified as follows:

[0616] – The following apply:

[0617] ref[x]=p[-1-refIdx+x][-1-refIdx], where x=0..nTbW+refIdx+1.

[0618] – If intraPredAngle is less than 0, the main reference sample array is extended as follows:

[0619] ref[x]=p[-1-refIdx][-1-refIdx+Min((x*invAngle+256)>>9,nTbH)], where x=-nTbH..-1.

[0620] -otherwise,

[0621] ref[x]=p[-1-refIdx+x][-1-refIdx], where x=nTbW+2+refIdx..refW+refIdx.

[0622] If cIdx is equal to 0, the other samples ref[refW+refIdx+x] (where x=1..(Max(1,nTbW / nTbH)*refIdx+1)) are derived as follows:

[0623] ref[refW+refIdx+x]=p[-1+refW][-1-refIdx].

[0624] The values ​​of the predicted samples predSamples[x][y] (where x=0..nTbW-1, y=0..nTbH-1) are derived as follows:

[0625] –The index variable iIdx and the multiplication factor iFact are derived as follows:

[0626] iIdx=(((y+1+refIdx)*intraPredAngle)>>5)+refIdx,

[0627] iFact=((y+1+refIdx)*intraPredAngle)&31.

[0628] – If cIdx is equal to 0, the following applies:

[0629] – The interpolation filter coefficients fT[j] (where j = 0..3) are derived as follows:

[0630] fT[j]=filterFlag? fG[iFact][j]:fC[iFact][j].

[0631] –The value of the predicted sample predSamples[x][y] is derived as follows:

[0632]

[0633] – Otherwise (cIdx is not equal to 0), depending on the value of iFact, the following applies:

[0634] –If iFact is not equal to 0, the value of the predicted sample predSamples[x][y] is derived as follows:

[0635] predSamples[x][y]=((32-iFact)*ref[x+iIdx+1]+iFact*ref[x+iIdx+2]+16)>>5.

[0636] – Otherwise, the value of the predicted sample predSamples[x][y] is derived as follows:

[0637] predSamples[x][y]=ref[x+iIdx+1].

[0638] Otherwise (predModeIntra is less than 34), the following steps, in order, apply:

[0639] The reference sample array ref[x] is specified as follows:

[0640] – The following apply:

[0641] ref[x]=p[-1-refIdx][-1-refIdx+x], where x=0..nTbH+refIdx+1.

[0642] – If intraPredAngle is less than 0, the main reference sample array is extended as follows:

[0643] ref[x]=p[-1-refIdx+Min((x*invAngle+256)>>9,nTbW)][-1-refIdx], where x=-nTbW..-1.

[0644] -otherwise,

[0645] ref[x]=p[-1-refIdx][-1-refIdx+x], where x=nTbH+2+refIdx..refH+refIdx.

[0646] If cIdx is equal to 0, the other samples ref[refH+refIdx+x] (where x=1..(Max(1,nTbH / nTbW)*refIdx+1)) are derived as follows:

[0647] ref[refH+refIdx+x]=p[-1+refH][-1-refIdx].

[0648] The values ​​of the predicted samples predSamples[x][y] (where x=0..nTbW-1, y=0..nTbH-1) are derived as follows:

[0649] –The index variable iIdx and the multiplication factor iFact are derived as follows:

[0650] iIdx=(((x+1+refIdx)*intraPredAngle)>>5)+refIdx,

[0651] iFact=((x+1+refIdx)*intraPredAngle)&31.

[0652] – If cIdx is equal to 0, the following applies:

[0653] – The interpolation filter coefficients fT[j] (where j = 0..3) are derived as follows:

[0654] fT[j]=filterFlag? fG[iFact][j]:fC[iFact][j].

[0655] –The value of the predicted sample predSamples[x][y] is derived as follows:

[0656]

[0657] – Otherwise (cIdx is not equal to 0), depending on the value of iFact, the following applies:

[0658] –If iFact is not equal to 0, the value of the predicted sample predSamples[x][y] is derived as follows:

[0659] predSamples[x][y]=((32-iFact)*ref[y+iIdx+1]+iFact*ref[y+iIdx+2]+16)>>5.

[0660] – Otherwise, the value of the predicted sample predSamples[x][y] is derived as follows:

[0661] predSamples[x][y]=ref[y+iIdx+1].

[0662] Embodiment 3 is implemented according to a mechanism whereby if the value of the interpolation filter coefficient is equal to 0 (e.g., (fT[i]==0)?0:fT[i]*ref[x+iIdx+i] or (fT[i]==0)?0:fT[i]*ref[y+iIdx+i]), the reference sample is not used.

[0663] Another embodiment (Embodiment 4) limits the number of padding reference samples for integer slope directivity patterns when:

[0664] – filtered or unfiltered reference samples are copied to the block to be predicted according to the selected mode, or

[0665] – The filtered or unfiltered reference samples are processed by a bypass filter (e.g., a bypass filter with coefficients [0,64,0,0] / 64, equivalent to a bypass filter with coefficients [1] / 1).

[0666] The two cases listed are mathematically equivalent. As described above in the quoted fragment of Section 8.4.5.2.5 "General Intra Sample Prediction" in the VVC draft specification (document JVET-P2001-vE: B. Bross, J. Chen, S. Liu, Y.-K. Wang, "General Video Coding (Draft 7)", output document JVET-P2001 of the 16th JVET meeting in Geneva, Switzerland), a set of integer slope directionality modes is exemplarily defined, and the value of the variable refFilterFlag indicates whether the selected mode is an integer slope mode. Therefore, the variable refFilterFlag can be used to define the number of padding samples (other samples) using the following formula:

[0667] – When predModeIntra is greater than or equal to 34, (refW + refIdx + Max(1, nTbW / nTbH)*refIdx + (refFilterFlag?0:1)); or

[0668] –When predModeIntra is less than 34, (refH+refIdx+Max(1,nTbH / nTbW)*refIdx+(refFilterFlag?0:1)).

[0669] Therefore, Section 8.4.5.2.12 “Detailed description of INTRA_ANGULAR2..INTRA_ANGULAR66 intra prediction modes” in the VVC draft specification (document JVET-P2001-vE: B. Bross, J. Chen, S. Liu, Y.-K. Wang, “Generic Video Coding (Draft 7)”, output document JVET-P2001 from the 16th JVET meeting in Geneva, Switzerland) can be rewritten as follows.

[0670] Example 4:

[0671] Detailed description of INTRA_ANGULAR2..INTRA_ANGULAR66 intra prediction modes

[0672] Inputs to this process include:

[0673] – Intra prediction mode predModeIntra;

[0674] – variable refIdx, indicating the intra prediction reference line index;

[0675] – variable nTbW, representing the transform block width;

[0676] – variable nTbH, representing the transform block height;

[0677] – variable refW, indicating the reference sample width;

[0678] – variable refH, indicating the reference sample height;

[0679] – variable nCbW, which represents the coding block width;

[0680] – variable nCbH, which represents the coding block height;

[0681] – variable refFilterFlag, indicating the value of the reference filter flag;

[0682] –Variable cIdx, which represents the color component of the current block.

[0683] – Neighboring samples p[x][y], where x = –1–refIdx, y = –1–refIdx..refH–1, and x = –refIdx..refW–1, y = –1–refIdx.

[0684] The output of this process is the predicted samples predSamples[x][y], where x = 0..nTbW–1 and y = 0..nTbH–1.

[0685] The variable nTbS is set to (Log2(nTbW)+Log2(nTbH))>>1.

[0686] The variable filterFlag is derived as follows:

[0687] – filterFlag is set to 0 if one or more of the following conditions are true:

[0688] –refFilterFlag is equal to 1;

[0689] –refIdx is not equal to 0;

[0690] –IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT.

[0691] – Otherwise, the following applies:

[0692] –The variable minDistVerHor is set to Min(Abs(predModeIntra–50),Abs(predModeIntra–18)).

[0693] –The variable intraHorVerDistThres[nTbS] is shown in Table 13.

[0694] –The variable filterFlag is derived as follows:

[0695] – If minDistVerHor is greater than intraHorVerDistThres[nTbS], filterFlag is set to 1.

[0696] – Otherwise, filterFlag is set to 0.

[0697] Table 13: Detailed description of intraHorVerDistThres[nTbS] for various transform block sizes nTbS

[0698] nTbS=2 nTbS=3 nTbS=4 nTbS=5 nTbS=6 nTbS=7 intraHorVerDistThres[nTbS] 24 14 2 0 0 0

[0699] Figure 6 93 prediction directions are shown, where the dashed directions are associated with wide-angle mode applied to non-square blocks. Table 14 shows a mapping table between predModeIntra and the angle parameter intraPredAngle.

[0700] Table 14: Detailed description of intraPredAngle

[0701]

[0702]

[0703] The inverse angle parameter invAngle is derived from intraPredAngle as follows:

[0704]

[0705] The interpolation filter coefficients fC[phase][j] and fG[phase][j] (where phase = 0..31, j = 0..3) are shown in Table 15.

[0706] Table 15: Detailed description of interpolation filter coefficients fC and fG

[0707]

[0708]

[0709] The values ​​of the predicted samples predSamples[x][y] (where x=0..nTbW-1, y=0..nTbH-1) are derived as follows:

[0710] – If predModeIntra is greater than or equal to 34, the following steps, in order, apply:

[0711] The reference sample array ref[x] is specified as follows:

[0712] – The following apply:

[0713] ref[x]=p[-1-refIdx+x][-1-refIdx], where x=0..nTbW+refIdx+1.

[0714] – If intraPredAngle is less than 0, the main reference sample array is extended as follows:

[0715] ref[x]=p[-1-refIdx][-1-refIdx+Min((x*invAngle+256)>>9,nTbH)], where x=-nTbH..-1.

[0716] -otherwise,

[0717] ref[x]=p[-1-refIdx+x][-1-refIdx], where x=nTbW+2+refIdx..refW+refIdx.

[0718] If cIdx is equal to 0, the other samples ref[refW+refIdx+x] (where x=1..(Max(1,nTbW / nTbH)*refIdx+(refFilterFlag?0:1))) are derived as follows:

[0719] ref[refW+refIdx+x]=p[-1+refW][-1-refIdx].

[0720] The values ​​of the predicted samples predSamples[x][y] (where x=0..nTbW-1, y=0..nTbH-1) are derived as follows:

[0721] –The index variable iIdx and the multiplication factor iFact are derived as follows:

[0722] iIdx=(((y+1+refIdx)*intraPredAngle)>>5)+refIdx,

[0723] iFact=((y+1+refIdx)*intraPredAngle)&31.

[0724] – If cIdx is equal to 0, the following applies:

[0725] – The interpolation filter coefficients fT[j] (where j = 0..3) are derived as follows:

[0726] fT[j]=filterFlag? fG[iFact][j]:fC[iFact][j].

[0727] –The value of the predicted sample predSamples[x][y] is derived as follows:

[0728]

[0729] – Otherwise (cIdx is not equal to 0), depending on the value of iFact, the following applies:

[0730] –If iFact is not equal to 0, the value of the predicted sample predSamples[x][y] is derived as follows:

[0731] predSamples[x][y]=((32-iFact)*ref[x+iIdx+1]+iFact*ref[x+iIdx+2]+16)>>5.

[0732] – Otherwise, the value of the predicted sample predSamples[x][y] is derived as follows:

[0733] predSamples[x][y]=ref[x+iIdx+1].

[0734] Otherwise (predModeIntra is less than 34), the following steps, in order, apply:

[0735] The reference sample array ref[x] is specified as follows:

[0736] – The following apply:

[0737] ref[x]=p[-1-refIdx][-1-refIdx+x], where x=0..nTbH+refIdx+1.

[0738] – If intraPredAngle is less than 0, the main reference sample array is extended as follows:

[0739] ref[x]=p[-1-refIdx+Min((x*invAngle+256)>>9,nTbW)][-1-refIdx], where x=-nTbW..-1.

[0740] -otherwise,

[0741] ref[x]=p[-1-refIdx][-1-refIdx+x], where x=nTbH+2+refIdx..refH+refIdx.

[0742] If cIdx is equal to 0, the other samples ref[refH+refIdx+x] (where x=1..(Max(1,nTbH / nTbW)*refIdx+(refFilterFlag?0:1))) are derived as follows:

[0743] ref[refH+refIdx+x]=p[-1+refH][-1-refIdx].

[0744] The values ​​of the predicted samples predSamples[x][y] (where x=0..nTbW-1, y=0..nTbH-1) are derived as follows:

[0745] –The index variable iIdx and the multiplication factor iFact are derived as follows:

[0746] iIdx=(((x+1+refIdx)*intraPredAngle)>>5)+refIdx,

[0747] iFact=((x+1+refIdx)*intraPredAngle)&31.

[0748] – If cIdx is equal to 0, the following applies:

[0749] – The interpolation filter coefficients fT[j] (where j = 0..3) are derived as follows:

[0750] fT[j]=filterFlag? fG[iFact][j]:fC[iFact][j].

[0751] –The value of the predicted sample predSamples[x][y] is derived as follows:

[0752]

[0753] – Otherwise (cIdx is not equal to 0), depending on the value of iFact, the following applies:

[0754] –If iFact is not equal to 0, the value of the predicted sample predSamples[x][y] is derived as follows:

[0755] predSamples[x][y]=((32-iFact)*ref[y+iIdx+1]+iFact*ref[y+iIdx+2]+16)>>5.

[0756] – Otherwise, the value of the predicted sample predSamples[x][y] is derived as follows:

[0757] predSamples[x][y]=ref[y+iIdx+1].

[0758] According to embodiment 1, embodiment 2, embodiment 3 or embodiment 4, padding processing is performed on the luma reference sample to further predict the luma block using the directional intra prediction mode. In embodiment 1, embodiment 2, embodiment 3 or embodiment 4, other color formats can replace YCbCr as the input of the video codec. Specifically, when encoding RGB content without converting it to YCbCr, it is recommended / possible to select the green component (G) as luma (for example, cIdx corresponding to component G is equal to 0) and select components (R) and (B) as chroma (for example, cIdx corresponding to component R or component B is not equal to 0). This mechanism is called GBR encoding.

[0759] Figure 13 The present invention provides a flowchart of a method for intra-frame prediction processing of a coding block. Figure 13 A method for intra-frame prediction processing of a coding block is shown. The method comprises:

[0760] (1601) Obtaining a code stream of the coding block;

[0761] (1603) obtaining a value of an intra-frame prediction mode of the coding block according to the code stream of the coding block;

[0762] (1605) Obtaining a value of an index of the coding block according to the code stream;

[0763] (1607) if the value of the index of the coding block is equal to a predefined value, (1609) obtaining a reference sample set including other samples and samples in the reconstructed block;

[0764] (1607) if the value of the index of the coding block is not equal to a predefined value, (1611) obtaining a reference sample set including samples in the reconstructed block but excluding other samples;

[0765] (1613) Obtain prediction values ​​of samples included in the coding block based on the value of the intra-frame prediction mode of the coding block and the reference sample set.

[0766] Figure 14 FIG. 2 shows an encoder 20 provided by an embodiment of the present invention. The encoder includes:

[0767] A first acquiring unit 2001 is configured to acquire a code stream of a coding block;

[0768] A second acquiring unit 2003 is configured to acquire a value of an intra-frame prediction mode of the coding block according to the bitstream of the coding block;

[0769] A third obtaining unit 2005 is configured to obtain a value of an index of the coding block according to the bitstream;

[0770] The fourth acquisition unit 2007 is configured to acquire a reference sample set of the coding block, wherein:

[0771] If the value of the index of the coding block is equal to a predefined value, the reference sample set includes other samples and samples in the reconstructed block;

[0772] If the value of the index of the coding block is not equal to a predefined value, the reference sample set includes samples in the reconstructed block but does not include other samples;

[0773] The fifth obtaining unit 2009 is configured to obtain a prediction value of the samples included in the coding block according to the value of the intra prediction mode of the coding block and the reference sample set.

[0774] Figure 15 FIG. 3 shows a decoder 30 provided by an embodiment of the present invention. The decoder 30 includes:

[0775] A first acquiring unit 3001 is configured to acquire a code stream of a coding block;

[0776] A second acquiring unit 3003 is configured to acquire a value of an intra-frame prediction mode of the coding block according to the bitstream of the coding block;

[0777] A third obtaining unit 3005 is configured to obtain a value of the index of the coding block according to the bitstream;

[0778] The fourth acquisition unit 3007 is configured to acquire a reference sample set of the coding block, wherein:

[0779] If the value of the index of the coding block is equal to a predefined value, the reference sample set includes other samples and samples in the reconstructed block;

[0780] If the value of the index of the coding block is not equal to a predefined value, the reference sample set includes samples in the reconstructed block but does not include other samples;

[0781] The fifth obtaining unit 3009 is configured to obtain a prediction value of the samples included in the coding block according to the value of the intra prediction mode of the coding block and the reference sample set.

[0782] The following describes applications of the encoding method and decoding method shown in the above embodiments and systems using these applications.

[0783] Figure 11FIG3 is a block diagram of a content delivery system 3100 for implementing a content distribution service. The content delivery system 3100 includes a capture device 3102, a terminal device 3106, and optionally a display 3126. The capture device 3102 communicates with the terminal device 3106 via a communication link 3104. The communication link may include the communication channel 13 described above. The communication link 3104 includes, but is not limited to, Wi-Fi, Ethernet, cable, wireless (3G / 4G / 5G), USB, or any combination thereof.

[0784] Capturing device 3102 is used to generate data, and can encode data by the coding method shown in the above embodiment. Optionally, capturing device 3102 can distribute data to a streaming media server (not shown), which encodes data and sends encoded data to terminal device 3106. Capturing device 3102 includes but is not limited to a camera, smart phone or tablet computer, computer or notebook computer, video conferencing system, PDA, vehicle-mounted device or its arbitrary combination etc. For example, capturing device 3102 can include source device 12 described above. When data includes video, the video encoder 20 included in capturing device 3102 can actually perform video coding. When data includes audio (i.e. sound), the audio encoder included in capturing device 3102 can actually perform audio coding. For some actual scenes, capturing device 3102 distributes encoded video data and encoded audio data by multiplexing encoded video data and encoded audio data together. For other actual scenes, for example, in a video conferencing system, encoded audio data and encoded video data are not multiplexed. The capture device 3102 distributes the encoded audio data and the encoded video data to the terminal device 3106 .

[0785] In content delivery system 3100, terminal device 310 receives and regenerates encoded data. Terminal device 3106 may be a device capable of receiving and recovering data, such as a smartphone or tablet 3108, a computer or laptop 3110, a network video recorder (NVR) / digital video recorder (DVR) 3112, a television 3114, a set-top box (STB) 3116, a video conferencing system 3118, a video surveillance system 3120, a personal digital assistant (PDA) 3122, an in-vehicle device 3124, or a combination thereof, or a device capable of decoding the encoded data. For example, terminal device 3106 may include destination device 14 described above. When the encoded data includes video, the video decoder 30 included in the terminal device preferentially performs video decoding. When the encoded data includes audio, the audio decoder included in the terminal device preferentially performs audio decoding processing.

[0786] For terminal devices with a display, such as a smartphone or tablet 3108, a computer or laptop 3110, a network video recorder (NVR) / digital video recorder (DVR) 3112, a television 3114, a personal digital assistant (PDA) 3122, or an in-vehicle device 3124, the terminal device can feed the decoded data to its display. For terminal devices without a display, such as an STB 3116, a video conferencing system 3118, or a video surveillance system 3120, an external display 3126 is connected thereto to receive and display the decoded data.

[0787] When each device in the system performs encoding or decoding, the image encoding device or the image decoding device as shown in the above-described embodiments can be used.

[0788] Figure 12FIG3 is a schematic diagram of an exemplary structure of a terminal device 3106. After the terminal device 3106 receives a stream from the capture device 3102, the protocol processing unit 3202 analyzes the transport protocol of the stream. The protocol includes, but is not limited to, Real Time Streaming Protocol (RTSP), Hyper Text Transfer Protocol (HTTP), HTTP Live Streaming Protocol (HLS), MPEG-DASH, Real-time Transport Protocol (RTP), Real-time Messaging Protocol (RTMP), or any combination thereof.

[0789] After the protocol processing unit 3202 processes the stream, a stream file is generated. The file is output to the demultiplexing unit 3204. The demultiplexing unit 3204 can separate the multiplexed data into encoded audio data and encoded video data. As mentioned above, for other actual scenarios, such as in a video conferencing system, the encoded audio data and the encoded video data are not multiplexed. In this case, the encoded data is sent to the video decoder 3206 and the audio decoder 3208 without passing through the demultiplexing unit 3204.

[0790] Through demultiplexing, a video elementary stream (ES), an audio ES, and optional subtitles are generated. The video decoder 3206 includes the video decoder 30 described in the above embodiment, and decodes the video ES using the decoding method shown in the above embodiment to generate video frames, and feeds this data to the synchronization unit 3212. The audio decoder 3208 decodes the audio ES to generate audio frames, and feeds this data to the synchronization unit 3212. Optionally, the video frames can be stored in a buffer ( Figure 12 Similarly, the audio frame can be stored in a buffer (not shown) before being sent to the synchronization unit 3212. Figure 12 Not shown).

[0791] Synchronization unit 3212 synchronizes video and audio frames and provides the video / audio to video / audio display 3214. For example, synchronization unit 3212 synchronizes the presentation of video information and audio information. The information can be coded in syntax using timestamps associated with the presentation of the decoded audio and visual data and timestamps associated with the transmission of the data stream itself.

[0792] If subtitles are included in the stream, the subtitle decoder 3210 decodes the subtitles, synchronizes the subtitles with the video frames and audio frames, and provides the video / audio / subtitles to the video / audio / subtitle display 3216.

[0793] The present invention is not limited to the above-mentioned system, and the image encoding device or the image decoding device in the above-mentioned embodiments may be included in other systems such as automobile systems.

[0794] Mathematical operators

[0795] The mathematical operators used in this application are similar to those used in the C programming language. However, the present invention precisely defines the results of integer division and arithmetic shift operations, and also defines other operations such as exponentiation and real-valued division. Numbering and counting conventions generally start at 0, for example, "first" is equivalent to the 0th, "second" is equivalent to the 1st, and so on.

[0796] Arithmetic operators

[0797] The arithmetic operators are defined as follows:

[0798]

[0799]

[0800] Logical operators

[0801] The logical operators are defined as follows:

[0802] x&&y Boolean logical "AND" operation of x and y

[0803] x||y Boolean logical OR operation of x and y

[0804] ! Boolean logic "not" operation

[0805] x? y:z If x is true or not equal to 0, find the value of y; otherwise, find the value of z.

[0806] Relational operators

[0807] The relational operators are defined as follows:

[0808] > greater than

[0809] >= greater than or equal to

[0810] < less than

[0811] <= less than or equal to

[0812] == equal to

[0813] ! = Not equal to

[0814] When a relational operator is applied to a grammar element or variable that has been assigned the value "na" (not applicable), the value "na" is treated as a distinct value for that grammar element or variable. The value "na" is considered not equal to any other value.

[0815] Bitwise operators

[0816] The definitions of bitwise operators are as follows:

[0817] & Bitwise AND operation. When operating on integer arguments, the operation is on the two's complement representation of the integer values. When operating on binary arguments, if it has fewer bits than the other argument, the shorter argument is extended by adding more significant bits equal to 0.

[0818] | Bitwise OR operation. When operating on integer arguments, the operation is on the two's complement representation of the integer values. When operating on binary arguments, if it has fewer bits than the other argument, the shorter argument is extended by adding more significant bits equal to 0.

[0819] ^ Bitwise XOR operation. When operating on integer arguments, the operation is on the two's complement representation of the integer values. When operating on binary arguments, if it has fewer bits than the other argument, the shorter argument is extended by adding more significant bits equal to 0.

[0820] x >> y shifts x arithmetically right by y binary digits in the form of a two's complement integer representation. This function is defined only when y is a non - negative integer value. The result of the right shift is that the bit shifted into the most significant bit (MSB) is equal to the MSB of x before the shift operation.

[0821] x << y shifts x arithmetically left by y binary digits in the form of a two's complement integer representation. This function is defined only when y is a non - negative integer value. The result of the left shift is that the bit shifted into the least significant bit (LSB) is equal to 0.

[0822] Assignment operators

[0823] The definitions of arithmetic operators are as follows:

[0824] = Assignment operator

[0825] ++ Increment, i.e., x++ is equivalent to x = x + 1; when used as an array subscript, the value of the variable is evaluated before the increment operation.

[0826] –– Decrement, i.e., x–– is equivalent to x = x - 1; when used as an array subscript, the value of the variable is evaluated before the decrement operation.

[0827] += Adds the specified value, that is, x+=3 is equivalent to x=x+3, and x+=(–3) is equivalent to x=x+(–3).

[0828] –= Decrements the specified value, that is, x–=3 is equivalent to x=x–3, and x–=(–3) is equivalent to x=x–(–3).

[0829] Range Notation

[0830] The following notation is used to specify value ranges:

[0831] x=y..zx takes integer values ​​from y to z (including y and z), where x, y and z are integers and z is greater than y.

[0832] Mathematical functions

[0833] The mathematical function is defined as follows:

[0834]

[0835] Asin(x) is the trigonometric inverse sine function. It operates on the argument x. When x is in the range of –1.0 to 1.0 (inclusive), the output value is in the range of –π÷2 to π÷2 (inclusive) and is expressed in radians.

[0836] Atan(x) is the trigonometric inverse tangent function. It operates on the argument x and produces an output value in the range of –π÷2 to π÷2 (inclusive) expressed in radians.

[0837]

[0838] Ceil(x) The smallest integer greater than or equal to x.

[0839] Clip1 Y (x)=Clip3(0,(1< <BitDepth Y )–1,x)

[0840] Clip1 C (x)=Clip3(0,(1< <BitDepth C )–1,x)

[0841]

[0842] Cos(x) Trigonometric cosine function, operates on parameter x, in radians.

[0843] Floor(x) The largest integer less than or equal to x.

[0844]

[0845] Ln(x) returns the natural logarithm of x (the base-e logarithm, where e is the base constant 2.718281828…).

[0846] Log2(x) is the base 2 logarithm of x.

[0847] Log10(x) is the base 10 logarithm of x.

[0848]

[0849]

[0850] Round(x)=Sign(x)*Floor(Abs(x)+0.5)

[0851]

[0852] Sin(x) trigonometric sine function, operates on parameter x, in radians.

[0853]

[0854] Swap(x,y)=(y,x)

[0855] Tan(x) trigonometric tangent function, operates on parameter x, in radians.

[0856] Operation priority order

[0857] When parentheses are not used to explicitly indicate precedence in an expression, the following rules apply:

[0858] – Operations with higher priority are evaluated before any operations with lower priority.

[0859] – Operations of the same priority are evaluated from left to right.

[0860] The following table describes the priority of operations from highest to lowest. The higher the position in the table, the higher the priority.

[0861] For operators that are also used in the C programming language, the operator precedence order in this specification is the same as that in the C programming language.

[0862] Table: Operation priority is sorted from highest (top of table) to lowest (bottom of table)

[0863]

[0864]

[0865] Text description of the logical operation

[0866] In the text, describe the following logical operation statement in mathematical form:

[0867]

[0868] It can be described in the following way:

[0869] ...as follows / ...the following applies:

[0870] – If condition 0 then statement 0

[0871] – Else if condition 1 then statement 1

[0872] –……

[0873] – Otherwise (informative description of the remaining conditions), statement n.

[0874] Each "if...else, if...else, ..." statement in the text is introduced with "...as follows" or "...the following applies" followed by "if...". The last condition of "if...else, if...else, ..." always has an "else, ...". Statements with "if...else, if...else" in the middle can be identified by matching "...as follows" or "...the following applies" with the final "else...".

[0875] In the text, describe the following logical operation statement in mathematical form:

[0876]

[0877] It can be described in the following way:

[0878] ...as follows / ...the following applies:

[0879] – If all of the following conditions are true, then statement 0:

[0880] –Condition 0a

[0881] – Condition 0b

[0882] – If one or more of the following conditions are met, then statement 1:

[0883] –Condition 1a

[0884] – Condition 1b

[0885] –……

[0886] – Otherwise, statement n.

[0887] In the text, describe the following logical operation statement in mathematical form:

[0888] if(condition 0)

[0889] statement 0

[0890] if(condition 1)

[0891] statement 1

[0892] It can be described in the following way:

[0893] When condition 0, statement 0

[0894] When condition 1, statement 1.

[0895] The embodiments of the encoder 20 and decoder 30, etc. and the functions described herein with reference to the encoder 20 and decoder 30, etc. can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, these functions can be stored as one or more instructions or codes in a computer-readable medium or sent via a communication medium and executed by a hardware-based processing unit. The computer-readable medium can include a computer-readable storage medium corresponding to a tangible medium (such as a data storage medium), or any communication medium that facilitates the transmission of a computer program from one place to another according to a communication protocol, etc. In this way, the computer-readable medium can generally correspond to (1) a non-transitory tangible computer-readable storage medium, or (2) a communication medium such as a signal or carrier wave. The data storage medium can be any available medium that is accessed by one or more computers or one or more processors to retrieve instructions, codes and / or data structures for implementing the techniques described in the present invention. The computer program product can include a computer-readable medium.

[0896] By way of example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory or any other medium that can be used to store the desired program code in the form of instructions or data structures and that can be accessed by a computer. In addition, any connection can be appropriately referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server or other remote source via a coaxial cable, fiber optic cable, twisted pair and digital subscriber line (DSL), or wireless technologies such as infrared, radio and microwave, then the coaxial cable, fiber optic cable, twisted pair and DSL, or wireless technologies such as infrared, radio and microwave are also included in the definition of the above medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals or other transient media, but rather refer to non-transient tangible storage media. Disks and optical disks as used herein include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs) and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs use lasers to reproduce data optically. Combinations of the above should also be included within the scope of computer-readable media.

[0897] Instructions can be executed by one or more processors such as one or more digital signal processors (DSPs), one or more general-purpose microprocessors, one or more application-specific integrated circuits (ASICs), one or more field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, the term "processor" as used herein may refer to any of the above structures or any other structure suitable for implementing the techniques described herein. In addition, in some aspects, the various functions described herein may be provided within dedicated hardware and / or software modules for encoding and decoding, or incorporated into a combined codec. In addition, these techniques may be fully implemented in one or more circuits or logic elements.

[0898] The techniques of this disclosure can be implemented in a variety of devices or apparatuses, including wireless handsets, integrated circuits (ICs), or a set of ICs (e.g., a chipset). The various components, modules, or units described herein are intended to emphasize functional aspects of an apparatus for performing the disclosed techniques, but they do not necessarily need to be implemented by different hardware units. In practice, as described above, the various units can be combined in a codec hardware unit in conjunction with appropriate software and / or firmware, or provided by a collection of interoperable hardware units including one or more processors as described above.

[0899] The present invention discloses the following 14 aspects.

[0900] 1. A method for intra-frame prediction processing of a coding block, wherein the method comprises:

[0901] Obtaining a code stream of the coding block;

[0902] Obtaining a value of an intra-frame prediction mode of the coding block according to the code stream of the coding block;

[0903] Obtaining a value of an index (e.g., cIdx) of the coding block (e.g., based on the code stream of the coding block);

[0904] Obtaining a reference sample set of the coding block, wherein when a value of the index of the coding block is equal to a predefined value (for example, the predefined value is 0), the reference sample set includes samples in the reconstructed block and padding samples;

[0905] According to the value of the intra prediction mode of the coding block and the reference sample set, a prediction value of a sample (or a predicted sample) included in the coding block is obtained.

[0906] 2. An aspect of the method according to aspect 1, wherein, when the value of the index of the coding block is not equal to the predefined value (for example, the predefined value is 0), the reference sample set includes samples in the reconstructed block but does not include padding samples.

[0907] 3. An aspect of the method according to aspect 1 or 2, wherein the value of the index represents a color component of the coding block.

[0908] 4. An aspect of the method according to any one of aspects 1 to 3, wherein, when the value of the index of the coding block is equal to the predefined value (for example, the predefined value is 0), the value of the index corresponds to the luminance component of the coding block.

[0909] 5. An aspect of the method according to any one of aspects 1 to 4, wherein, when the value of the index of the coding block is not equal to the predefined value (for example, the predefined value is 0), the value of the index corresponds to the chrominance component of the coding block.

[0910] 6. An aspect of the method according to any one of aspects 1 to 5, wherein the value of the intra-frame prediction mode of the coding block indicates that the coding block uses a directional intra-frame prediction mode.

[0911] 7. An aspect of the method according to any one of aspects 1 to 5, wherein the number of padding (other) samples is obtained based on whether the selected directional mode is a positive slope mode (in some examples, the value of refFilterFlag indicates whether the selected directional mode is an integer slope mode).

[0912] 8. An aspect of the method according to clause 7, wherein the filling (other) samples (or the number of the filling samples) are obtained according to the following formula:

[0913] – When predModeIntra is greater than or equal to 34, (refW+refIdx+Max(1,nTbW / nTbH)*refIdx+(refFilterFlag?0:1));

[0914] or

[0915] –When predModeIntra is less than 34, (refH+refIdx+Max(1,nTbH / nTbW)*refIdx+(refFilterFlag?0:1)).

[0916] 9. An encoder (20), wherein the encoder (20) comprises a processing circuit for performing the method according to any one of aspects 1 to 8.

[0917] 10. A decoder (30), wherein the decoder (30) comprises a processing circuit for performing the method according to any one of aspects 1 to 8.

[0918] 11. A computer program product comprising a program code, wherein when the program code is executed on a computer or a processor, the program code is configured to perform the method according to any one of the above aspects.

[0919] 12. A decoder, wherein the decoder comprises:

[0920] one or more processors;

[0921] A non-transitory computer-readable storage medium is coupled to the one or more processors and stores a program executed by the one or more processors, wherein when the program is executed by the one or more processors, the decoder is used to perform the method according to any one of the above aspects.

[0922] 13. An encoder, wherein the encoder comprises:

[0923] one or more processors;

[0924] A non-transitory computer-readable storage medium is coupled to the one or more processors and stores a program executed by the one or more processors, wherein when the program is executed by the one or more processors, the encoder is used to perform the method according to any one of the above aspects.

[0925] 14. A non-transitory computer-readable medium comprising a program code, wherein, when the program code is executed by a computer device, the program code causes the computer device to perform the method according to any one of the above aspects.

Claims

1. A method for intra-frame prediction processing of a coding block, characterized in that: The method comprises: (1601) Obtaining a code stream of the coding block; (1603) Obtaining a value of an intra-frame prediction mode of the coding block according to the code stream of the coding block; (1605) Obtaining, according to the code stream, a value of an index of the coding block, where the value of the index represents a color component of the coding block; (1607) If the value of the index of the coding block is equal to a predefined value, (1609) obtaining a reference sample set of the coding block, wherein the reference sample set includes other samples and samples in the reconstructed block; (1607) If the value of the index of the coding block is not equal to the predefined value, (1611) obtaining a reference sample set of the coding block, wherein the reference sample set includes samples in the reconstructed block but does not include other samples; (1613) obtaining a prediction value of a sample included in the coding block according to the value of the intra prediction mode of the coding block and the reference sample set; The other samples are padding reference samples, which are generated by copying the latest available reference samples belonging to the main reference side of the coding block, or are padded with a first predefined value.

2. The method according to claim 1, characterized in that The index of the coding block is cIdx, and the predefined value is 0.

3. The method according to claim 1 or 2, characterized in that When the value of the index of the coding block is equal to the predefined value, the value of the index corresponds to the luminance component of the coding block.

4. The method according to claim 1 or 2, characterized in that When the value of the index of the coding block is not equal to the predefined value, the value of the index corresponds to the chrominance component of the coding block.

5. The method according to claim 1, wherein The value of the intra prediction mode of the coding block indicates that the coding block uses a directional intra prediction mode.

6. The method according to claim 5, characterized in that The number of the other samples is obtained according to whether the selected directional intra prediction mode is an integer slope mode.

7. The method according to claim 6, characterized in that The value of refFilterFlag indicates whether the selected directional intra prediction mode is an integer slope mode.

8. The method according to claim 7, characterized in that The number of other samples is obtained according to the following formula: When the value of the intra prediction mode predModeIntra is greater than or equal to 34, the number of other samples is: refW + refIdx + Max( 1, nTbW / nTbH ) * refIdx + ( refFilterFlag ? 0 : 1 ); or When the value of the intra prediction mode predModeIntra is less than 34, the number of other samples is: refH +refIdx + Max(1, nTbH / nTbW)*refIdx + (refFilterFlag?0:1); Among them, refW represents the reference sample width; refIdx represents the intra-frame prediction reference line index; nTbW represents the reconstructed residual block width; nTbH represents the reconstructed residual block height; refH represents the reference sample height; refFilterFlag ? 0 : 1 is used to indicate whether the selected directional intra-frame prediction mode is an integer slope mode, so as to set the value to 0 or 1.

9. An encoder (20), characterized in that The encoder (20) comprises a processing circuit for performing the method according to any one of claims 1 to 8.

10. A decoder (30), characterized in that The decoder (30) comprises a processing circuit for performing the method according to any one of claims 1 to 8.

11. A computer program product comprising program code, characterized in that When the program code is executed on a computer or a processor, the program code is used to perform the method according to any one of claims 1 to 8.

12. A decoder (30), characterized in that The decoder (30) comprises: one or more processors; A non-transitory computer-readable storage medium is coupled to the one or more processors and stores a program executed by the one or more processors, wherein when the program is executed by the one or more processors, the decoder is configured to perform the method according to any one of claims 1 to 8.

13. An encoder (20), characterized in that The encoder (20) comprises: one or more processors; A non-transitory computer-readable storage medium, coupled to the one or more processors and storing a program executed by the one or more processors, wherein when the program is executed by the one or more processors, the encoder is configured to perform the method according to any one of claims 1 to 8.

14. A non-transitory computer-readable medium comprising a program code, characterized in that When the program code is executed by a computer device, the computer device is caused to perform the method according to any one of claims 1 to 8.

15. An encoder (20), characterized in that The encoder (20) comprises: A first acquisition unit (2001) is used to acquire a code stream of a coding block; A second acquiring unit (2003) is configured to acquire a value of an intra-frame prediction mode of the coding block according to the code stream of the coding block; A third acquiring unit (2005) is configured to acquire, according to the bitstream, a value of an index of the coding block, wherein the value of the index represents a color component of the coding block; The fourth acquisition unit (2007) is configured to acquire a reference sample set of the coding block, wherein: If the value of the index of the coding block is equal to a predefined value, the reference sample set includes other samples and samples in the reconstructed block; If the value of the index of the coding block is not equal to a predefined value, the reference sample set includes samples in the reconstructed block but does not include other samples; a fifth acquiring unit (2009), configured to acquire, according to the value of the intra prediction mode of the coding block and the reference sample set, a prediction value of a sample included in the coding block; The other samples are padding reference samples, which are generated by copying the latest available reference samples belonging to the main reference side of the coding block, or are padded with a first predefined value.

16. A decoder (30), characterized in that The decoder (30) comprises: A first acquisition unit (3001) is used to acquire a code stream of a coding block; A second acquiring unit (3003) is configured to acquire a value of an intra-frame prediction mode of the coding block according to the code stream of the coding block; A third acquiring unit (3005) is configured to acquire, according to the code stream, a value of an index of the coding block, wherein the value of the index represents a color component of the coding block; The fourth acquisition unit (3007) is used to acquire a reference sample set of the coding block, wherein: If the value of the index of the coding block is equal to a predefined value, the reference sample set includes other samples and samples in the reconstructed block; If the value of the index of the coding block is not equal to a predefined value, the reference sample set includes samples in the reconstructed block but does not include other samples; A fifth acquiring unit (3009) is configured to acquire, based on the value of the intra prediction mode of the coding block and the reference sample set, a prediction value of a sample included in the coding block; The other samples are padding reference samples, which are generated by copying the latest available reference samples belonging to the main reference side of the coding block, or are padded with a first predefined value.

Images