Method and apparatus for interpolation filtering for predictive coding

Abstract

The present invention provides an apparatus and method for encoding and decoding. [Solution] This disclosure relates to intra or interprediction for video coding and decoding. For that purpose, the apparatus and method acquire a reference sample and subpixel offset values. To acquire predicted sample values, a subpixel 4-tap interpolation filter is used to filter the reference sample. The filter coefficients c0, c1, c2, and c3 of the subpixel 4-tap interpolation filter are: The image satisfies the condition JPEG2026091851000062.jpg10170, where p is the fractional part of the subpixel offset value.

Description

[Technical Field]

[0001] Embodiments of this application (disclosure) generally relate to the field of picture processing, more particularly to image and / or video coding and decoding, and in particular to methods and apparatus for intra / inter prediction. [Background technology]

[0002] Video coding (video encoding and video decoding) is used in a wide range of digital video applications, such as broadcast digital TV, video transmission over the internet and mobile networks, real-time conversational applications like video chat and video conferencing, DVD and Blu-ray® discs, video content collection and editing systems, and camcorders in security applications.

[0003] Even the amount of video data required to depict a relatively short video can be substantial, which can pose challenges when data is streamed or otherwise transmitted across communication networks with limited bandwidth. Therefore, video data is generally compressed before being transmitted across modern telecommunications networks. Video size can also be a concern when video is stored on storage devices, as memory resources can be limited. Video compression devices often encode the video data at the source using software and / or hardware before transmission or storage, thereby reducing the amount of data required to represent the digital video image. The compressed data is then received at the destination by a video decompression device that decodes the video data. As network resources become limited and the demand for higher video quality constantly increases, improved compression and decompression techniques that improve compression ratios with little sacrifice in picture quality are desirable. [Overview of the Initiative] [Means for solving the problem]

[0004] Embodiments of this application provide apparatus and methods for encoding and decoding according to independent and dependent claims.

[0005] The above and other objectives are achieved by the subject matter of the independent and dependent claims. Further implementations are evident from the dependent claims, description, and figures.

[0006] Embodiments of the present invention are defined by the features of the independent claims, and more advantageous implementations of these embodiments are defined by the features of the dependent claims.

[0007] According to one aspect of the present disclosure, a method is provided for intra or interpredictive processing of a video frame, the method comprising the steps of: obtaining a reference sample; obtaining a subpixel offset value; and filtering the reference sample using a subpixel 4-tap interpolation filter to obtain a predictive sample value, wherein the filter coefficients of the subpixel 4-tap interpolation filter are

[0008]

number

[0009] The following conditions are met: p is the fractional part of the subpixel offset value, and c0, c1, c2, and c3 are the filter coefficients of the subpixel 4-tap interpolation filter.

[0010] In one exemplary implementation, a reference sample may be referenced as ref[x], which corresponds to an array of reference samples. ref[x]=p[-1-refIdx+x][-1-refIdx](where x=0..nTbW+refIdx+1) Here, "p" refers to a 2D xy array p[x][y] containing the sample values. The number of reference samples used may be at least one. In another example, the number of reference samples may be four.

[0011] In an exemplary implementation, the obtained sub-pixel offset value may be referred to as (y + 1 + refIdx)*intraPredAngle where "IntraPredAngle" is the value of the intra prediction angle.

[0012] In an exemplary implementation, the predicted sample value "predSamples[x][y]" may be

[0013] [Number]

[0014] obtained by, where fT[i] refers to the filter coefficient. These coefficients may be, respectively, the luma or chroma filter coefficients for inter prediction, referred to as fG and fC.

[0015] The choice of whether the filter coefficient is luma or chroma may be made, for example, fT[j] = filterFlag? fG[iFact][j] : fC[iFact][j] by using and setting the flag "filterFlag", as iFact = ((y + 1 + refIdx)*intraPredAngle)&31 and

[0016] The value "31" refers to the fractional part of the sub-pixel offset value and may take on other values different from "31". The values of the filter coefficients fG (luma) and / or fC (chroma) can be obtained analytically on-the-fly using the above analytical expressions for the filter coefficients of the 4-tap filter. Thus, the filter coefficients are defined according to the sub-pixel offset value.

[0017] Therefore, the filter coefficients are obtained without accessing the individual values ​​of the filter coefficients from the LUT, but rather by calculation.

[0018] Alternatively, the filter coefficients may be calculated using the above formula, and the values ​​may be stored in the LUT.

[0019] According to one aspect of this disclosure, the filter coefficients of a subpixel 4-tap interpolation filter are given in the following table, i.e.,

[0020] [Table 1A]

[0021] [Table 1B]

[0022] It is defined within the following.

[0023] According to one aspect of the present disclosure, a method is provided for intra or interpredictive processing of a video frame, the method comprising the steps of: obtaining a reference sample; obtaining a subpixel offset value; and filtering the reference sample using a subpixel 4-tap interpolation filter to obtain a predictive sample value, wherein the filter coefficients of the subpixel 4-tap interpolation filter are as follows, i.e.,

[0024] [Table 2]

[0025] It is defined as follows, where p is the fractional part of the subpixel offset value, and c0, c1, c2, and c3 are filter coefficients.

[0026] In one exemplary implementation, the filter coefficient values ​​can be stored in the LUT. This means that each filter coefficient value will be fetched from the LUT for subpixel interpolation filtering.

[0027] Figure 23 shows a flowchart of the intra / inter prediction process. In step 2301, a reference sample is acquired. The reference sample may consist of one or more reference samples. In step 2302, the subpixel offset value is then acquired. In step 2303, the reference sample is then filtered by the filter coefficient {c i The image is filtered using a subpixel interpolation filter with} (where i=0, ..., 3). This is called a 4-tap filter and / or 4-point filter.

[0028] Filter coefficients can be obtained from an analytical representation using a given fractional part p of the subpixel offset value. Alternatively, filter coefficients can be obtained from a LUT by referencing the p value.

[0029] According to one aspect of this disclosure, the bit of p is set to 0.

[0030] According to one aspect of the present disclosure, p is incremented by 1, then the lower bit of the incremented p is set to 0, and then p is used to obtain the filter coefficient.

[0031] According to one aspect of this disclosure, obtaining filter coefficients is

[0032]

number

[0033] The procedure includes a step of calculating the filter coefficients according to the previous filter coefficients c. i From the filter coefficient c i+1This can offer the advantage of not having to calculate the filter coefficients. This could further reduce the time required to calculate the filter coefficients.

[0034] According to one aspect of the present disclosure, a video coding method is provided, comprising an intra-predictive processing of a block of video frames, the method comprising filtering a reference sample of a video frame using a subpixel 4-tap interpolation filter, wherein the subpixel 4-tap interpolation filter is a combination of filters, and at least one of the filters has a filter coefficient.

[0035]

number

[0036] The following conditions are met: p is the fractional sample position of the subpixel offset, and c0, c1, c2, and c3 are filter coefficients.

[0037] Figure 24 shows a flowchart of video coding. In step 2401, an intra-predictive process is performed by filtering the reference samples of the block using a subpixel 4-tap interpolation filter.

[0038] According to one aspect of the present disclosure, the subpixel 4-tap interpolation filter is a smoothing filter which is a parallel combination of two phase-dependent linear filters and a low-pass filter.

[0039] According to one aspect of the present disclosure, the subpixel 4-tap interpolation filter is a sharpening filter which is a parallel combination of a linear filter that depends on one phase and an intensity-variable high-pass filter.

[0040] According to one aspect of this disclosure, the intensity-variable high-pass filter is a phase-independent high-pass finite impulse response FIR filter whose output is multiplied by a phase-dependent intensity parameter.

[0041] According to any one of the prior embodiments of this disclosure, the lower bit of p is set to 0.

[0042] According to one aspect of the present disclosure, p is incremented by 1, then the lower bits of the incremented value are set to 0, and then this value is used to derive the interpolation filter coefficient.

[0043] According to one aspect of this disclosure, a clipping operation is performed on the output of an intensity-variable high-pass filter.

[0044] According to one aspect of the present disclosure, an apparatus for intra or interpredictive processing of video frames is provided, the apparatus comprising a reference sample acquisition unit configured to acquire a reference sample, a subpixel offset value acquisition unit configured to acquire a subpixel offset value, and a subpixel 4-tap interpolation filter configured to filter the reference sample to acquire a predictive sample value, wherein the filter coefficients of the subpixel 4-tap interpolation filter are

[0045]

number

[0046] The following conditions are met: p is the fractional part of the subpixel offset value, and c0, c1, c2, and c3 are the filter coefficients of the subpixel 4-tap interpolation filter.

[0047] According to one aspect of this disclosure, the filter coefficients of a subpixel 4-tap interpolation filter are given in the following table, i.e.,

[0048] [Table 3A]

[0049] [Table 3B]

[0050] It is defined within the following.

[0051] According to one aspect of the present disclosure, an apparatus for intra- or inter-predictive processing of video frames is provided, the apparatus comprising a reference sample acquisition unit configured to acquire a reference sample, a subpixel offset value acquisition unit configured to acquire a subpixel offset value, and a subpixel 4-tap interpolation filter configured to filter the reference sample to acquire a predictive sample value, wherein the filter coefficients of the subpixel 4-tap interpolation filter are as follows, i.e.,

[0052] [Table 4]

[0053] It is defined as follows, where p is the fractional part of the subpixel offset value, and c0, c1, c2, and c3 are filter coefficients.

[0054] Figure 25 shows a schematic diagram of the intra / inter processing module 2500, which includes a reference sample acquisition unit 2510, a subpixel offset value acquisition unit 2520, and a subpixel interpolation filter 2530.

[0055] According to one aspect of the present disclosure, an apparatus for video coding is provided, comprising an intra-prediction processing unit configured to intra-predict blocks of video frames, and a subpixel 4-tap interpolation filter configured to filter reference samples of video frames, wherein the subpixel 4-tap interpolation filter is a combination of filters, and at least one of the filters has a filter coefficient.

[0056]

number

[0057] The following conditions are met: p is the fractional sample position of the subpixel offset, and c0, c1, c2, and c3 are filter coefficients.

[0058] Figure 26 shows a schematic diagram of a video coding module 2600, which includes an intra-predictive processing unit 2610 and a subpixel interpolation filter 2620.

[0059] According to one aspect of the present disclosure, an encoder (20) is provided, comprising a processing circuit configuration for performing a method according to any one of the prior aspects of the present disclosure.

[0060] According to one aspect of the present disclosure, a decoder (30) is provided having a processing circuit configuration for performing a method according to any one of the prior aspects of the present disclosure.

[0061] According to one aspect of this disclosure, a computer program product is provided which comprises program code for performing a method according to any one of the prior aspects of this disclosure.

[0062] According to one aspect of the present disclosure, a decoder is provided comprising one or more processors and a non-temporary computer-readable storage medium coupled to the processors and storing a program for execution by the processors, wherein the program, when executed by the processors, is configured to perform a method according to any one of the prior aspects of the present disclosure.

[0063] According to one aspect of the present disclosure, an encoder is provided comprising one or more processors and a non-temporary computer-readable storage medium coupled to the processors and storing a program for execution by the processors, wherein the program, when executed by the processors, configures the encoder to perform a method according to any one of the prior aspects of the present disclosure.

[0064] Any of the preceding embodiments of this disclosure may offer the advantage of performing intra / interpretation of video frames at high speed because the filter coefficients of the interpolation filter are obtained analytically, i.e., the coefficients are calculated on the fly. This avoids storing the coefficients in a look-up table (LUT), which requires access time to read the coefficients from memory.

[0065] Therefore, the prediction becomes more efficient and requires less memory than necessary. This also enables a low-cost implementation of the prediction. Furthermore, the analytical filter coefficient {c i Since the expression is linear at fractional sample position p and involves division by 2, each operation can be efficiently performed by employing fast, low-level bitwise operations. The time required to perform the bitwise operations and to calculate the filter coefficients is shorter than the time required to access the stored coefficients from the LUT. Therefore, latency is reduced.

[0066] Furthermore, the specific analytical structure of the filter coefficients can offer the advantage of a low-complexity implementation of the filter unit. In addition, the filter response (i.e., frequency response) for different subpixel positions matches in terms of magnitude and phase, avoiding artifacts in the response, especially at high frequencies. The linearity of the filter coefficients can offer the advantage of hardware reuse.

[0067] Details of one or more embodiments are described in the accompanying drawings and the following description. Other features, purposes, and advantages will become apparent from the description, drawings, and claims.

[0068] The present invention can be implemented in hardware and / or software.

[0069] For clarity, any one of the embodiments described above may be combined with any one or more of the other embodiments described above to create new embodiments within the scope of this disclosure.

[0070] These and other features will be more clearly understood from the following modes for carrying out the invention, which will be understood together with the attached drawings and claims.

[0071] Embodiments of the present invention will be described in more detail below with reference to the accompanying figures and drawings. [Brief explanation of the drawing]

[0072] [Figure 1A] This is a block diagram showing an example of a video coding system configured to implement embodiments of the present invention. [Figure 1B] This block diagram shows another example of a video coding system configured to carry out embodiments of the present invention. [Figure 2] This is a block diagram showing an example of a video encoder configured to carry out embodiments of the present invention. [Figure 3] This is a block diagram showing an exemplary structure of a video decoder configured to carry out embodiments of the present invention. [Figure 4] This is a block diagram showing an example of an encoding or decoding device. [Figure 5] This is a block diagram showing another example of an encoding or decoding device. [Figure 6] This figure shows the angular intra-prediction direction and associated intra-prediction mode in HEVC. [Figure 7] This figure shows the angular intra-prediction direction and associated intra-prediction mode in JEM. [Figure 8] This figure shows the angular intra-prediction direction and associated intra-prediction modes in VTM-3.0 and the VVC specification draft v.3. [Figure 9]This figure shows an example of a filter combination for smoothing, where a set of reference samples is input to an interpolation filter. [Figure 10] This figure shows an exemplary combination of filters for sharpening interpolation filtering, including clipping. [Figure 11] This figure shows an exemplary combination of filters for sharpening interpolation filtering with alternative positions for clipping. [Figure 12A] This figure shows an exemplary combination of filters for sharpening interpolation filtering using filters of adjustable intensity. [Figure 12B] This figure shows another exemplary combination of filters for sharpening interpolation filtering, using a filter with adjustable intensity. [Figure 13] This figure shows various combinations of LUT-based and analytical representations of interpolation filter coefficients. [Figure 14] This figure shows various combinations of LUT-based and analytical representations of interpolation filter coefficients. [Figure 15] This figure shows various combinations of LUT-based and analytical representations of interpolation filter coefficients. [Figure 16] This figure shows various combinations of LUT-based and analytical representations of interpolation filter coefficients. [Figure 17] This figure shows various combinations of LUT-based and analytical representations of interpolation filter coefficients. [Figure 18] This figure shows various combinations of LUT-based and analytical representations of interpolation filter coefficients. [Figure 19] This figure shows various combinations of LUT-based and analytical representations of interpolation filter coefficients. [Figure 20] This figure shows various combinations of LUT-based and analytical representations of interpolation filter coefficients. [Figure 21] This figure shows various combinations of LUT-based and analytical representations of interpolation filter coefficients. [Figure 22] This figure shows an example implementation of a coefficient calculation method. [Figure 23] This is a flowchart for intra / internet prediction processing. [Figure 24] This is a flowchart for video coding. [Figure 25] This is a block diagram of an intra / inter prediction processing module, including a reference sample acquisition unit, a subpixel offset value acquisition unit, and a subpixel interpolation filter. [Figure 26] This is a block diagram of a video coding module, including an interpretation processing unit and a subpixel interpolation filter. [Modes for carrying out the invention]

[0073] In the following, unless otherwise explicitly specified, the same reference numeral refers to the same or at least functionally equivalent feature.

[0074] The following description includes references to accompanying drawings that form part of the present disclosure and illustrate specific embodiments of the present invention or specific embodiments of the present invention in which they may be used. It is understood that embodiments of the present invention may be used in other embodiments and may include structural or logical modifications not shown in the drawings. Accordingly, the embodiments for carrying out the invention described below should not be understood in an restrictive sense, and the scope of the invention is defined by the accompanying claims.

[0075] For example, disclosures relating to a method described may also apply to a corresponding device or system configured to perform the Method, and vice versa. For example, if one or more specific method steps are described, the corresponding device may include one or more units for performing the one or more method steps described (e.g., one unit performing one or more steps, or multiple units each performing one or more of the steps), such as functional units, even if such one or more units are not explicitly described or shown in the figures. On the other hand, if a particular device is described based on one or more units, such as functional units, the corresponding method may include one step for performing the function of one or more units (e.g., one step performing the function of one or more units, or multiple steps each performing one or more of the functions of the units), even if such one or more steps are not explicitly described or shown in the figures. Furthermore, unless otherwise specifically stated, it is understood that the features of the various exemplary embodiments and / or aspects described herein may be combined with each other.

[0076] Video coding typically refers to the processing of a sequence of pictures that form a video or video sequence. Instead of the term "picture," the terms "frame" or "image" may be used synonymously in the field of video coding. Video coding (or coding in general) comprises two parts: video encoding and video decoding. Video encoding is performed on the source side and typically involves processing the original video picture (e.g., by compression) to reduce the amount of data required to represent the video picture (for more efficient storage and / or transmission). Video decoding is performed on the destination side and typically involves the reverse processing compared to the encoder, for the purpose of reconstructing the video picture. Embodiments referring to "coding" of a video picture (or picture in general) are understood to relate to the "encoding" or "decoding" of the video picture or its respective video sequence. The combination of the encoding and decoding parts is also called a codec (coding and decoding).

[0077] In the case of reversible video coding, the original video picture can be reconstructed, meaning that (assuming there is no transmission loss or other data loss during storage or transmission) the reconstructed video picture will have the same quality as the original video picture. In the case of irreversible video coding, further compression is performed, for example by quantization, to reduce the amount of data representing the video picture, and the video picture may not be able to be completely reconstructed in the decoder, meaning that the quality of the reconstructed video picture is lower, or worse, compared to the quality of the original video picture.

[0078] Some video coding standards belong to the group of “irreversible hybrid video codecs” (i.e., combining spatial and temporal prediction in the sample domain with 2D transform coding to apply quantization in the transform domain). Each picture in a video sequence is typically divided into a set of non-overlapping blocks, and coding is typically performed at the block level. In other words, in the encoder, video is typically processed, or coded, at the block (video block) level by, for example, using spatial (in-picture) and / or temporal (inter-picture) prediction to generate prediction blocks, subtracting the prediction blocks from the current block (the block currently being processed / to be processed) to obtain residual blocks, transforming the residual blocks, and quantizing the residual blocks in the transform domain to reduce (compress) the amount of data to be transmitted. On the other hand, in the decoder, the reverse processing compared to the encoder is applied to the coded or compressed blocks in order to reconstruct the current block for representation. Furthermore, the encoder replicates the decoder processing loop so that both generate the same predictions (e.g., intra-predictions and inter-predictions) and / or reconstructions in order to process, or code, subsequent blocks.

[0079] Hereinafter, embodiments of the video coding system 10, video encoder 20, and video decoder 30 will be described with reference to Figures 1 to 3.

[0080] Figure 1A is a schematic block diagram showing an exemplary coding system 10 that may utilize the techniques of this application, for example, a video coding system 10 (or short coding system 10). The video encoder 20 (or short encoder 20) and video decoder 30 (or short decoder 30) of the video coding system 10 represent examples of devices that may be configured to perform the techniques described in the various examples in this application.

[0081] As shown in Figure 1A, the coding system 10 includes a source device 12 configured to provide coded picture data 21 to a destination device 14 for decoding coded picture data 13.

[0082] The source device 12 includes an encoder 20 and may additionally, i.e., optionally, include a picture source 16, a preprocessor (or preprocessing unit) 18, for example, a picture preprocessor 18, and a communication interface or communication unit 22.

[0083] The picture source 16 may comprise, or may comprise, any kind of picture capture device, e.g., a camera for capturing real-world pictures, and / or any kind of picture generation device, e.g., a computer graphics processor for generating computer-animated pictures, or any other kind of device for acquiring and / or providing real-world pictures, computer-generated pictures (e.g., screen content, virtual reality (VR) pictures), and / or any combination thereof (e.g., augmented reality (AR) pictures). The picture source may comprise any kind of memory or storage for storing any of the pictures described above.

[0084] In contrast to the processing performed by the preprocessor 18 and the preprocessing unit 18, the picture or picture data 17 is sometimes referred to as the raw picture or raw picture data 17.

[0085] The preprocessor 18 is configured to receive (raw) picture data 17, perform preprocessing on the picture data 17 to obtain a preprocessed picture 19 or preprocessed picture data 19. The preprocessing performed by the preprocessor 18 may include, for example, cropping, color format conversion (e.g., from RGB to YCbCr), color correction, or denoising. It can be understood that the preprocessing unit 18 may be an optional component.

[0086] The video encoder 20 is configured to receive pre-processed picture data 19 and provide encoded picture data 21 (further details are described below, for example, based on Figure 2).

[0087] The communication interface 22 of the source device 12 may be configured to receive encoded picture data 21 and transmit the encoded picture data 21 (or any further processed version thereof) to another device, such as the destination device 14 or any other device, via the communication channel 13 for storage or direct reconstruction.

[0088] The destination device 14 includes a decoder 30 (for example, a video decoder 30) and may additionally, i.e., optionally, include a communication interface or communication unit 28, a post-processor 32 (or post-processing unit 32), and a display device 34.

[0089] The communication interface 28 of the destination device 14 is configured to receive encoded picture data 21 (or any further processed version thereof) from, for example, directly from the source device 12 or from any other source, for example, a storage device, for example, an encoded picture data storage device, and to provide the encoded picture data 21 to the decoder 30.

[0090] Communication interfaces 22 and 28 may be configured to transmit or receive encoded picture data 21 or encoded data 13 via a direct communication link between the source device 12 and the destination device 14, for example, via a direct wired connection or a direct wireless connection, or via any type of network, for example, a wired network or a wireless network or any combination thereof, or any type of private network and public network or any type of combination thereof.

[0091] The communication interface 22 may be configured to package the encoded picture data 21 in an appropriate format, for example, in a packet, and / or to process the encoded picture data using any kind of transmit encoding or transmit processing for transmission over a communication link or communication network.

[0092] A communication interface 28 that forms the counterpart to communication interface 22 may be configured, for example, to receive transmitted data and process the transmitted data using any kind of corresponding transmit decoding or transmit processing and / or depackaging to obtain encoded picture data 21.

[0093] Both communication interfaces 22 and 28 may be configured as unidirectional or bidirectional communication interfaces, as indicated by the arrows to communication channel 13 in Figure 1A, pointing from source device 12 to destination device 14, and may be configured to send and receive messages to set up a connection, for example, to recognize, respond to, and exchange any other information relating to the communication link and / or data transmission, such as the transmission of encoded picture data.

[0094] The decoder 30 is configured to receive encoded picture data 21 and provide decoded picture data 31 or decoded picture 31 (further details are described below, for example, based on Figure 3 or Figure 5).

[0095] The post-processor 32 of the destination device 14 is configured to post-process the decoded picture data 31 (also called reconstructed picture data), for example, the decoded picture 31, to obtain post-processed picture data 33, for example, the post-processed picture 33. The post-processing performed by the post-processing unit 32 may include, for example, color format conversion (for example, from YCbCr to RGB), color correction, cropping, or resampling, or any other processing, to prepare the decoded picture data 31 for display by the display device 34, for example.

[0096] The display device 34 of the destination device 14 is configured to receive post-processed picture data 33 for displaying the picture to, for example, a user or viewer. The display device 34 may be, or comprise, any type of display for representing the reconstructed picture, such as an integrated or external display or monitor. The display may comprise, for example, a liquid crystal display (LCD), an organic light-emitting diode (OLED) display, a plasma display, a projector, a microLED display, a liquid crystal on silicon (LCoS), a digital optical processor (DLP), or any other type of display.

[0097] Figure 1A shows the source device 12 and the destination device 14 as separate devices, but the device embodiment may also have the functionality of both or both of the source device 12 or its corresponding functionality and the destination device 14 or its corresponding functionality. In such embodiments, the source device 12 or its corresponding functionality and the destination device 14 or its corresponding functionality may be implemented using the same hardware and / or software, or by separate hardware and / or software, or any combination thereof.

[0098] As will be apparent to those skilled in the art based on the description, the presence and (exact) division of functions in different units or functions within source device 12 and / or destination device 14 as shown in Figure 1A may vary depending on the actual device and application.

[0099] An encoder 20 (for example, a video encoder 20) or a decoder 30 (for example, a video decoder 30), or both an encoder 20 and a decoder 30, may be implemented via a processing circuit configuration such as that shown in Figure 1B, which may include one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, dedicated to video coding, or any combination thereof. The encoder 20 may be implemented via a processing circuit configuration 46 to embody various modules and / or any other encoder systems or subsystems described herein, such as those described with respect to the encoder 20 in Figure 2. The decoder 30 may be implemented via a processing circuit configuration 46 to embody various modules and / or any other decoder systems or subsystems described herein, such as those described with respect to the decoder 30 in Figure 3. The processing circuit configuration may be configured to perform various operations, as described later. If the technique is implemented in part in software, as shown in Figure 5, the device may store instructions for the software in a suitable non-temporary computer-readable storage medium, and may use one or more processors to execute the instructions in hardware to perform the technique of this disclosure. Both the video encoder 20 and the video decoder 30 can be integrated into a single device as part of a combined encoder / decoder (codec), for example, as shown in Figure 1B.

[0100] The source device 12 and destination device 14 may comprise any of a wide range of devices, including handheld or stationary devices of any kind, such as notebook or laptop computers, mobile phones, smartphones, tablets or tablet computers, cameras, desktop computers, set-top boxes, televisions, display devices, digital media players, video game consoles, video streaming devices (such as content service servers or content distribution servers), broadcast receiver devices, broadcast transmitter devices, etc., and may or may not have an operating system. In some cases, the source device 12 and destination device 14 may be equipped for wireless communication. Therefore, the source device 12 and destination device 14 may be wireless communication devices.

[0101] In some cases, the video coding system 10 shown in Figure 1A is merely an example, and the techniques of this application may be applied to video coding configurations (e.g., video coding or video decoding) that do not necessarily involve arbitrary data communication between the coding device and the decoding device. In other examples, the data may be retrieved from local memory or streamed over a network. The video coding device may code the data and store it in memory, and / or the video decoding device may retrieve the data from memory and decode it. In some examples, coding and decoding are performed by devices that do not communicate with each other but simply code the data into memory and / or retrieve the data from memory and decode it.

[0102] For the sake of explanation, embodiments of the present invention are described herein by reference to, for example, High-Efficiency Video Coding (HEVC) or Versatile Video Coding (VVC), i.e., to reference software for next-generation video coding standards developed by the Joint Collaboration Team on Video Coding (JCT-VC) of the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Motion Picture Experts Group (MPEG). Those skilled in the art will understand that embodiments of the present invention are not limited to HEVC or VVC.

[0103] Encoder and encoding method Figure 2 shows a schematic block diagram of an exemplary video encoder 20 configured to implement the technique of the present application. In the example of Figure 2, the video encoder 20 comprises an input unit 201 (or input interface 201), a residual calculation unit 204, a transformation unit 206, a quantization unit 208, an inverse quantization unit 210 and an inverse transformation unit 212, a reconstruction unit 214, a loop filter unit 220, a decoded picture buffer (DPB) 230, a mode selection unit 260, an entropy coding unit 270, and an output unit 272 (or output interface 272). The mode selection unit 260 may include an inter-prediction unit 244, an intra-prediction unit 254, and a segmentation unit 262. The inter-prediction unit 244 may include a motion estimation unit and a motion compensation unit (not shown). The video encoder 20 as shown in Figure 2 is sometimes called a hybrid video encoder, i.e., a video encoder with a hybrid video codec.

[0104] The residual calculation unit 204, the conversion processing unit 206, the quantization unit 208, and the mode selection unit 260 may be referred to as forming the forward signal path of the encoder 20, while the inverse quantization unit 210, the inverse conversion processing unit 212, the reconstruction unit 214, the buffer 216, the loop filter 220, the decoding picture buffer (DPB) 230, the inter-prediction unit 244, and the intra-prediction unit 254 may be referred to as forming the reverse signal path of the video encoder 20, where the reverse signal path of the video encoder 20 corresponds to the signal path of the decoder (see video decoder 30 in Figure 3). The inverse quantization unit 210, the inverse conversion processing unit 212, the reconstruction unit 214, the loop filter 220, the decoding picture buffer (DPB) 230, the inter-prediction unit 244, and the intra-prediction unit 254 are also referred to as forming the “built-in decoder” of the video encoder 20.

[0105] Pictures and picture sections (pictures and blocks) The encoder 20 may be configured to receive, for example, a picture 17 (or picture data 17), for example, a picture of a sequence of pictures that make up a video or video sequence, via the input unit 201. The received picture or picture data may also be a pre-processed picture 19 (or pre-processed picture data 19). For simplicity, the following description refers to picture 17. Picture 17 is sometimes called the current picture, or (in video coding in particular to distinguish the current picture from other pictures of the same video sequence, i.e., a video sequence that also has a current picture, e.g., pictures that have been previously encoded and / or decoded).

[0106] A (digital) picture is, or can be considered as, a two-dimensional array or matrix of samples having intensity values. A sample in an array is sometimes called a pixel (a picture element in short form) or pel. The number of samples in the horizontal and vertical directions (or axes) of the array or picture defines the size and / or resolution of the picture. For color representation, typically three color components are employed; that is, a picture may represent, or contain, three sample arrays. In the RGB format or color space, a picture comprises corresponding red, green, and blue sample arrays. However, in video coding, each pixel is typically represented in a luminance and chrominance format or color space, for example, YCbCr, which comprises a luminance component represented by Y (sometimes L is also used instead), as well as two chrominance components represented by Cb and Cr. The luminance (or short lumar) component Y represents brightness or gray level intensity (for example, in a grayscale picture), and the two chrominance (or short chromar) components Cb and Cr represent chromaticity or color information components. Therefore, a picture in YCbCr format comprises a luminance sample array of luminance sample values ​​(Y) and two chrominance sample arrays of chrominance values ​​(Cb and Cr). A picture in RGB format may be converted to or transformed to YCbCr format, and vice versa; this process is also called color transformation or color conversion. If a picture is monochrome, it may consist only of a luminance sample array. Thus, a picture may be, for example, an array of lumar samples in a monochrome format, or, in 4:2:0, 4:2:2, and 4:4:4 color formats, an array of lumar samples and two corresponding arrays of chromar samples.

[0107] Embodiments of the video encoder 20 may include a picture partitioning unit (not shown in Figure 2) configured to divide a picture 17 into multiple (usually non-overlapping) picture blocks 203. These blocks may also be called root blocks, macro blocks (H.264 / AVC), or coding tree blocks (CTB) or coding tree units (CTU) (H.265 / HEVC and VVC). The picture partitioning unit may be configured to use the same block size for all pictures in a video sequence and a corresponding grid defining the block size, or to change the block size between pictures or subsets or groups of pictures, dividing each picture into a corresponding block.

[0108] In further embodiments, the video encoder may be configured to directly receive blocks 203 of picture 17, for example, one, more, or all of the blocks that make up picture 17. Picture blocks 203 are sometimes referred to as picture blocks, or picture blocks to be coded.

[0109] Like picture 17, picture block 203 is again a two-dimensional array or matrix of samples having intensity values ​​(sample values), but smaller in dimensions than picture 17, or can be considered as such. In other words, block 203 may comprise, for example, one sample array (e.g., a lumar array in the case of monochrome picture 17, or a lumar array or chromar array in the case of a color picture), or three sample arrays (e.g., a lumar array and two chromar arrays in the case of color picture 17), or any other number and / or type of arrays depending on the color format applied. The number of samples in the horizontal and vertical directions (or axes) of block 203 defines the size of block 203. Thus, the block may be, for example, an M×N (M columns × N rows) array of samples, or an M×N array of conversion coefficients.

[0110] An embodiment of the video encoder 20, as shown in Figure 2, may be configured to encode the picture 17 block by block, for example, encoding and prediction may be performed in units of block 203.

[0111] Residual calculation The residual calculation unit 204 may be configured to obtain the residual block 205 (also called residual 205) based on picture block 203 and prediction block 265 (further details about prediction block 265 will be provided later) by subtracting the sample value of prediction block 265 from the sample value of picture block 203 for each sample (for each pixel), thereby obtaining the residual block 205 in the sample region.

[0112] conversion The transformation processing unit 206 may be configured to apply a transformation, such as a discrete cosine transform (DCT) or discrete sine transform (DST), to the sample values ​​of the residual block 205 to obtain transformation coefficients 207 in the transformation domain. The transformation coefficients 207 may also be called transformation residual coefficients and may represent the residual block 205 in the transformation domain.

[0113] The conversion processing unit 206 may be configured to apply an integer approximation of DCT / DST, such as a specified conversion for H.265 / HEVC. Compared to an orthogonal DCT conversion, such an integer approximation is typically scaled by several coefficients. Additional scaling coefficients are applied as part of the conversion process to preserve the norm of the residual blocks processed by the forward and inverse conversions. The scaling coefficients are typically chosen based on several constraints, such as the scaling coefficient being a power of 2 for shift operations, the bit depth of the conversion coefficients, and the trade-off between accuracy and implementation cost. For example, a specific scaling coefficient may be specified for the inverse conversion (and, for example, the corresponding inverse conversion by the inverse conversion processing unit 312 in the video decoder 30) by the inverse conversion processing unit 212, and a corresponding scaling coefficient for the forward conversion in the encoder 20, for example, by the conversion processing unit 206, may be specified accordingly.

[0114] Embodiments of the video encoder 20 (each a conversion processing unit 206) may be configured to output conversion parameters, for example, one or more types of conversion, which are encoded or compressed, either directly or via the entropy coding unit 270, so that, for example, the video decoder 30 can receive and use the conversion parameters for decoding.

[0115] quantization The quantization unit 208 may be configured to quantize the transformation coefficients 207 to obtain the quantization coefficients 209, for example, by applying scalar quantization or vector quantization. The quantization coefficients 209 are sometimes called the quantization transformation coefficients 209 or the quantization residual coefficients 209.

[0116] The quantization process can reduce the bit depth associated with some or all of the transformation coefficients 207. For example, n-bit transformation coefficients may be truncated to m-bit transformation coefficients during quantization, where n is greater than m. The degree of quantization can be modified by adjusting the Quantization Parameter (QP). For example, in the case of scalar quantization, various scalings can be applied to achieve finer or coarser quantization. Smaller quantization step sizes correspond to finer quantization, while larger quantization step sizes correspond to coarser quantization. Applicable quantization step sizes can be indicated by the Quantization Parameter (QP). The Quantization Parameter may be, for example, an index to a default set of applicable quantization step sizes. For example, a small quantization parameter may correspond to finer quantization (smaller quantization step size), a large quantization parameter may correspond to coarser quantization (larger quantization step size), and vice versa. Quantization may involve division by the quantization step size, and for example, the corresponding inverse quantization and / or inverse inverse quantization by the inverse quantization unit 210 may involve multiplication by the quantization step size. Some standards, e.g., embodiments by HEVC, may be configured to use quantization parameters to determine the quantization step size. Generally, the quantization step size may be calculated based on the quantization parameters using fixed-point approximations of the equations involving division. Additional scaling factors may be introduced for quantization and inverse quantization to restore the norm of the residual block, which may be modified for the scaling used in the fixed-point approximations of the equations for the quantization step size and quantization parameters. In one exemplary implementation, the scaling of the inverse transform and inverse quantization may be combined. Alternatively, a customized quantization table may be used, for example, signaled from encoder to decoder in a bitstream. Quantization is an irreversible operation, and the loss increases with increasing quantization step size.

[0117] Embodiments of the video encoder 20 (each a quantization unit 208) may be configured to output quantization parameters (QP) encoded, for example, directly or via an entropy coding unit 270, so that, for example, a video decoder 30 can receive and apply the quantization parameters for decoding.

[0118] inverse quantization The inverse quantization unit 210 is configured to obtain inverse quantization coefficients 211 by applying the inverse of the quantization scheme applied by the quantization unit 208 to the quantization coefficients, for example, based on or using the same quantization step size as the quantization unit 208. The inverse quantization coefficients 211 are sometimes called inverse quantization residual coefficients 211, and are usually not identical to the transformation coefficients due to losses due to quantization, but may correspond to the transformation coefficients 207.

[0119] Inverse Transform The inverse transformation processing unit 212 is configured to obtain a reconstructed residual block 213 (or the corresponding inverse quantization coefficient 213) in the sample region by applying the inverse transformation of the transformation applied by the transformation processing unit 206, for example, the inverse discrete cosine transform (DCT) or the inverse discrete sine transform (DST), or other inverse transformations. The reconstructed residual block 213 is sometimes referred to as the transformation block 213.

[0120] Reconstruction The reconstruction unit 214 (for example, an adder or summerer 214) is configured to add the transformed block 213 (i.e., the reconstructed residual block 213) to the predicted block 265 by adding the sample values ​​of the reconstructed residual block 213 and the sample values ​​of the predicted block 265 - sample by sample - to obtain the reconstructed block 215 in the sample region.

[0121] Filtering The loop filter unit 220 (or short “loop filter” 220) is configured to filter the reconstructed block 215 to obtain a filtered block 221, or more generally, to filter the reconstructed samples to obtain filtered samples. The loop filter unit is configured, for example, to smooth pixel transitions or to improve video quality in other ways. The loop filter unit 220 may comprise one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or one or more other filters, such as a bidirectional filter, an adaptive loop filter (ALF), a sharpening filter, a smoothing filter, or a co-filter, or any combination thereof. The loop filter unit 220 is shown in Figure 2 as an in-loop filter, but in other configurations, the loop filter unit 220 may be implemented as a post-loop filter. The filtered block 221 is sometimes referred to as the filtered reconstructed block 221.

[0122] Embodiments of the video encoder 20 (each a loop filter unit 220) may be configured to output loop filter parameters (such as sample-adaptive offset information) encoded, for example, directly or via the entropy coding unit 270, so that, for example, the decoder 30 can receive and apply the same loop filter parameters or the respective loop filters for decoding.

[0123] Decode picture buffer The decoded picture buffer (DPB) 230 may be a memory that stores a reference picture or generally reference picture data for encoding video data by the video encoder 20. The DPB 230 may be formed from any of a variety of memory devices, such as dynamic random access memory (DRAM) including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. The decoded picture buffer (DPB) 230 may be configured to store one or more filtered blocks 221. The decoded picture buffer 230 may be further configured to store other previously filtered blocks, for example, previously reconstructed and filtered blocks 221 of the same current picture or different pictures, for example, a previously reconstructed picture, for example, for interpretation, which may provide a previously reconstructed i.e., decoded whole picture (and corresponding reference blocks and samples) and / or a partially reconstructed current picture (and corresponding reference blocks and samples). For example, if the reconstructed block 215 is not filtered by the loop filter unit 220, or is any other further processed version of the reconstructed block or sample, the decoded picture buffer (DPB) 230 may also be configured to store one or more unfiltered reconstructed blocks 215, or generally, unfiltered reconstructed samples.

[0124] Mode selection (category and prediction) The mode selection unit 260 comprises a segmentation unit 262, an inter-prediction unit 244, and an intra-prediction unit 254, and is configured to receive or acquire original picture data, for example, the original block 203 (the current block 203 of the current picture 17), and reconstructed picture data, for example, filtered reconstructed samples or blocks, and / or unfiltered reconstructed samples or blocks, from the same (current) picture and / or from one or more previously decoded pictures, for example, from the decoded picture buffer 230 or other buffers (for example, line buffers not shown). The reconstructed picture data is used as reference picture data for predictions, for example, inter-prediction or intra-prediction, in order to acquire prediction blocks 265 or predictors 265.

[0125] The mode selection unit 260 may be configured to determine or select a division for the current block prediction mode (excluding divisions) and a prediction mode (e.g., intra or inter prediction mode), and to generate a corresponding prediction block 265 to be used for calculating the residual block 205 and for reconstructing the reconstructed block 215.

[0126] Embodiments of the mode selection unit 260 may be configured to select a partition and prediction mode (for example, from those supported by or available to the mode selection unit 260) that gives the best fit, i.e., minimum residual (meaning better compression for transmission or storage), or minimum signaling overhead (meaning better compression for transmission or storage), or both, or a balance between both. The mode selection unit 260 may be configured to determine the partition and prediction modes based on Rate Distortion Optimization (RDO), i.e., to select a prediction mode that yields the minimum rate distortion. In this context, terms such as “best,” “minimum,” and “optimal” do not necessarily refer to the overall “best,” “minimum,” and “optimal,” but may refer to the satisfaction of a termination criterion or selection criterion, such as a value above or below a threshold or other constraint, which may potentially lead to a “suboptimal selection” but reduce computational complexity and processing time.

[0127] In other words, the partitioning unit 262 may be configured to partition block 203 into smaller block partitions or subblocks (which then form blocks) using, for example, quadtree partitions (QT), binary partitions (BT), or tripletree partitions (TT), or any combination thereof, and to perform, for example, predictions for each of the block partitions or subblocks, wherein mode selection comprises a selection of the tree structure of the block 203 to be partitioned, and prediction modes are applied to each of the block partitions or subblocks.

[0128] The following describes in more detail the segmentation (by the segmentation unit 260, for example) and prediction (by the inter-prediction unit 244 and the intra-prediction unit 254) processes performed by the exemplary video encoder 20.

[0129] classification The partitioning unit 262 can now partition (i.e., divide) block 203 into smaller partitions, for example, smaller blocks of square or rectangular size. These smaller blocks (sometimes called subblocks) can be further partitioned into even smaller partitions. This is also called tree partitioning or hierarchical tree partitioning. For example, the root block at root tree level 0 (hierarchy level 0, depth 0) may be recursively partitioned, for example, into two or more blocks at the next lowest tree level, for example, into nodes at tree level 1 (hierarchy level 1, depth 1). These blocks may then be partitioned again into two or more blocks at the next lowest level, for example, tree level 2 (hierarchy level 2, depth 2), etc., until the partition is terminated, for example, because a termination criterion has been met, for example, the maximum tree depth or minimum block size has been reached. Blocks that are not further partitioned are also called leaf blocks or leaf nodes of the tree. A tree that uses a division into two partitions is called a binary tree (BT), a tree that uses a division into three partitions is called a ternary tree (TT), and a tree that uses a division into four partitions is called a quaternary tree (QT).

[0130] As previously stated, the term “block” as used herein may refer to a portion of a picture, particularly a square or rectangular portion. For example, with reference to HEVC and VVC, a block may be, or equivalent to, a coding tree unit (CTU), a coding unit (CU), a prediction unit (PU), and a transform unit (TU), as well as the corresponding blocks, such as a coding tree block (CTB), a coding block (CB), a transform block (TB), or a prediction block (PB).

[0131] For example, a coding tree unit (CTU) may be a CTB for a lumen sample of a picture having three sample arrays, two corresponding CTBs for a chroma sample, or a CTB for a sample of a monochrome picture or a picture coded using three separate color planes, and a syntax structure used to code the sample. Correspondingly, a coding tree block (CTB) may be an N×N block of sample for some value of N such that the division of components into the CTB is a partition. A coding unit (CU) may be a coding block for a lumen sample of a picture having three sample arrays, two corresponding coding blocks for a chroma sample, or a coding block for a sample of a monochrome picture or a picture coded using three separate color planes, and a syntax structure used to code the sample. Correspondingly, a coding block (CB) may be an M×N block of sample for some value of M and N such that the division of the CTB into the coding block is a partition.

[0132] For example, in an embodiment using HEVC, a coding tree unit (CTU) can be divided into CUs by using a quadtree structure, which is represented as a coding tree. The decision of whether to code a picture area using (temporal) interpicture prediction or (spatial) intrapicture prediction is made at the CU level. Each CU can be further divided into one, two, or four PUs according to the PU partitioning type. Within a single PU, the same prediction process is applied, and the relevant information is sent to the decoder on a PU-by-PU basis. After obtaining residual blocks by applying the prediction process based on the PU partitioning type, the CU can be divided into transformation units (TUs) according to another quadtree structure similar to the coding tree for the CU.

[0133] In embodiments, for example, according to the latest video coding standard currently under development called Multipurpose Video Coding (VVC), quad-tree and binary tree (QTBT) segmentation is used to segment coding blocks. Within the QTBT block structure, CUs can have either a square or rectangular shape. For example, a coding tree unit (CTU) is initially segmented by a quad-tree structure. The quad-tree leaf nodes are further segmented by a binary or ternary (or triple) tree structure. The segmentation of the tree leaf nodes is called a coding unit (CU), and its segmentation is used for prediction and transformation processing without further segmentation. This means that CUs, PUs, and TUs have the same block size within the QTBT coding block structure. In parallel, multiple segmentation, such as triple-tree segmentation, has also been proposed for use with the QTBT block structure.

[0134] In one example, the mode selection unit 260 of the video encoder 20 may be configured to perform any combination of the segmentation techniques described herein.

[0135] As described above, the video encoder 20 is configured to determine or select the best or most optimal prediction mode from a set of (predetermined) prediction modes. The set of prediction modes may include, for example, an intra-prediction mode and / or an inter-prediction mode.

[0136] Intra Prediction The set of intra-prediction modes may comprise, for example, 35 different intra-prediction modes, such as DC (or average) mode and omnidirectional or directional modes such as planar mode, as defined in HEVC, or 67 different intra-prediction modes, such as DC (or average) mode and omnidirectional or directional modes such as planar mode, as defined for VVC.

[0137] The intra-prediction unit 254 is configured to use reconfigured samples of adjacent blocks of the same current picture to generate an intra-prediction block 265 according to the intra-prediction mode of a set of intra-prediction modes.

[0138] The intra-prediction unit 254 (or generally the mode selection unit 260) is further configured to output intra-prediction parameters (or generally information indicating a selected intra-prediction mode for a block) to the entropy coding unit 270 in the form of syntax elements 266 for inclusion in the coded picture data 21, so that, for example, the video decoder 30 can receive and use the prediction parameters for decoding.

[0139] Interpretation The set of interpretation modes (or possible interpretation modes) depends on the available reference picture (i.e., a previous picture, at least partially decoded, stored in DBP230, for example), as well as other interpretation parameters, such as whether the entire reference picture is used to search for the best-matching reference block, or only a part of it, for example, the search window area around the current block area of ​​the reference picture, and / or whether pixel interpolation, such as half-PEL interpolation / semi-PEL interpolation and / or quarter-PEL interpolation, is applied.

[0140] In addition to the prediction modes described above, skip mode and / or direct mode may be applied.

[0141] The interpretation unit 244 may include a motion estimation (ME) unit and a motion compensation (MC) unit (neither of which are shown in Figure 2). The motion estimation unit may be configured to receive or acquire, for motion estimation, a picture block 203 (the current picture block 203 of the current picture 17) and a decoded picture 231, or at least one or more previously reconstructed blocks, for example, one or more other / different reconstructed blocks of the previously decoded picture 231. For example, a video sequence may comprise a current picture and a previously decoded picture 231, i.e., the current picture and the previously decoded picture 231 may be part of, or may form part of, a sequence of pictures that make up the video sequence.

[0142] The encoder 20 may be configured, for example, to select a reference block from multiple reference blocks of the same or different pictures among several other pictures, and to provide the motion estimation unit with the reference picture (or reference picture index), and / or the offset (spatial offset) between the position (x, y coordinates) of the reference block and the position of the current block as interpretation parameters. This offset is also called the motion vector (MV).

[0143] The motion compensation unit is configured to acquire, for example, interprediction parameters and perform interprediction to acquire interprediction blocks 265 based on or using the interprediction parameters. The motion compensation performed by the motion compensation unit may involve fetching or generating prediction blocks based on motion / block vectors determined by motion estimation, and possibly performing interpolation to sub-pixel precision. Interpolation filtering may generate additional pixel samples from known pixel samples, thus potentially increasing the number of candidate prediction blocks that can be used to code picture blocks. Now receiving a motion vector for a picture block PU, the motion compensation unit may locate the position of the prediction block pointed to by the motion vector in one of the reference picture lists.

[0144] The motion compensation unit may also generate syntax elements related to blocks and video slices for use by the video decoder 30 when decoding picture blocks of video slices.

[0145] Entropy coding The entropy coding unit 270 applies the quantization coefficients 209, inter-prediction parameters, intra-prediction parameters, loop filter parameters, and / or other syntax elements to, for example, entropy coding algorithms or entropy coding schemes (e.g., Variable Length Coding (VLC), Context Adaptive VLC (CAVLC), arithmetic coding schemes, binarization schemes, Context Adaptive Binary Arithmetic Coding (CABAC), Syntax-Based Context-Adaptive Binary Arithmetic Coding (SBAC), Probability Interval Partitioning (PIPE)). The system is configured to obtain encoded picture data 21 that can be output via the output unit 272 in the form of an encoded bitstream 21, for example, by applying entropy coding (or another entropy coding method or technique) or bypass (no compression), so that, for example, the video decoder 30 can receive and use parameters for decoding. The encoded bitstream 21 may be sent to the video decoder 30 or stored in memory for later transmission or retrieval by the video decoder 30.

[0146] Other structural variations of the video encoder 20 may be used to encode a video stream. For example, a non-conversion-based encoder 20 can directly quantize the residual signal for some blocks or frames without using a conversion processing unit 206. In another implementation, the encoder 20 may have a quantization unit 208 and an inverse quantization unit 210 combined in a single unit.

[0147] Decoder and decoding method Figure 3 shows an example of a video decoder 30 configured to implement the technique of this invention. The video decoder 30 is configured to receive encoded picture data 21 (e.g., encoded bitstream 21), encoded by an encoder 20, for example, and to obtain a decoded picture 331. The encoded picture data or bitstream comprises information for decoding the encoded picture data, for example, data representing picture blocks of an encoded video slice, and associated syntax elements.

[0148] In the example in Figure 3, the decoder 30 comprises an entropy decoding unit 304, an inverse quantization unit 310, an inverse transformation processing unit 312, a reconstruction unit 314 (e.g., an adder 314), a loop filter 320, a decoded picture buffer (DBP) 330, an interpretation unit 344, and an intraprediction unit 354. The interpretation unit 344 may be a motion compensation unit, or may include a motion compensation unit. In some examples, the video decoder 30 may perform a decoding path that is generally the opposite of the encoding path described with respect to the video encoder 100 from Figure 2.

[0149] As described with respect to encoder 20, the inverse quantization unit 210, inverse processing unit 212, reconstruction unit 214, loop filter 220, decoding picture buffer (DPB) 230, inter-prediction unit 344, and intra-prediction unit 354 are also referred to as forming the “built-in decoder” of video encoder 20. Thus, the inverse quantization unit 310 may have the same function as the inverse quantization unit 110, the inverse processing unit 312 may have the same function as the inverse processing unit 212, the reconstruction unit 314 may have the same function as the reconstruction unit 214, the loop filter 320 may have the same function as the loop filter 220, and the decoding picture buffer 330 may have the same function as the decoding picture buffer 230. Accordingly, the descriptions provided for each unit and function of video encoder 20 apply correspondingly to each unit and function of video decoder 30.

[0150] Entropy decoding The entropy decoding unit 304 is configured to parse the bitstream 21 (or generally, the encoded picture data 21) and, for example, perform entropy decoding to the encoded picture data 21 to obtain, for example, quantization coefficients 309 and / or decoded coding parameters (not shown in Figure 3), such as inter-prediction parameters (e.g., reference picture index and motion vector), intra-prediction parameters (e.g., intra-prediction mode or index), transformation parameters, quantization parameters, loop filter parameters, and / or other syntax elements. The entropy decoding unit 304 may be configured to apply a decoding algorithm or decoding scheme corresponding to an encoding scheme such as those described with respect to the entropy coding unit 270 of the encoder 20. The entropy decoding unit 304 may be further configured to provide the inter-prediction parameters, intra-prediction parameters, and / or other syntax elements to the mode selection unit 360, and other parameters to other units of the decoder 30. The video decoder 30 may receive syntax elements at the video slice level and / or video block level.

[0151] inverse quantization The inverse quantization unit 310 may be configured to receive quantization parameters (QP) (or generally, information related to inverse quantization) and quantization coefficients from the encoded picture data 21 (for example, by parsing and / or decoding by the entropy decoding unit 304), and to apply inverse quantization to the decoded quantization coefficients 309 based on the quantization parameters to obtain inverse quantization coefficients 311, sometimes called transformation coefficients 311. The inverse quantization process may include the use of quantization parameters determined by the video encoder 20, per video block in the video slice, to determine the degree of quantization and, similarly, the degree of inverse quantization to be applied.

[0152] Inverse Transform The inverse transformation processing unit 312 may be configured to receive inverse quantization coefficients 311, also called transformation coefficients 311, and to apply a transformation to the inverse quantization coefficients 311 to obtain a reconstructed residual block 213 in the sample region. The reconstructed residual block 213 may also be called a transformation block 313. The transformation may be an inverse transformation, such as an inverse DCT transformation, an inverse DST transformation, an inverse integer transformation, or a conceptually similar inverse transformation process. The inverse transformation processing unit 312 may be further configured to receive transformation parameters or corresponding information from the encoded picture data 21 (for example, by an entropy decoding unit 304, for example, by parsing and / or decoding) in order to determine the transformation to be applied to the inverse quantization coefficients 311.

[0153] Reconstruction The reconstruction unit 314 (for example, an adder or summerer 314) may be configured to add the reconstruction residual block 313 to the prediction block 365 by adding the sample value of the reconstruction residual block 313 to the sample value of the prediction block 365, for example, to obtain the reconstruction block 315 in the sample region.

[0154] Filtering The loop filter unit 320 (either within or after the coding loop) is configured to filter the reconstructed block 315 to obtain the filtered block 321, for example, to smooth pixel transitions or to improve video quality in another way. The loop filter unit 320 may comprise one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or one or more other filters, such as a bidirectional filter, an adaptive loop filter (ALF), a sharpening filter, a smoothing filter, or a co-filter, or any combination thereof. The loop filter unit 320 is shown in Figure 3 as an in-loop filter, but in other configurations, the loop filter unit 320 may be implemented as a post-loop filter.

[0155] Decode picture buffer The decoded video block 321 of the picture is then stored in a decoded picture buffer 330, which stores the decoded picture 331 as a reference picture for subsequent motion compensation for other pictures and / or for output display, respectively.

[0156] The decoder 30 is configured to output the decoded picture 311, for example, via the output unit 312, for presentation or viewing by the user.

[0157] prediction The inter-prediction unit 344 may be identical to the inter-prediction unit 244 (more specifically, the motion compensation unit), and the intra-prediction unit 354 may have the same functionality as the inter-prediction unit 254, and performs partitioning or partitioning decision and prediction based on partitioning and / or prediction parameters, or each piece of information received from the encoded picture data 21 (for example, by the entropy decoding unit 304, for example, by parsing and / or decoding). The mode selection unit 360 may be configured to obtain a predicted block 365 by performing a block-by-block prediction (intra-prediction or inter-prediction) based on the reconstructed picture, block, or each sample (filtered or unfiltered).

[0158] When a video slice is coded as an intra-coding (I) slice, the intra-prediction unit 354 of the mode selection unit 360 is configured to generate a prediction block 365 for the picture block of the current video slice based on the signaled intra-prediction mode and data from previously decoded blocks of the current picture. When a video picture is coded as an inter-coding (i.e., B or P) slice, the inter-prediction unit 344 (e.g., a motion compensation unit) of the mode selection unit 360 is configured to produce a prediction block 365 for the video block of the current video slice based on the motion vector and other syntax elements received from the entropy decoding unit 304. In the case of inter-prediction, the prediction block may be produced from one of the reference pictures in one of the reference picture lists. The video decoder 30 may construct reference frame lists, i.e., list 0 and list 1, using default construction techniques based on the reference pictures stored in the DPB 330.

[0159] The mode selection unit 360 is configured to determine prediction information for the video blocks of the current video slice by parsing motion vectors and other syntax elements, and to use the prediction information to create prediction blocks for the current video blocks being decoded. For example, the mode selection unit 360 uses some of the received syntax elements to determine the prediction mode used to code the video blocks of the video slice (e.g., intra-prediction or inter-prediction), the inter-prediction slice type (e.g., B-slice, P-slice, or GPB-slice), configuration information for one or more of the reference picture lists for the slice, motion vectors for each inter-coded video block of the slice, the inter-prediction status for each inter-coded video block of the slice, and other information for decoding the video blocks in the current video slice.

[0160] Other variations of the video decoder 30 may be used to decode the encoded picture data 21. For example, the decoder 30 can produce an output video stream without using the loop filtering unit 320. For example, a non-transformation-based decoder 30 can directly dequantize the residual signal for some blocks or frames without using the inverse transformation unit 312. In another implementation, the video decoder 30 may have an inverse quantization unit 310 and an inverse transformation unit 312 combined in a single unit.

[0161] It should be understood that in encoder 20 and decoder 30, the processing result of the current step may be further processed and then output to the next step. For example, after interpolation filtering, motion vector derivation, or loop filtering, further operations such as clipping or shifting may be performed on the processing result of interpolation filtering, motion vector derivation, or loop filtering.

[0162] Note that further operations may be applied to the derived motion vectors of the current block (including, but not limited to, the control point motion vectors in affine mode, the affine mode, the plane mode, the sub-block motion vectors in ATMVP mode, the temporal motion vectors, etc.). For example, the values of the motion vectors are constrained to a predetermined range according to their representation bits. When the representation bit of the motion vector is bitDepth, the range is -2^(bitDepth-1) to 2^(bitDepth-1)-1, where "^" means exponentiation. For example, when bitDepth is set equal to 16, the range is -32768 to 32767, and when bitDepth is set equal to 18, the range is -131072 to 131071. For example, the values of the derived motion vectors (e.g., the MVs of four 4×4 sub-blocks within one 8×8 block) are constrained such that the maximum difference between the integer parts of the four 4×4 sub-block MVs is no more than N pixels, such as no more than 1 pixel. Here, two methods for constraining the motion vector according to bitDepth are provided.

[0163] Method 1: Remove the overflow MSB (most significant bit) by a flowing operation. ux=(mvx+2 bitDepth )%2 bitDepth (1) mvx=(ux>=2 bitDepth-1 )? (ux-2 bitDepth ) : ux (2) uy=(mvy+2 bitDepth )%2 bitDepth (3) mvy=(uy>=2 bitDepth-1 )? (uy-2 bitDepth ) : uy (4) However, mvx is the horizontal component of the motion vector of the image block or sub-block, mvy is the vertical component of the motion vector of the image block or sub-block, and ux and uy represent intermediate values.

[0164] For example, if the value of mvx is -32769, then after applying equations (1) and (2), the resulting value is 32767. Computer systems store decimal numbers as two's complement. The two's complement of -32769 is 1,0111,1111,1111,1111 (17 bits), then the MSB is discarded, and so the resulting two's complement is 0111,1111,1111,1111 (decimal is 32767), which is the same as the output by applying equations (1) and (2). ux=(mvpx+mvdx+2 bitDepth )%2 bitDepth (5) mvx=(ux>=2 bitDepth-1 ) ? (ux-2 bitDepth ) : ux (6) uy=(mvpy+mvdy+2 bitDepth )%2 bitDepth (7) mvy=(uy>=2 bitDepth-1 ) ? (uy-2 bitDepth ) : uy (8)

[0165] As shown in equations (5) to (8), the calculation can be applied between the sums of MVP and MVD.

[0166] Method 2: Remove the overflow MSB by clipping the value. vx=Clip3(-2 bitDepth-1 ,2 bitDepth-1 -1,vx) vy=Clip3(-2 bitDepth-1 ,2 bitDepth-1 -1, vy) However, vx is the horizontal component of the motion vector of an image block or subblock, vy is the vertical component of the motion vector of an image block or subblock, x, y, and z correspond to the three input values ​​of the MV clipping process, respectively, and the definition of the function Clip3 is as follows:

[0167]

number

[0168] Figure 4 is a schematic diagram of a video coding device 400 according to one embodiment of the present disclosure. The video coding device 400 is suitable for carrying out the embodiments disclosed as described herein. In one embodiment, the video coding device 400 may be a decoder, such as the video decoder 30 in Figure 1A, or an encoder, such as the video encoder 20 in Figure 1A.

[0169] The video coding device 400 comprises an inlet port 410 (or input port 410) and a receiver unit (Rx) 420 for receiving data, a processor, logic unit, or central processing unit (CPU) 430 for processing data, a transmitter unit (Tx) 440 and an exit port 450 (or output port 450) for transmitting data, and memory 460 for storing data. The video coding device 400 may also comprise optical-to-electrical (OE) components and electrical-to-optical (EO) components coupled to the inlet port 410, receiver unit 420, transmitter unit 440, and exit port 450 for the exit or input of optical or electrical signals.

[0170] The processor 430 is implemented by hardware and software. The processor 430 may be implemented as one or more CPU chips, cores (e.g., a multi-core processor), FPGAs, ASICs, and DSPs. The processor 430 communicates with an inlet port 410, a receiver unit 420, a transmitter unit 440, an exit port 450, and memory 460. The processor 430 includes a coding module 470. The coding module 470 implements the embodiments disclosed above. For example, the coding module 470 performs, processes, prepares, or provides various coding operations. Thus, including the coding module 470 results in a significant improvement in the functionality of the video coding device 400 and results in the transformation of the video coding device 400 into different states. Alternatively, the coding module 470 is implemented as instructions stored in memory 460 and executed by the processor 430.

[0171] Memory 460 may comprise one or more disks, tape drives, and solid-state drives, and may be used as an overflow data storage device for storing a program when such a program is selected for execution, and for storing instructions and data read during program execution. Memory 460 may be, for example, volatile and / or non-volatile, and may be read-only memory (ROM), random access memory (RAM), ternary content-addressable memory (TCAM), and / or static random access memory (SRAM).

[0172] Figure 5 is a simplified block diagram of a device 500 that can be used as either or both of the source device 12 and destination device 14 from Figure 1, according to an exemplary embodiment.

[0173] The processor 502 in the device 500 may be a central processing unit. Alternatively, the processor 502 may be any other type of device, existing or to be developed in the future, capable of manipulating or processing information. The disclosed implementation may be practiced using a single processor, e.g., processor 502, as shown in the figures, but advantages in speed and efficiency may be achieved using two or more processors.

[0174] The memory 504 in the device 500 may be a read-only memory (ROM) device or a random access memory (RAM) device in one implementation. Any other suitable type of storage device may be used as memory 504. Memory 504 may contain code and data 506 accessed by the processor 502 using the bus 512. Memory 504 may further contain an operating system 508 and an application program 510, the application program 510 including at least one program that enables the processor 502 to perform the method described herein. For example, the application program 510 may include applications 1 to N, the applications 1 to N further including a video coding application that performs the method described herein.

[0175] The device 500 may also include one or more output devices, such as a display 518. The display 518 may, in one example, be a touch-sensitive display that combines the display with a touch-sensitive element capable of operating to sense touch input. The display 518 may be coupled to the processor 502 via the bus 512.

[0176] Although shown here as a single bus, the bus 512 of device 500 may consist of multiple buses. Furthermore, the secondary storage 514 may be directly coupled to other components of device 500 or accessed via a network, and may comprise a single integrated unit such as a memory card, or multiple units such as multiple memory cards. Thus, device 500 can be implemented in a wide variety of configurations.

[0177] Figure 6 shows a schematic diagram of several intra-prediction modes used in the HEVC UIP scheme. For luminance blocks, the intra-prediction modes may comprise up to 36 modes, including three omnidirectional modes and 33 directional modes. The omnidirectional modes may comprise a planar prediction mode, a mean (DC) prediction mode, and a lumen-to-chroma (LM) prediction mode. The planar prediction mode can perform predictions by assuming a block angle plane with horizontal and vertical slopes derived from the block boundaries. The DC prediction mode can perform predictions by assuming a flat block plane with values ​​consistent with the average value of the block boundaries. The LM prediction mode can perform predictions by assuming that the chroma value for a block is consistent with the lumen value for a block.

[0178] Figure 7 shows an example of 67 intra-prediction modes, such as those proposed for VVC, where multiple intra-prediction modes of the 67 intra-prediction modes include a planar mode (index 0), a dc mode (index 1), and angular modes with indices 2 through 66, where the lower left angular mode in Figure 7 is index 2, and the index numbering is incremented until index 66 is the upper right angular mode in Figure 7.

[0179] As shown in Figure 8, VVC, including the second version, has several modes corresponding to inclined intra-prediction directions, including a wide-angle mode (shown as a dashed line). For any of these modes, interpolation of an adjacent set of reference samples should be performed to predict a sample in a block if the corresponding position within the block side is fractional. HEVC and VVC use linear interpolation between two adjacent reference samples. JEM uses a more sophisticated 4-tap interpolation filter. The filter coefficients are selected to be either a Gaussian filter or a cubic filter, depending on the width or height value. The decision of whether to use width or height is in conjunction with the decision on primary reference side selection; i.e., when the intra-prediction mode is diagonal or greater, the upper side of the reference samples is selected to be the primary reference side, and the width value is selected to determine the interpolation filter to be used. Otherwise, the primary side reference is selected from the left side of the block, and the height controls the filter selection process. In detail, if the selected side length is 8 samples or less, cubic interpolation 4-tap is applied. Otherwise, the interpolation filter is a 4-tap Gaussian filter.

[0180] Similar to interpretation, intrapretation may require interpolation filtering when samples within a block are predicted according to a fractional gradient directional mode. When a linear filter is used for this purpose, the filter coefficients can be easily calculated if the samples in the block to be predicted fall within fractional (sub-per) positions in the reference sample. Therefore, a linear filter does not require a LUT (lookup table) to store its coefficients. However, a LUT may be used instead of direct calculation. Nevertheless, when the prediction module uses an interpolation filter with a long tap (e.g., 4 or 8 taps), it may require a LUT to store the interpolation filter coefficients, as done in the interpretation module, where 8-tap DCT-IFs for lumens and 4-tap DCT-IFs for chromens are constructed as shown in Table 1 (Table 5) and Table 2 (Table 6), respectively.

[0181] [Table 5]

[0182] [Table 6]

[0183] A specific set of coefficients may be defined as shown in Table 3 (Table 7).

[0184] [Table 7]

[0185] The intra-predicted samples are calculated by convolving with coefficients defined according to the subpixel offset and filter type, as follows:

[0186]

number

[0187] In this expression, ">>" indicates a bitwise right shift operation.

[0188] Set of coefficients {c i The values ​​are fetched from the lookup table (LUT). Table 3 (Table 7) gives an example of values ​​stored according to the current design of the VVC, as described in the version 3 specification draft (JVET-L1001 "Versatile Video Coding (Draft 3)"). The selection between the smoothing (fG) interpolation filter and the sharpening (fC) interpolation filter is performed using the MDIS (Mode-Dependent Intra-Smoothing) condition. Therefore, either fC or fG may be used to generate predictive samples of a block.

[0189] If a sharpening filter is selected, the predicted sample s(x) is further clipped to a range of acceptable values, i.e., either defined using SPS or derived from the bit depth of the selected component.

[0190] For some use cases, LUT processing requires additional memory and therefore increases energy consumption and die size, so it may be beneficial to avoid storing explicit LUTs for interpolation filters. The first is significant for mobile applications. The second has a negative impact on price.

[0191] In the use case described, it is appropriate to analytically define the coefficients of the interpolation filter. Rather than performing a fetch from the LUT, the filter coefficients can be calculated from the input fractional sample position p.

[0192] According to one embodiment of the present disclosure, a method is provided for intra or interpredictive processing of a video frame, the method comprising the steps of: obtaining a reference sample; obtaining a subpixel offset value; and filtering the reference sample using a subpixel 4-tap interpolation filter to obtain a predictive sample value, wherein the filter coefficients of the subpixel 4-tap interpolation filter are

[0193]

number

[0194] The following conditions are met: p is the fractional part of the subpixel offset value, and c0, c1, c2, and c3 are the filter coefficients of the subpixel 4-tap interpolation filter.

[0195] In one exemplary implementation, a reference sample may be referenced as ref[x], which corresponds to an array of reference samples. ref[x]=p[-1-refIdx+x][-1-refIdx](where x=0..nTbW+refIdx+1) Here, "p" refers to a 2D xy array p[x][y] containing the sample values. The number of reference samples used may be at least one. In another example, the number of reference samples may be four.

[0196] In one example implementation, the acquired subpixel offset value is: (y+1+refIdx)*intraPredAngle Often referred to as such, "IntraPredAngle" is the value of the intra-predicted angle.

[0197] In one example implementation, the predicted sample values ​​"predSamples[x][y]" are:

[0198]

number

[0199] These can be obtained by fT[i], where fT[i] refers to the filter coefficients. These coefficients may be lumer or chroma filter coefficients for interprediction, referred to as fG and fC, respectively.

[0200] The choice of whether the filter coefficients are lumens or chromens is, for example, fT[j]=filterFlag ? fG[iFact][j] : fC[iFact][j] This can be done by using and setting the flag "filterFlag", iFact=((y+1+refIdx)*intraPredAngle)&31 That is the case.

[0201] The value "31" refers to the fractional part of the subpixel offset value and may take other values ​​different from "31". The values ​​of the filter coefficients fG (luma) and / or fC (chroma) can be obtained analytically on the fly using the analytical representation above for the filter coefficients of a 4-tap filter. Thus, the filter coefficients are defined according to the subpixel offset value.

[0202] Therefore, the filter coefficients are obtained without accessing the individual values ​​of the filter coefficients from the LUT, but rather by calculation.

[0203] Alternatively, the filter coefficients may be calculated using the above formula, and the values ​​may be stored in the LUT.

[0204] According to one embodiment of the present disclosure, the filter coefficients of the subpixel 4-tap interpolation filter are given in the following table, i.e.,

[0205] [Table 8A]

[0206] [Table 8B]

[0207] It is defined within the following.

[0208] According to one embodiment of the present disclosure, a method for intra or interpredictive processing of a video frame is provided, the method comprising the steps of: obtaining a reference sample; obtaining a subpixel offset value; and filtering the reference sample using a subpixel 4-tap interpolation filter to obtain a predictive sample value, wherein the filter coefficients of the subpixel 4-tap interpolation filter are as follows, i.e.,

[0209] [Table 9]

[0210] It is defined as follows, where p is the fractional part of the subpixel offset value, and c0, c1, c2, and c3 are filter coefficients.

[0211] In one exemplary implementation, the values ​​of the filter coefficients can be stored in the LUT. This means that each value of the filter coefficient will be fetched from the LUT for subpixel interpolation filtering.

[0212] Figure 23 shows a flowchart of the intra / inter prediction process. In step 2301, a reference sample is acquired. The reference sample may consist of one or more reference samples. In step 2302, the subpixel offset value is then acquired. In step 2303, the reference sample is then filtered by the filter coefficient {c i The image is filtered using a subpixel interpolation filter with} (where i=0, ..., 3). This is called a 4-tap filter and / or 4-point filter.

[0213] The filter coefficient can be obtained from an analytical expression using a given fractional part p of the sub-pixel offset value. The filter coefficient can also be obtained from a LUT referring to the p value.

[0214] According to one embodiment of the present disclosure, the bit of p is set to 0.

[0215] According to one embodiment of the present disclosure, p is incremented by 1, and then after the lower bit of the incremented p is set to 0, the said p is used to obtain the filter coefficient.

[0216] According to one embodiment of the present disclosure, obtaining the filter coefficient comprises

[0217]

Equation

[0218] the step of calculating the filter coefficient according to. This can bring the advantage of calculating the filter coefficient c i from the previous filter coefficient c i+1 to the filter coefficient c. This can further shorten the time for calculating the filter coefficient.

[0219] The proposed invention enables both methods to derive the filter coefficient, that is, the LUT fetches the calculation and guides the calculation according to the proposed formula.

[0220] One of the functions of the interpolation filter consists of its linear dependence of the phase shift on the fractional sample position p. Embodiments of the present invention utilize this dependence to obtain the value of the filter coefficient using a simple operation for the fractional sample position p.

[0221] The linear dependence is used to derive the smoothing interpolation filter coefficient fG. The derivation of the sharpening interpolation filter coefficient fC is based on taking the absolute value of a linear expression based on p.

[0222] According to one embodiment of the present disclosure, a video coding method is provided, comprising an intra-predictive processing of a block of video frames, the method comprising filtering a reference sample of a video frame using a subpixel 4-tap interpolation filter, wherein the subpixel 4-tap interpolation filter is a combination of filters, and at least one of the filters has a filter coefficient,

[0223]

number

[0224] The following conditions are met: p is the fractional sample position of the subpixel offset, and c0, c1, c2, and c3 are filter coefficients.

[0225] Figure 24 shows a flowchart of the video coding process. In step 2410, an intra-predictive process is performed by filtering the reference samples of the block using a subpixel 4-tap interpolation filter.

[0226] According to one embodiment of the present disclosure, the subpixel 4-tap interpolation filter is a smoothing filter which is a parallel combination of two phase-dependent linear filters and a low-pass filter.

[0227] According to one embodiment of the present disclosure, the subpixel 4-tap interpolation filter is a sharpening filter which is a parallel combination of a linear filter that depends on one phase and an intensity-variable high-pass filter.

[0228] According to one embodiment of the present disclosure, the intensity-variable high-pass filter is a phase-independent high-pass finite impulse response FIR filter whose output is multiplied by a phase-dependent intensity parameter.

[0229] According to any one of the prior embodiments of this disclosure, the lower bits of p are set to 0.

[0230] According to one embodiment of the present disclosure, p is incremented by 1, and then, after the least significant bit of the incremented value is set to 0, this value is used to derive an interpolation filter coefficient.

[0231] According to one embodiment of the present disclosure, a clipping operation is performed on the output of the intensity fluctuation high-pass filter.

[0232] Embodiments of the present invention represent filter coefficients as a combination of two or more FIR (finite impulse response) filters. This combination may have the following options. - Serial connection, that is, convolution of coefficients. - Parallel connection, that is, weighted sum of coefficients.

[0233] Each of these filters may or may not depend on a phase parameter P that is a function of the fractional sample position p. The coefficients of the filters belonging to the combination have a simple expression from the phase parameter P.

[0234] In one embodiment, in FIG. 9, a set of reference samples is input to an interpolation filter.

[0235] In one embodiment, in the case of a smoothing interpolation filter, the value of the phase parameter P is the same as the value of the fractional sample position p. In FIG. 9, blocks F1 and F2 represent linear interpolation filters. The coefficients for each of these filters are expressed as follows from the phase parameter p.

[0236]

Equation

[0237] In one example, here, furthermore, a division operation may be defined with or without rounding, that is,

[0238]

Equation

[0239] or

[0240]

number

[0241] Here, "a" is the divider nominator, and "n" is the power of the two parameters in the denominator.

[0242] Block F3 has constant coefficients. c0=16 c1=16 This represents a two-tap low-pass filter with the following characteristics:

[0243] In one example, filters F1 and F3 are not normalized, and F3 has a greater DC gain than F1, and F3 has a greater DC gain than F2.

[0244] The outputs of filters F1..F3 are summed and normalized. Normalization (block ">>") can be performed, for example, by right-shifting or by division with rounding as described above.

[0245] The equation obtained for this combination (an equivalent 4-tap filter) can be expressed as follows:

[0246]

number

[0247] This embodiment enables a LUT-based implementation. According to this implementation, coefficient values ​​can also be defined using a LUT. The values ​​stored in this LUT are defined using Table 4 (Table 10) (for both rounded and unrounded cases).

[0248] [Table 10]

[0249] From Table 4 (Table 10), it can be noted that the coefficients are within the range [0, 31]. This fact explains the technical advantage of the present invention, which lies in the possibility of using a 16-bit multiplier for 8-bit and 10-bit pictures, which are, and will be, most frequently used, at least from a medium-term perspective. A typical implementation of the present invention would have four parallel multiplication operations, with the operands of the multiplication having a bit depth of up to 6 for the filter coefficients and 10 for the samples. The result of the multiplication would not exceed a 16-bit value, thus making the proposed coefficients favorable from an implementation standpoint.

[0250] In interprediction mode, block motion compensation is performed. Motion compensation may involve intra-interpolation filtering and a similar interpolation filtering step. Another beneficial feature of the present invention is that the interpolation filter coefficients have the same precision (i.e., the bit depth of the coefficient values ​​is the same) for both intraprediction and interprediction modes. If the precision of the interpolation filter coefficients is higher in the intraprediction case, it can negatively impact the computational complexity of the video codec. This is because intraprediction is inherently continuous, as it requires reconstructed samples of previous blocks. In contrast, interprediction can be performed in parallel. Therefore, if the interpolation precision is higher for interprediction than for intraprediction, it can deepen the implementation misbalance between the intraprediction pipeline and the interprediction pipeline. This precision alignment makes it possible to avoid such negative conclusions.

[0251] Another advantage of the proposed invention lies in the reduction of coefficient diversity. Considering that adjacent row pairs in Table 4 (Table 10) are identical, a practical implementation of the invention would fetch from a LUT with only 16 rows (rows corresponding to even values ​​of p in the "no rounding" case and rows corresponding to odd values ​​of p in the "rounded" case) instead of 32 rows.

[0252] The technical effect of the present invention is that the present invention is implemented in at least the following types of configurations, namely, - An implementation without a LUT that calculates the filter coefficients using the value of p, based on the analytical formula described above, and - A LUT-based implementation that fetches filter coefficients from the LUT using index values ​​based on the value of p. The goal is to make this possible.

[0253] The precision of the value of p is reduced when calculating the coefficient value. Specifically, the coefficient value can be implemented as integer division by 2, i.e., a right shift of only 1, which is easily implemented in hardware.

[0254]

number

[0255] This is derived based on the results. In LUT-based implementations, this reduction in precision results in a reduction in the memory required to store the LUTs. In hardware, this also means a reduction in the number of wires on the LUT address bus.

[0256] According to one embodiment of the present disclosure, an apparatus for intra or interpredictive processing of video frames is provided, the apparatus comprising a reference sample acquisition unit configured to acquire a reference sample, a subpixel offset value acquisition unit configured to acquire a subpixel offset value, and a subpixel 4-tap interpolation filter configured to filter the reference sample to acquire a predictive sample value, wherein the filter coefficients of the subpixel 4-tap interpolation filter are

[0257]

number

[0258] The following conditions are met: p is the fractional part of the subpixel offset value, and c0, c1, c2, and c3 are the filter coefficients of the subpixel 4-tap interpolation filter.

[0259] According to one embodiment of the present disclosure, the filter coefficients of the subpixel 4-tap interpolation filter are given in the following table, i.e.,

[0260] [Table 11A]

[0261] [Table 11B]

[0262] It is defined within the following.

[0263] According to one embodiment of the present disclosure, an apparatus for intra or interpredictive processing of video frames is provided, the apparatus comprising a reference sample acquisition unit configured to acquire a reference sample, a subpixel offset value acquisition unit configured to acquire a subpixel offset value, and a subpixel 4-tap interpolation filter configured to filter the reference sample to acquire a predictive sample value, wherein the filter coefficients of the subpixel 4-tap interpolation filter are as follows, i.e.,

[0264] [Table 12]

[0265] It is defined as follows, where p is the fractional part of the subpixel offset value, and c0, c1, c2, and c3 are filter coefficients.

[0266] Figure 25 shows a schematic diagram of the intra / inter processing module 2500, which includes a reference sample acquisition unit 2510, a subpixel offset value acquisition unit 2520, and a subpixel interpolation filter 2530.

[0267] According to one embodiment of the present disclosure, an apparatus for video coding is provided, comprising an intra-prediction processing unit configured to intra-predict blocks of video frames, and a subpixel 4-tap interpolation filter configured to filter reference samples of video frames, wherein the subpixel 4-tap interpolation filter is a combination of filters, and at least one of the filters has a filter coefficient.

[0268]

number

[0269] The following conditions are met: p is the fractional sample position of the subpixel offset, and c0, c1, c2, and c3 are filter coefficients.

[0270] Figure 26 shows a schematic diagram of a video coding module 2600, which includes an intra-predictive processing unit 2610 and a subpixel interpolation filter 2620.

[0271] According to one embodiment of the present disclosure, an encoder (20) is provided having a processing circuit configuration for performing a method according to any one of the embodiments prior to the present disclosure.

[0272] According to one embodiment of the present disclosure, a decoder (30) is provided having a processing circuit configuration for performing a method according to any one of the embodiments prior to the present disclosure.

[0273] According to one embodiment of the present disclosure, a computer program product is provided which comprises program code for performing a method according to any one of the embodiments prior to the present disclosure.

[0274] According to one embodiment of the present disclosure, a decoder is provided comprising one or more processors and a non-temporary computer-readable storage medium coupled to the processors for storing a program for execution by the processors, wherein the decoder is configured to perform a method according to any one of the prior embodiments of the present disclosure when the program is executed by the processors.

[0275] According to one embodiment of the present disclosure, an encoder is provided comprising one or more processors and a non-temporary computer-readable storage medium coupled to the processors for storing a program for execution by the processors, wherein the program, when executed by the processors, configures the encoder to perform a method according to any one of the prior embodiments of the present disclosure.

[0276] Any of the preceding embodiments of this disclosure may offer the advantage of performing intra / interpretation of video frames at high speed because the filter coefficients of the interpolation filter are obtained analytically, i.e., the coefficients are calculated on the fly. This avoids storing the coefficients in a lookup table (LUT), which requires access time to read the coefficients from memory.

[0277] Therefore, the prediction becomes more efficient and requires less memory than necessary. This also enables a low-cost implementation of the prediction. Furthermore, the analytical filter coefficient {c i Since the expression is linear at fractional sample position p and involves division by 2, each operation can be efficiently performed by employing fast, low-level bitwise operations. The time required to perform the bitwise operations and to calculate the filter coefficients is shorter than the time required to access the stored coefficients from the LUT. Therefore, latency is reduced.

[0278] Furthermore, the specific analytical structure of the filter coefficients can offer the advantage of a low-complexity implementation of the filter unit. In addition, the filter response (i.e., frequency response) for different subpixel positions matches in terms of magnitude and phase, avoiding artifacts in the response, especially at high frequencies. The linearity of the filter coefficients can offer the advantage of hardware reuse.

[0279] In one embodiment, for a sharpening interpolation filter, the intensity parameter S is defined as follows:

[0280]

number

[0281] This intensity parameter S is defined to have a maximum value at the half-pel position (p=16). For subpixel positions closer to integer positions, the intensity parameter S has a smaller value. An exemplary combination of filters is shown in Figure 10. Block F1 in this figure represents a linear filter whose coefficients are defined as follows: c0 = 64 - (p << 1) c1 = p << 1

[0282] Block F2 represents a high-pass filter with the following coefficients: c0 = -1 c1=1 c2=1 c3 = -1

[0283] The output of block F2 is being multiplied by the intensity parameter S. The result of this multiplication is then summed together with the output of the linear filter F1, and the resulting sum is normalized.

[0284] The equation obtained for this combination (an equivalent 4-tap filter) can be expressed as follows: c0 = -S c1 = 64 - (p << 1) + S c² = (p << 1) + S c3 = -S

[0285] A LUT-based implementation is also possible for this formula. Table 5 (Table 13) provides the coefficient values.

[0286] [Table 13]

[0287] When filter F2 has a negative coefficient, the normalized result is further clipped to fit a tolerance range, which may be signaled in the SPS or derived from the sample's bit depth.

[0288] An alternative embodiment involves specifying an alternative position for the clipping operation (see Figure 11). This alternative design is based on the fact that the linear filter F1 does not need to have an output value that is lower or higher than any of its input values.

[0289] In one embodiment, the clipping block operates using the following steps.

[0290] Step 1 is to decide whether to perform minimum thresholding or maximum thresholding.

[0291] Step 2 depends on the results of Step 1. Specifically, either minimum thresholding or maximum thresholding is performed.

[0292] If the input to the clipping block is negative, minimum thresholding is applied. Otherwise, maximum thresholding is applied. Output v of step 2 (and the clipping block) c It can be defined as follows:

[0293]

number

[0294] For an input value v, the output v of the minimum thresholding process cmin It is calculated as follows:

[0295]

number

[0296] The output of the maximum thresholding process is calculated as follows:

[0297]

number

[0298] p max This is the maximum value within the acceptable range of the sample.

[0299] value ref x and ref x-1 This is the input reference sample shown in Figure 11, which is the input to the linear filter F1.

[0300] In one embodiment, if clipping is performed after the multiplication operation, v c It is calculated as follows:

[0301]

number

[0302] In some embodiments, the filter F3 in Figure 9 and the filters F2 in Figures 10 and 11 do not need to depend on the phase parameter P. Since the outputs of these filters may be the same for each input row of predicted samples, and therefore this step can be performed before the interpolation process begins, for example, at the stage of the reference sample filtering process, these embodiments simplify implementation. This technique increases parallelism and therefore reduces the latency of directional filtering.

[0303] Another alternative embodiment (see Figure 12) uses a filter of adjustable strength, as well as a multiplier of the filter output. Figure 12A shows an example where clipping is performed on the output of the high-pass filter F2. Figure 12B shows an example where clipping is performed on the output of the normalized output section of the combination of F1 and F2. Examples of filters of adjustable strength include, namely, bidirectional filters and inverse transform filters.

[0304] Another embodiment uses a reduced resolution of p, i.e., only 16 of the 32 entries are used. This is achieved, for example, by setting the least significant bit of p to 0.

[0305] As shown in Figures 13 to 21, various combinations of LUT-based and analytical representations of interpolation filter coefficients are possible for both intra-prediction and inter-prediction. For example, Figure 14 shows a use case where coefficients for a smoothing interpolation filter used in an intra-prediction example are calculated, while coefficients for other interpolation filters for both intra-prediction and inter-prediction are stored in the LUT. In addition, video coding specifications may include both LUT-based and analytical representations of interpolation filter coefficients to enable various implementation forms. When both representations are provided, the LUT-based and analytical representations should provide identical coefficients.

[0306] Figure 22 shows an implementation of the coefficient calculation method shown in Figure 1.

[0307] c0 = -S c1 = 64 - (p << 1) + S c² = (p << 1) + S c3 = -S In the following equation, the intensity parameter S is defined as follows:

[0308]

number

[0309] This implementation corresponds to a different method for deriving the intensity parameter S (shown as the obtained value of "q" in Figure 22). The value of the fractional sample position p is transferred into a demultiplexer DEMUX controlled by an input 1-bit signal SEL, which is set to the value of the fifth least significant bit of p (which has an index of 4 if indexing starts from 0). If this bit is equal to "1", the SEL signal indicates that the value of p is greater than 16, and the following values ​​should be calculated. q = -(not(31 xor p)) Here, "not" and "xor" are bitwise NOT and XOR operations, respectively. This expression is equivalent to q=(32-p), and in the C / C++ programming language... q = -(~(0x1F^p)) It can be written as follows.

[0310] Otherwise (i.e., if the input signal SEL of the demultiplexer DEMUX is equal to "0"), any calculations are bypassed and the signal p is transferred to the multiplexer MUX as signal q. The output signal r of the multiplexer MUX is then passed to a module that calculates the value of t as follows: t=((r<<1)+r)>>3 Here, "<<" and ">>" represent left-shift and right-shift operations, respectively.

[0311] In effect, this value of t is the intensity parameter S. Further optimization is possible by performing the steps shown in Figure 22 in parallel with one of the coefficients of the linear filter. z = (64 - (p << 1))

[0312] As soon as the values ​​of both z and S are calculated, they can be added together to obtain c1 = z + S.

[0313] While embodiments of the present invention are described primarily in relation to video coding, it should be noted that embodiments of the coding system 10, encoder 20, and decoder 30 (and correspondingly, system 10), as well as other embodiments described herein, may also be configured for still image processing or still image coding, i.e., for processing or coding individual pictures independent of any preceding or consecutive pictures, as in video coding. Generally, when picture processing coding is limited to a single picture 17, only the interpretation units 244 (encoder) and 344 (decoder) may not be available. All other functionalities (also called tools or techniques) of the video encoder 20 and video decoder 30 can be equally used for still image processing, such as residual calculation 204 / 304, transformation 206, quantization 208, inverse quantization 210 / 310, (inverse) transformation 212 / 312, segmentation 262 / 362, intra prediction 254 / 354 and / or loop filtering 220, 320, as well as entropy coding 270 and entropy decoding 304.

[0314] For example, embodiments of the encoder 20 and decoder 30, and the functions described herein with reference to the encoder 20 and decoder 30, may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or codes on a computer-readable medium, or transmitted over a communication medium, or executed by a hardware-based processing unit. The computer-readable medium may include a computer-readable storage medium that corresponds to a tangible medium such as a data storage medium, or a communication medium that includes, for example, any medium that facilitates the transfer of computer programs from one place to another according to a communication protocol. Thus, the computer-readable medium may generally correspond to (1) a non-transient tangible computer-readable storage medium, or (2) a communication medium such as a signal or carrier wave. The data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementing the techniques described herein. A computer program product may include a computer-readable medium.

[0315] For example, and without limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM, or other optical disk storage, magnetic disk storage, or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and can be accessed by a computer. Also, any connection is appropriately called a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of a medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carriers, signals, or other temporary media, but instead refer to non-temporary tangible storage media. As used herein, the terms "disk" and "disc" include Compact Disc (CD), LaserDisc® (disc), Optical Disc (disc), Digital Multipurpose Disc (disc) (DVD), Floppy Disk (disk), and Blu-ray® Disc (disc), where a disk typically reproduces data magnetically, and a disc reproduces data optically using a laser. Combinations of the above should also be included within the scope of computer-readable media.

[0316] Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable logic arrays (FPGAs), or other uniform integrated or discrete logic circuit configurations. Therefore, the term “processor” as used herein may refer to any of the above-described structures or any other structure suitable for implementing the techniques described herein. In addition, in some embodiments, the functionality described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into a combined codec. Furthermore, the techniques may be fully implemented within one or more circuits or logic elements.

[0317] The techniques of this disclosure can be implemented in a wide variety of devices or apparatus, including wireless handsets, integrated circuits (ICs), or sets of ICs (e.g., chipsets). Various components, modules, or units are described in this disclosure to highlight the functional aspects of a device configured to perform the techniques disclosed, but do not necessarily require implementation by various hardware units. Rather, as described above, various units may be combined within a codec hardware unit, or may be provided by a set of interoperable hardware units, including one or more processors as described above, along with suitable software and / or firmware.

[0318] In summary, this disclosure relates to intra or interprediction for video coding and decoding. For that purpose, the apparatus and method acquire a reference sample and subpixel offset values. To acquire predicted sample values, a subpixel 4-tap interpolation filter is used to filter the reference sample.

[0319]

number

[0320] As shown, the filter coefficients of the subpixel 4-tap interpolation filter are defined according to the value of the subpixel offset, where p is the fractional part of the value of the subpixel offset.

[0321] The following sections summarize additional embodiments.

[0322] Section 1: A method for an intra or interpredictive process of video coding technology, the method being The steps include obtaining a reference sample (i.e., at least one reference sample, in one example, the quantity of reference samples may be four), Steps to obtain the subpixel offset value, The process comprises the steps of filtering a reference sample using a subpixel interpolation filter to obtain a predicted sample value, wherein the filter coefficients of the subpixel interpolation filter are obtained according to the subpixel offset value.

[0323] Section 2: The method of Section 1, wherein the filter coefficients of the subpixel interpolation filter are obtained according to the fractional part of the subpixel offset value.

[0324] Section 3: The method of Section 1 or Section 2, wherein the subpixel interpolation filter is an equivalent 4-tap filter.

[0325] Section 4: The filter coefficients of the subpixel interpolation filter are,

[0326]

number

[0327] The method in Section 3 is obtained according to the formula, where p is the fractional part of the subpixel offset value.

[0328] Section 5: The filter coefficients of the subpixel interpolation filter are c0 = -S c1 = 64 - (p << 1) + S c² = (p << 1) + S c3 = -S The intensity parameter S is obtained according to the following, namely,

[0329]

number

[0330] The method in Section 3 is defined as follows, where p is the fractional part of the subpixel offset value.

[0331] Section 6: One of the methods described in Sections 1 through 5, which sets a bit of the subpixel offset p (for example, the least significant bit or the lowest bit) to 0.

[0332] Section 7: One of the methods described in Sections 1 through 5, in which the subpixel offset p is incremented by 1, the lower bit of the incremented value is set to 0, and then this value is used to derive the filter coefficients.

[0333] Section 8: A video coding method comprising an intra-prediction process of a block, which includes a sub-pixel interpolation filter process applied to a reference sample, A subpixel interpolation filter is a combination of filters, where at least one of the filters is determined according to the subpixel offset.

[0334] Section 9: The method of Section 8, wherein the interpolation filter is a smoothing filter which is a parallel combination of a linear filter and a low-pass filter that depend on two phases.

[0335] Section 10: The method of Section 8, wherein the interpolation filter is a sharpening filter which is a parallel combination of a linear filter dependent on one phase and an intensity-variable high-pass filter.

[0336] Section 11: The method of Section 10, wherein the intensity-variable high-pass filter is a phase-independent high-pass FIR filter, and its output is multiplied by a phase-dependent intensity parameter.

[0337] Section 12: The combination of filters,

[0338]

number

[0339] The method of Section 9 is an equivalent 4-tap filter having coefficients specified by the formula, where p is a fractional sample position p.

[0340] Section 13: The combination of filters, c0 = -S c1 = 64 - (p << 1) + S c² = (p << 1) + S c3 = -S This is an equivalent 4-tap filter having coefficients specified by the following equation, where the intensity parameter S is as follows, i.e.,

[0341]

number

[0342] The method of Section 11, as defined above.

[0343] Section 14: One of the methods in the previous section that sets the lower bit of the fractional sample position p to 0.

[0344] Section 15: One of the methods described in Sections 8 through 13, in which the fractional sample position p is incremented by 1, the lower bits of the incremented value are set to 0, and then this value is used to derive the interpolation filter coefficients.

[0345] Section 16: The method of Section 10 or 11, wherein a clipping operation is performed on the output of the intensity-variable high-pass filter.

[0346] Section 17: An encoder (20) having a processing circuit configuration for performing any one of the methods described in Sections 1 to 16.

[0347] Section 18: A decoder (30) having a processing circuit configuration for performing any one of the methods described in Sections 1 to 16.

[0348] Section 19: A computer program product comprising program code for performing any one of the methods described in Sections 1 through 16.

[0349] Section 20: Decoder, One or more processors, The system comprises a processor and a non-temporary computer-readable storage medium that stores a program for execution by the processor, wherein the program, when executed by the processor, configures the decoder to perform one of the methods described in Sections 1 to 16.

[0350] Section 21: Encoder, One or more processors, The system comprises a processor and a non-temporary computer-readable storage medium that stores a program for execution by the processor, wherein the program, when executed by the processor, configures the encoder to perform one of the methods described in Sections 1 to 16.

[0351] Section 22: A method for an intra or interpredictive process of video coding technology, the method being The steps include obtaining a reference sample (i.e., at least one reference sample, in one example, the quantity of reference samples may be four), Steps to obtain fractional sample positions, The process includes the step of filtering a reference sample using a subpixel interpolation filter to obtain a predicted sample value, wherein the filter coefficients of the subpixel interpolation filter are c0 = -S c1 = 64 - (p << 1) + S c² = (p << 1) + S c3 = -S The results are obtained according to the formula, where S is the intensity parameter, p is the fractional sample position, and c0, c1, c2, and c3 are the filter coefficients.

[0352] Section 23: Methods, A step of obtaining the value of the SEL signal by taking the Nth least significant bit of the fractional sample position p, where N is a positive integer, and a step of A step to obtain an intermediate value q, wherein when the SEL signal is 0, q is set to be equal to the value of the fractional sample position p, or when the SEL signal is not 0, q is set to -(not(31 xor p)), One addition and one shift operation, namely, S=((q<<1)+q)>>3 The method of Section 22 further comprises the step of obtaining an intensity parameter S using

[0353] Section 24: The method in Section 23, where N is equal to 5.

[0354] Section 25: The subpixel interpolation filter is an equivalent 4-tap filter, using one of the methods described in Sections 22-24.

[0355] Section 26: The intensity parameter S is calculated in parallel with the coefficients of the linear filter z, and the coefficients of the linear filter z are used to obtain the filter coefficients c1 of the subpixel interpolation filter. z = 64 - (p << 1) One of the methods described in sections 22 through 25.

[0356] Section 27: One of the methods described in Sections 22 through 25, which sets the bits of the fractional sample position p (for example, the least significant bit or the lowest bit) to 0.

[0357] Section 28: Any one of the methods described in Sections 22-26, in which the fractional sample position p is incremented by 1, the lower bit of the incremented value is set to 0, and then this value is used to derive the filter coefficients.

[0358] Section 29: A video coding method comprising an intra-prediction process of a block, which comprises a sub-pixel interpolation filter process applied to a reference sample, A subpixel interpolation filter is a combination of filters, where at least one of the filters is determined according to the fractional sample position.

[0359] Section 29: The method of Section 28, wherein the interpolation filter is a smoothing filter which is a parallel combination of a linear filter and a low-pass filter that depend on two phases.

[0360] Section 30: The method of Section 29, wherein the interpolation filter is a sharpening filter which is a parallel combination of a linear filter dependent on one phase and an intensity-variable high-pass filter.

[0361] Section 31: The method of Section 30, wherein the intensity-variable high-pass filter is a phase-independent high-pass FIR filter, and its output is multiplied by a phase-dependent intensity parameter.

[0362] Section 32: An encoder (20) having a processing circuit configuration for performing any one of the methods described in Sections 22 to 31.

[0363] Section 33: A decoder (30) having a processing circuit configuration for performing any one of the methods described in Sections 22 to 31.

[0364] Section 34: A computer program product comprising program code for performing any one of the methods described in Sections 22 through 31.

[0365] Section 35: Decoder, One or more processors, The system comprises a processor and a non-temporary computer-readable storage medium that stores a program for execution by the processor, wherein the program, when executed by the processor, configures the decoder to perform one of the methods described in Sections 22 to 31.

[0366] Section 36: Encoder, One or more processors, The system comprises a processor and a non-temporary computer-readable storage medium that stores a program for execution by the processor, wherein the program, when executed by the processor, configures the encoder to perform one of the methods described in Sections 22 to 31. [Explanation of Symbols]

[0367] Figure 1A 10 Video Coding Systems 12 Source Devices 13 Communication Channels 14 Destination device 16 Picture Sources 17 Picture Data 18 preprocessors 19 Pre-processed picture data 20 Video Encoders 21 Encoded Picture Data 22 Communication Interfaces 28 Communication Interfaces 30 video decoders 31 Decoded picture data 32 Post-Processors 33 Post-processed picture data 34 Display Devices Figure 1B 40 Video Coding Systems 41 Imaging devices 42 Antennas 43 processors 44 Memory Store 45 Display Devices 46 Processing Circuit Configuration 20 Video Encoders 30 video decoders Figure 2 17. Pictures (Data) 19 Pre-processed picture (data) 20 Video Encoders 21 Encoded Picture Data 201 Input section (interface) 204 Residual Calculation [Units or Steps] 206 Conversion Processing Unit 208 Quantization Units 210 Inverse Quantization Unit 212 Inverse Transform Processing Unit 214 Reconfiguration Unit 220 Loop Filter Unit 230 Decoded Picture Buffer (DPB) 260 Mode Selection Unit 270 Entropy Coding Units 272 Output section (interface) 244 Interpretation Units 254 Intra Prediction Units 262 division units 203 Picture Block 205 Residual Block 213 Reconstruction Residual Block 215 Reconstruction Blocks 221 Filtered Blocks 231 Decoded Picture 265 Prediction Blocks 266 Syntax Elements 207 Conversion coefficient 209 Quantization coefficients 211 Inverse Quantization Coefficient Figure 3 21 Encoded Picture Data 30 video decoders 304 Entropy Decoding Unit 309 Quantization coefficient 310 Inverse Quantization Unit 311 Inverse Quantization Coefficient 312 Inverse Transform Processing Unit 313 Reconstruction Residual Block 314 Reconfiguration Unit 315 Reconstruction Block 320 Loop Filter 321 Filtered Blocks 330 Decode Picture Buffer DBP 331 Decrypted Picture 360 Mode Applicable Unit 365 Prediction Block 366 Syntax Elements 344 Interpretation Units 354 Intra Prediction Units Figure 4 400 video coding devices 410 Inlet / Input Port 420 Receiver Unit Rx 430 processors 440 Transmitter Unit Tx 450 Exit Ports / Output Ports 460 memory 470 coding modules Figure 5 500 Source device or destination device 502 Processors 504 memory 506 Codes and Data 508 Operating Systems 510 Application Programs 512 Bus 518 displays Figure 23 2300 Flowchart of Intra / Interface Prediction Processing Method Figure 24 Flowchart of 2400 Video Coding Methods Figure 25 2500 Intra / Interprocessing Modules 2510 Reference Sample Acquisition Unit 2520 Subpixel Offset Value Acquisition Unit 2530 subpixel interpolation filter Figure 26 2600 video coding modules 2610 Intra Prediction Processing Unit 2620 subpixel interpolation filter

Claims

1. A method for intra or interpredictive processing of video frames, Steps to obtain a reference sample, Steps to obtain the subpixel offset value, The steps include: filtering the reference sample using a subpixel 4-tap interpolation filter to obtain a predicted sample value; Equipped with, The filter coefficients of the aforementioned subpixel 4-tap interpolation filter are, [Number 31] Satisfying the conditions, p is the fractional part of the aforementioned value of the subpixel offset, and c 0 , c 1 , c 2 , and c 3 However, the filter coefficients of the subpixel 4-tap interpolation filter are method.

2. The filter coefficients of the aforementioned subpixel 4-tap interpolation filter are given in the following table, namely, Table 1A Table 1B The method according to claim 1, as defined in [the text].

3. A method for intra or interpredictive processing of video frames, Steps to obtain a reference sample, Steps to obtain the subpixel offset value, The steps include: filtering the reference sample using a sub-pixel 4-tap interpolation filter to obtain a predicted sample value; Equipped with, The filter coefficients of the aforementioned subpixel 4-tap interpolation filter are as follows, that is, Table 2A Table 2B It is defined as follows: p is the fractional part of the value of the subpixel offset, and c 0 , c 1 , c 2 , and c 3 However, the filter coefficients are, method.

4. The method according to any one of claims 1 to 3, wherein the bit of p is set to 0.

5. The method according to any one of claims 1 to 3, wherein p is incremented by 1, then the lower bit of the incremented p is set to 0, and then p is used to obtain the filter coefficient.

6. Obtaining the aforementioned filter coefficients [Number 32] The method according to any one of claims 1 to 5, further comprising the step of calculating the filter coefficients accordingly.

7. A method of video coding, The process includes an intra-prediction process for a block of video frames, comprising the step of filtering a reference sample of the video frame using a subpixel 4-tap interpolation filter, The aforementioned subpixel 4-tap interpolation filter is a combination of filters, and at least one of the filter coefficients of the filters is [Number 33] Satisfying the conditions, p is the fractional sample position of the subpixel offset, and c 0 , c 1 , c 2 , and c 3 are the filter coefficients, method.

8. The method according to claim 7, wherein the subpixel 4-tap interpolation filter is a smoothing filter which is a parallel combination of two phase-dependent linear filters and a low-pass filter.

9. The method according to claim 7, wherein the subpixel 4-tap interpolation filter is a sharpening filter which is a parallel combination of a linear filter that depends on one phase and an intensity-variable high-pass filter.

10. The method according to claim 9, wherein the intensity-variable high-pass filter is a phase-independent high-pass finite impulse response FIR filter, and its output is multiplied by a phase-dependent intensity parameter.

11. The method according to any one of claims 7 to 10, wherein the lower bit of p is set to 0.

12. The method according to any one of claims 7 to 10, wherein p is incremented by 1, then the lower bits of the incremented value are set to 0, and then this value is used to derive the interpolation filter coefficient.

13. The method according to claim 9 or 10, wherein a clipping operation is performed on the output of the intensity-variable high-pass filter.

14. Apparatus for intra or interpredictive processing of video frames, A reference sample acquisition unit configured to acquire a reference sample, A subpixel offset value acquisition unit configured to acquire the value of the subpixel offset, A subpixel 4-tap interpolation filter configured to filter the reference sample in order to obtain predicted sample values, Includes, The filter coefficients of the aforementioned subpixel 4-tap interpolation filter are, [Number 34] Satisfying the conditions, p is the fractional part of the value of the subpixel offset, and c 0 , c 1 , c 2 , and c 3 However, the filter coefficients of the subpixel 4-tap interpolation filter are Device.

15. The filter coefficients of the aforementioned subpixel 4-tap interpolation filter are given in the following table, namely, Table 3A Table 3B The apparatus according to claim 14, as defined in [the text].

16. Apparatus for intra or interpredictive processing of video frames, A reference sample acquisition unit configured to acquire a reference sample, A subpixel offset value acquisition unit configured to acquire the value of the subpixel offset, A subpixel 4-tap interpolation filter configured to filter the reference sample in order to obtain predicted sample values, Equipped with, The filter coefficients of the aforementioned subpixel 4-tap interpolation filter are as follows, that is, Table 4A Table 4B It is defined as follows: p is the fractional part of the value of the subpixel offset, and c 0 , c 1 , c 2 , and c 3 However, the filter coefficients are, Device.

17. A device for video coding, An intra-prediction processing unit configured to perform intra-prediction on blocks of video frames, A subpixel 4-tap interpolation filter configured to filter the reference samples of the aforementioned video frame, Equipped with, The aforementioned subpixel 4-tap interpolation filter is a combination of filters, and at least one of the filter coefficients of the filters is [Number 35] Satisfying the conditions, p is the fractional sample position of the subpixel offset, and c 0 , c 1 , c 2 , and c 3 However, the filter coefficients are, Device.

18. An encoder (20) comprising a processing circuit configuration for performing the method described in any one of claims 1 to 13.

19. A decoder (30) comprising a processing circuit configuration for performing the method described in any one of claims 1 to 13.

20. A computer program product comprising program code for performing the method described in any one of claims 1 to 13.

21. It is a decoder, One or more processors, A non-temporary computer-readable storage medium coupled to the processor and storing a program for execution by the processor. Equipped with, When the programming is executed by the processor, the decoder is configured to perform the method according to any one of claims 1 to 13. decoder.

22. It is an encoder, One or more processors, A non-temporary computer-readable storage medium coupled to the processor and storing a program for execution by the processor. Equipped with, When the programming is performed by the processor, the encoder is configured to perform the method according to any one of claims 1 to 13. Encoder.

Images