Method and apparatus for decoding a block of an image, decoding apparatus, and computer readable medium

Abstract

The application provides a method and device for decoding a block of an image. The method comprises: for a pixel point in the block, using a DC intra prediction mode to perform intra prediction, obtaining a predicted pixel point value according to one or more reference pixel point values; multiplying the predicted pixel point value by a pixel point weighting factor; adding the weighted predicted pixel point value to other values; and normalizing the non-normalized predicted pixel point value by performing an arithmetic right shift on an integer representation of the non-normalized predicted pixel point value, wherein the pixel point weighting factor is ((2<<p)-wL-wT), and p is a parameter of the pixel point weighting factor, wL is a horizontal weighting factor, and wT is a vertical weighting factor.

Description

[0001] This application is a divisional application. The original application has the application number 201980067060.6 and the original application date is November 26, 2019. The entire contents of the original application are incorporated herein by reference. Technical Field

[0002] This invention relates to the technical field of image and / or video decoding, and more particularly to a method and apparatus for intra-frame prediction. Background Technology

[0003] Since the advent of DVDs, digital video has been widely used. Video is encoded before being sent, and then transmitted via a transmission medium. Viewers receive the video and use their viewing devices to decode and display it. Over the years, video quality has improved due to advancements in resolution, color depth, and frame rate. This has led to larger data streams currently transmitted via the internet and mobile communication networks.

[0004] However, higher resolution videos typically contain more information and therefore require more bandwidth. To reduce bandwidth requirements, video decoding standards involving video compression were introduced. When encoding video, the bandwidth requirement (or the corresponding memory requirement for storage) is reduced. This reduction often sacrifices quality. Therefore, video decoding standards attempt to find a balance between bandwidth requirements and quality.

[0005] High Efficiency Video Coding (HEVC) is an example of a video decoding standard well-known to those skilled in the art. In HEVC, the coding unit (CU) is divided into a prediction unit (PU) or a transform unit (TU). The next-generation standard for Versatile Video Coding (VVC) is a recent joint video project between the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Moving Picture Experts Group (MPEG). These two standardization organizations collaborated in a partnership known as the Joint Video Exploration Team (JVET). VVC is also known as the ITU-T H.266 / Next Generation Video Coding (NGVC) standard. VVC eliminates the concept of multiple segmentation types, meaning it does not distinguish between CU, PU, ​​and TU (unless the CU size is too large for the maximum transform length) and supports more flexible CU segmentation shapes.

[0006] The processing of these coding units (CUs) (also called blocks) depends on their size, spatial location, and the decoding mode specified by the encoder. Based on the prediction type, decoding modes can be divided into two categories: intra-frame prediction modes and inter-frame prediction modes. Intra-frame prediction modes use pixels (samples) from the same picture / image (also called a frame) to generate reference pixels and calculate the predicted values ​​for the pixels of the reconstructed block. Intra-frame prediction is also called spatial prediction. Inter-frame prediction modes are designed for temporal prediction and use reference pixels from previous or subsequent images to predict the pixels of blocks in the current image.

[0007] ITU-T VCEG (Q6 / 16) and ISO / IEC MPEG (JTC 1 / SC 29 / WG 11) are studying the potential need for standardization of future video decoding technologies, where the compression capabilities will significantly exceed those of the current HEVC standard (including current and recent extensions for screen content decoding and high dynamic range decoding). These two expert groups are jointly conducting this exploration, known as the Joint Video Exploration Team (JVET), to evaluate compression technology designs proposed by their experts in this area.

[0008] The Versatile Test Model (VTM) version 3.0 uses 93 intra-prediction modes and several intra-smoothing tools, including 4-tap subpixel intra-interpolation filtering and position-dependent prediction combination (PDPC). PDPC is a unified mechanism for modifying predicted pixels that are the result of intra-prediction using DC, planar, or angular intra-prediction modes. Summary of the Invention

[0009] This invention provides apparatus and methods for improving intra-frame prediction of the current block in an image. The invention is defined by the independent claims. The dependent claims describe preferred embodiments. Other implementations will be apparent from the description and drawings.

[0010] According to a first aspect, a method for intra-frame prediction of blocks in an image is provided. The method includes: for pixels in the block, performing intra-frame prediction using a DC intra-frame prediction mode; obtaining predicted pixel values ​​based on one or more reference pixel values; multiplying the predicted pixel values ​​by a pixel weighting factor to obtain a weighted predicted pixel value; adding the weighted predicted pixel value to other values ​​to obtain a non-normalized predicted pixel value; and normalizing the non-normalized predicted pixel value by performing an arithmetic right shift on the integer representation of the non-normalized predicted pixel value, wherein the pixel weighting factor is ((2 << p) - wL - wT), where p is a parameter of the pixel weighting factor, wL is a horizontal weighting factor, and wT is a vertical weighting factor.

[0011] According to a second aspect, a method for intra-frame prediction of a first block and a second block in an image is provided. The method includes: for pixels in the first block and pixels in the second block: performing intra-frame prediction using an intra-frame prediction mode; obtaining (S100) predicted pixel values ​​based on one or more reference pixel values; multiplying the predicted pixel values ​​by a pixel weighting factor to obtain a weighted predicted pixel value; adding the weighted predicted pixel value to other values ​​to obtain a non-normalized predicted pixel value; normalizing the non-normalized predicted pixel value by performing an arithmetic right shift on the integer representation of the non-normalized predicted pixel value, wherein the pixel weighting factor is ((2 << p) - wL - wT), where p is a parameter of the pixel weighting factor, wL is a horizontal weighting factor, and wT is a vertical weighting factor; the intra-frame prediction mode used to obtain the predicted pixel values ​​of the first block is a DC intra-frame prediction mode, and the intra-frame prediction mode used to obtain the predicted pixel values ​​of the second block is a planar intra-frame prediction mode.

[0012] For example, the parameter of the pixel weighting factor is the precision of the pixel weighting factor.

[0013] In one embodiment, the normalized predicted pixel value can be the final result of the prediction process.

[0014] The method provided by the first or second aspect can improve intra-frame prediction of the current block, wherein erroneous predicted pixel values ​​can be avoided when performing DC intra-frame prediction on the current pixel.

[0015] In one embodiment, a planar intra-frame prediction mechanism is used to compute the other values.

[0016] This reduces complexity by simplifying the intra-frame prediction process, as the already implemented planar intra-frame prediction mechanism can be reused to compute other values ​​used to determine the non-normalized predicted pixel values.

[0017] In one embodiment, the pixel weighting factor is (64-wL-wT).

[0018] In one embodiment, the other value is the sum of one or more addends, including addends determined based on one or more of the reference pixels.

[0019] For example, the one or more addends may include a rounding offset.

[0020] Adding a rounding offset enables the correct rounding of the arithmetic right shift result of the integer representation of the non-normalized predicted pixel.

[0021] In one embodiment, the addend determined based on one or more reference pixels is wL×R. -1,y +wT× R x,-1 , where R x,-1 and R -1,y This represents the value of the nearest reference pixel located above and to the left of the predicted pixel.

[0022] In one embodiment, the image is part of a video sequence.

[0023] In one embodiment, the horizontal weighting factor wL and the vertical weighting factor wT are powers of 2.

[0024] This allows the multiplication calculation with the weighting factor to be performed by shifting the integer representation of the factor to be multiplied by the weighting factor.

[0025] In one embodiment, the horizontal weighting factor is wL = (2 << (p-1)) >> ((x << 1) >> nScale), where x is the horizontal coordinate of the pixel; the vertical weighting factor is wT = (2 << (p-1)) >> ((y << 1) >> nScale), where y is the vertical coordinate of the pixel; and nScale is the scaling parameter.

[0026] In one embodiment, the scaling parameter nScale is derived based on the size of the block.

[0027] The parameter nScale, derived from the size of the block, enables the calculation of horizontal and vertical weights in a suitable manner, thereby improving prediction efficiency.

[0028] In one embodiment, the scaling parameter nScale is determined to be ((Log2(nTbW)+Log2(nTbH)-2)>>2), where nTbW is the width of the block and nTbH is the height of the block.

[0029] In one embodiment, calculating a normalized predicted pixel value based on the predicted pixel value includes calculating...

[0030] (wL×R -1,y +wT×R x,-1 +(64-wL-wT)×P(x,y)+32)>>6

[0031] in,

[0032] P(x,y) is the predicted pixel value.

[0033] R x,-1 and R -1,y This represents the value of the nearest reference pixel located above and to the left of the predicted pixel.

[0034] wL is the level weighting factor.

[0035] wT is the vertical weighting factor.

[0036] In this embodiment, the added rounding offset is 32.

[0037] This ensures that the result of the right shift operation >> 6 is correctly rounded.

[0038] In one embodiment, the unnormalized predicted pixel values ​​are normalized to obtain normalized predicted pixel values.

[0039] In one embodiment, the plurality of pixels in the block includes each pixel in the block.

[0040] A method for encoding or decoding an image is also provided. The method includes: obtaining normalized predicted pixel values ​​by performing any of the steps in the above methods; and adding the normalized predicted pixel values ​​to a residual value to obtain reconstructed pixel values.

[0041] According to a third aspect, an apparatus for encoding or decoding images is provided. The apparatus includes processing circuitry for performing any of the methods described above.

[0042] In one embodiment, the processing circuitry includes one or more processors and a non-transitory computer-readable medium connected to the one or more processors. The non-transitory computer-readable medium includes program code; when the one or more processors execute the program code, the device performs the method.

[0043] According to a fourth aspect, a non-transitory computer-readable medium is provided. The non-transitory computer-readable medium includes program code; when a computer device executes the program code, the computer device performs any of the methods described above.

[0044] According to a fifth aspect, a computer program is provided. The computer program includes program code for performing any of the methods described above.

[0045] According to one aspect, an encoder device is provided. The encoder device is used to perform intra-frame prediction on blocks in an image. The encoder device includes: an acquirer, configured to: perform intra-frame prediction using a DC intra-frame prediction mode, and acquire predicted pixel values ​​based on one or more reference pixel values; a multiplier, configured to multiply the predicted pixel values ​​by a pixel weighting factor to obtain a weighted predicted pixel value; an adder, configured to add the weighted predicted pixel value to other values ​​to obtain a non-normalized predicted pixel value; and a normalizer, configured to: normalize the non-normalized predicted pixel value by performing an arithmetic right shift on the integer representation of the non-normalized predicted pixel value, wherein the pixel weighting factor is ((2 << p) - wL - wT), where p is a parameter of the pixel weighting factor, wL is a horizontal weighting factor, and wT is a vertical weighting factor.

[0046] According to one aspect, an encoder device is provided. The encoder device is used to perform intra-frame prediction on a first block and a second block in an image. The encoder device includes: an acquirer, configured to: perform intra-frame prediction using an intra-frame prediction mode, and acquire predicted pixel values ​​based on one or more reference pixel values; a multiplier, configured to multiply the predicted pixel values ​​by a pixel weighting factor to obtain a weighted predicted pixel value; an adder, configured to add the weighted predicted pixel value to other values ​​to obtain a non-normalized predicted pixel value; and a normalizer, configured to: normalize the non-normalized predicted pixel value by performing an arithmetic right shift on the integer representation of the non-normalized predicted pixel value, wherein the pixel weighting factor is ((2 << p) - wL - wT), where p is a parameter of the pixel weighting factor, wL is a horizontal weighting factor, and wT is a vertical weighting factor. The intra-frame prediction mode used to obtain the predicted pixel values ​​of the pixels in the first block is DC intra-frame prediction mode, and the intra-frame prediction mode used to obtain the predicted pixel values ​​of the pixels in the second block is planar intra-frame prediction mode.

[0047] According to one aspect, a decoder device is provided. The decoder device is used to perform intra-frame prediction on blocks in an image. The decoder device includes: an acquirer, configured to: perform intra-frame prediction using a DC intra-frame prediction mode, and acquire predicted pixel values ​​based on one or more reference pixel values; a multiplier, configured to multiply the predicted pixel values ​​by a pixel weighting factor to obtain a weighted predicted pixel value; an adder, configured to add the weighted predicted pixel value to other values ​​to obtain a non-normalized predicted pixel value; and a normalizer, configured to: normalize the non-normalized predicted pixel value by performing an arithmetic right shift on the integer representation of the non-normalized predicted pixel value, wherein the pixel weighting factor is ((2 << p) - wL - wT), where p is a parameter of the pixel weighting factor, wL is a horizontal weighting factor, and wT is a vertical weighting factor.

[0048] According to one aspect, a decoder device is provided. The decoder device is used to perform intra-frame prediction on a first block and a second block in an image. The decoder device includes: an acquirer, configured to: perform intra-frame prediction using an intra-frame prediction mode, and acquire predicted pixel values ​​based on one or more reference pixel values; a multiplier, configured to multiply the predicted pixel values ​​by a pixel weighting factor to obtain a weighted predicted pixel value; an adder, configured to add the weighted predicted pixel value to other values ​​to obtain a non-normalized predicted pixel value; and a normalizer, configured to: normalize the non-normalized predicted pixel value by performing an arithmetic right shift on the integer representation of the non-normalized predicted pixel value, wherein the pixel weighting factor is ((2 << p) - wL - wT), where p is a parameter of the pixel weighting factor, wL is a horizontal weighting factor, and wT is a vertical weighting factor. The intra-frame prediction mode used to obtain the predicted pixel values ​​of the pixels in the first block is DC intra-frame prediction mode, and the intra-frame prediction mode used to obtain the predicted pixel values ​​of the pixels in the second block is planar intra-frame prediction mode.

[0049] According to one aspect, a predictor device for intra-frame prediction of blocks in an image includes: an acquirer, configured to: perform intra-frame prediction using a DC intra-frame prediction mode for pixels in the block, and acquire predicted pixel values ​​based on one or more reference pixel values; a multiplier, configured to multiply the predicted pixel value by a pixel weighting factor to obtain a weighted predicted pixel value; an adder, configured to add the weighted predicted pixel value to other values ​​to obtain a non-normalized predicted pixel value; and a normalizer, configured to: normalize the non-normalized predicted pixel value by performing an arithmetic right shift on the integer representation of the non-normalized predicted pixel value, wherein the pixel weighting factor is ((2 << p) - wL - wT), where p is a parameter of the pixel weighting factor, wL is a horizontal weighting factor, and wT is a vertical weighting factor.

[0050] According to one aspect, a predictor device for intra-frame prediction of a first block and a second block in an image includes: an acquirer, configured to: perform intra-frame prediction using an intra-frame prediction mode for pixels in the first block or the second block, and acquire predicted pixel values ​​based on one or more reference pixel values; a multiplier, configured to multiply the predicted pixel values ​​by a pixel weighting factor to obtain a weighted predicted pixel value; an adder, configured to add the weighted predicted pixel value to other values ​​to obtain a non-normalized predicted pixel value; and a normalizer, configured to: normalize the non-normalized predicted pixel value by performing an arithmetic right shift on the integer representation of the non-normalized predicted pixel value, wherein the pixel weighting factor is ((2 << p) - wL - wT), where p is a parameter of the pixel weighting factor, wL is a horizontal weighting factor, and wT is a vertical weighting factor. The intra-frame prediction mode used to obtain the predicted pixel values ​​of the first block is DC intra-frame prediction mode, and the intra-frame prediction mode used to obtain the predicted pixel values ​​of the second block is planar intra-frame prediction mode.

[0051] According to one aspect, a method for intra-frame prediction of blocks in an image includes: for a pixel (x, y) in the block,

[0052] Intra-frame prediction is performed using DC intra-frame prediction mode, and the predicted pixel value P(x,y) is obtained based on one or more reference pixel values.

[0053] Based on the predicted pixel value PP(x,y) and the reference pixel values ​​R(x,–1) and R(–1,y), a predicted pixel value P'(x,y) is generated, where,

[0054] P'(x,y)=(wL×R(–1,y)+wT×R(x,–1)+(64–wL–wT)×P(x,y)+32))>>6

[0055] Wherein, the reference pixel value R(x,–1) is the value of the pixel (x,–1) located above the block, the reference pixel value R(–1,y) is the value of the pixel (–1,y) located to the left of the block, wL is the horizontal weighting factor, and wT is the vertical weighting factor.

[0056] The following drawings and description illustrate one or more embodiments in detail. Other features, objects, and advantages will be apparent from the description, drawings, and claims. Attached Figure Description

[0057] The embodiments of the present invention will now be described in more detail with reference to the accompanying drawings and schematic diagrams, wherein:

[0058] Figure 1AA block diagram illustrating an example of a video decoding system for implementing embodiments of the present invention;

[0059] Figure 1B A block diagram illustrating another example of a video decoding system for implementing embodiments of the present invention;

[0060] Figure 2 A block diagram illustrating an example of a video encoder used to implement an embodiment of the present invention;

[0061] Figure 3 This is a block diagram of an exemplary structure for implementing a video decoder according to an embodiment of the present invention;

[0062] Figure 4 A block diagram of an example encoding or decoding device;

[0063] Figure 5 A block diagram of another example of an encoding or decoding device;

[0064] Figure 6 The intra-frame prediction direction and mode of the angle and the p of the vertical prediction direction are shown. ang The relevant values;

[0065] Figure 7 The p of a 4×4 block is shown. ref Transform to p 1,ref An example;

[0066] Figure 8 This illustrates the construction of p for horizontal angle prediction. 1,ref An example;

[0067] Figure 9 This illustrates the construction of p for vertical angle prediction. 1,ref An example;

[0068] Figure 10A An example of angular intra-prediction direction and associated intra-prediction mode in JEM and BMS-1 ​​is shown;

[0069] Figure 10B An example of angular intra-prediction direction and associated intra-prediction mode in VTM-2 is shown;

[0070] Figure 10C An example of angular intra-prediction direction and associated intra-prediction mode in VTM-3 is shown;

[0071] Figure 11 An example of angular intra-prediction direction and associated intra-prediction mode in HEVC is shown;

[0072] Figure 12 An example of QTBT is shown;

[0073] Figure 13 An example of DC-mode PDPC weights for positions (0, 0) and (1, 0) within a 4×4 block is shown;

[0074] Figure 14 An example of intra-frame prediction of a block based on reference pixels on the main reference edge is shown;

[0075] Figure 15 An embodiment of a method for intra-frame prediction of blocks in an image is shown;

[0076] Figure 16A An encoder device or decoder device according to one embodiment is shown;

[0077] Figure 16B An encoder device or decoder device according to one embodiment is shown;

[0078] Figure 17 An example of DC-mode PDPC weights for positions (0, 0) and (1, 0) within a 4×4 block is shown;

[0079] Figure 18 An example of intra-frame prediction for pixel blocks is shown.

[0080] In the following text, the same reference numerals denote the same features or at least functionally equivalent features, unless otherwise expressly specified. Detailed Implementation

[0081] In the following description, reference is made to the accompanying drawings, which form part of this invention and illustrate specific aspects of embodiments of the invention or from which specific aspects of embodiments of the invention may be used. It should be understood that embodiments of the invention may be used in other aspects and may include structural or logical variations not depicted in the drawings. Therefore, the following detailed description should not be construed in a limiting sense, and the scope of the invention is defined by the appended claims.

[0082] For example, it should be understood that the disclosure relating to the described method can also apply to the corresponding device or system for performing the method, and vice versa. For example, if one or more specific method steps are described, the corresponding device may include one or more units (e.g., functional units) to perform the described one or more method steps (e.g., one unit performs one or more steps, or multiple units perform one or more of a plurality of steps respectively), even if such one or more units are not explicitly described or illustrated in the drawings. On the other hand, for example, if a specific apparatus is described according to one or more units (e.g., functional units), the corresponding method may include a step to perform the function of one or more units (e.g., one step performs the function of one or more units, or multiple steps perform the function of one or more of a plurality of units respectively), even if such one or more steps are not explicitly described or illustrated in the drawings. Furthermore, it should be understood that, unless otherwise expressly stated, features of the various exemplary embodiments and / or aspects described herein can be combined with each other.

[0083] Video decoding generally refers to the processing of image sequences that form a video or video sequence. In the field of video decoding, the terms "frame" and "picture / image" can be used synonymously. Video decoding (or decoding in general) consists of two parts: video encoding and video decoding. Video encoding is performed on the source side and typically involves processing (e.g., by compression) the raw video image to reduce the amount of data required to represent the video image (thus enabling more efficient storage and / or transmission). Video decoding is performed on the destination side and typically involves inverse processing relative to the encoder to reconstruct the video image. The "decoding" of the video image (or image in general) mentioned in the embodiments should be understood as the "encoding" or "decoding" of the video image or corresponding video sequence. The encoding and decoding parts are also collectively referred to as codec (CODEC) (encoding and decoding).

[0084] In lossless video decoding, the original video image can be reconstructed, meaning the reconstructed video image has the same quality as the original (assuming no transmission loss or other data loss during storage or transmission). In lossy video decoding, further compression is performed through quantization to reduce the amount of data representing the video image, and the decoder cannot completely reconstruct the video image, meaning the quality of the reconstructed video image is lower or worse than the quality of the original video image.

[0085] Several video decoding standards belong to the "lossy hybrid video codec" group (e.g., combining spatial and temporal prediction in the pixel domain with 2D transform decoding in the transform domain for applying quantization). Each image in a video sequence is typically segmented into a set of non-overlapping blocks, and decoding is typically performed at the block level. In other words, on the encoder side, the video is typically processed (i.e., encoded) at the block (video block) level, for example, generating prediction blocks through spatial (intra-frame) prediction and / or temporal (inter-frame) prediction; subtracting the prediction blocks from the current block (the currently processed block / the block to be processed) to obtain residual blocks; transforming and quantizing the residual blocks in the transform domain to reduce the amount of data to be sent (compressed), while on the decoder side, the encoded or compressed blocks are processed inversely relative to the encoder to reconstruct the representation of the current block. Furthermore, the encoder and decoder processing steps are identical, such that the encoder and decoder generate the same prediction blocks (e.g., intra-frame and inter-frame prediction blocks) and / or reconstructed blocks for processing (e.g., decoding) of subsequent blocks.

[0086] In the following embodiment of the video decoding system 10, the video encoder 20 and the video decoder 30 are configured according to Figures 1 to 30. Figure 3 Describe it.

[0087] Figure 1A This is a schematic block diagram of an exemplary decoding system 10, such as a video decoding system 10 (or simply decoding system 10) that can utilize the technology described in this application. The video encoder 20 (or simply encoder 20) and video decoder 30 (or simply decoder 30) in the video decoding system 10 are one example and can be devices that implement the technology using various examples described in this application.

[0088] like Figure 1A As shown, the decoding system 10 includes a source device 12, which provides encoded image data 21 to a destination device 14, etc., for decoding the encoded image data 13.

[0089] The source device 12 includes an encoder 20 and may additionally (optionally) include an image source 16, a preprocessor (or preprocessing unit) 18 (e.g., an image preprocessor 18), and a communication interface or communication unit 22.

[0090] Image source 16 may include or may be any type of image capture device for capturing real-world images, etc.; and / or any type of image generation device (e.g., a computer graphics processor for generating computer-animated images); or any type of device for acquiring and / or providing real-world images, computer-animated images (e.g., screen content, virtual reality (VR) images) and / or any combination thereof (e.g., augmented reality (AR) images). Image source may be any type of memory / storage for storing any of the aforementioned images.

[0091] To distinguish between the processing performed by the preprocessor 18 and the preprocessing unit 18, the image or image data 17 may also be referred to as the raw image or raw image data 17.

[0092] The preprocessor 18 receives (raw) image data 17 and performs preprocessing on the image data 17 to obtain a preprocessed image 19 or preprocessed image data 19. The preprocessing performed by the preprocessor 18 may include cropping, color format conversion (e.g., from RGB to YCbCr), color correction, or noise reduction. It is understood that the preprocessing unit 18 may be an optional component.

[0093] Video encoder 20 is used to receive preprocessed image data 19 and provide encoded image data 21 (in combination) Figure 2 (Wait for more details).

[0094] The communication interface 22 in the source device 12 can be used to receive encoded image data 21 and send the encoded image data 21 (or data obtained after further processing of the encoded image data 21) to another device (e.g., destination device 14) or any other device via the communication channel 13 for storage or direct reconstruction.

[0095] Destination device 14 includes decoder 30 (e.g., video decoder 30) and may additionally (optionally) include communication interface or communication unit 28, post-processor 32 (or post-processing unit 32) and display device 34.

[0096] The communication interface 28 in the destination device 14 is used to receive, for example, encoded image data 21 (or data obtained after further processing of encoded image data 21) directly from the source device 12 or from any other source such as a storage device (e.g., an encoded image data storage device) and to provide the encoded image data 21 to the decoder 30.

[0097] Communication interfaces 22 and 28 can be used to send or receive encoded image data 21 or encoded data 13 via a direct communication link (e.g., a direct wired or wireless connection) between source device 12 and destination device 14 or via any type of network (e.g., a wired network, a wireless network or any combination thereof, or any type of private and public network or any combination thereof).

[0098] For example, the communication interface 22 can be used to encapsulate the encoded image data 21 into a suitable format (e.g., data packets) and / or process the encoded image data through any type of transmission encoding or processing method for transmission over a communication link or communication network.

[0099] For example, the communication interface 28 corresponding to the communication interface 22 can be used to receive transmitted data and process the transmitted data through any type of corresponding transmission decoding or processing and / or decapsulation method to obtain encoded image data 21.

[0100] Both communication interface 22 and communication interface 28 can be configured as Figure 1A The communication channel 13, from source device 12 to destination device 14, is a one-way communication interface indicated by the arrow, or configured as a two-way communication interface, and can be used to send and receive messages, etc., to establish connections, acknowledge and exchange any other information related to the communication link and / or data transmission (e.g., encoded image data transmission), etc.

[0101] Decoder 30 is used to receive encoded image data 21 and provide decoded image data 31 or decoded image 31 (hereinafter combined) Figure 3 or Figure 5 (Wait for more details).

[0102] The post-processor 32 in the destination device 14 is used to post-process the decoded image data 31 (also known as reconstructed image data) (e.g., decoded image 31) to obtain post-processed image data 33 (e.g., post-processed image 33). The post-processing performed by the post-processing unit 32 may include color format conversion (e.g., from YCbCr to RGB), color adjustment, trimming or resampling, or any other processing, in order to provide the decoded image data 31 for display by the display device 34, etc.

[0103] The display device 34 in the destination device 14 is used to receive post-processed image data 33 in order to display the image to a user or viewer. The display device 34 can be or may include any type of display (e.g., integrated or external display or screen) to represent the reconstructed image. For example, the display may include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro LED display, a liquid crystal on silicon (LCoS), a digital light processor (DLP), or any other type of display.

[0104] although Figure 1A The source device 12 and destination device 14 are shown as separate devices. However, in embodiments, the device may also include the functions of both the source device 12 and the destination device 14, i.e., the source device 12 or its corresponding function and the destination device 14 or its corresponding function. In these embodiments, the source device 12 or its corresponding function and the destination device 14 or its corresponding function may be implemented using the same hardware and / or software, or using separate hardware and / or software, or any combination thereof.

[0105] According to the description, Figure 1A The presence and (precise) division of different units or functions in the source device 12 and / or destination device 14 shown may vary depending on the actual device and application, which is obvious to a person skilled in the art.

[0106] Encoder 20 (e.g., video encoder 20) and decoder 30 (e.g., video decoder 30) can be implemented as any of the various suitable circuits shown in FIG. 1B, such as one or more microprocessors, one or more digital signal processors (DSPs), one or more application-specific integrated circuits (ASICs), one or more field-programmable gate arrays (FPGAs), one or more discrete logic, one or more hardware, or any combination thereof. If the foregoing technical portions are implemented in software, a device can store the instructions of the software in a suitable non-transitory computer-readable medium, and can execute these instructions in hardware using one or more processors to perform the techniques of the present invention. Any of the foregoing (including hardware, software, combinations of hardware and software, etc.) can be considered as one or more processors. Video encoder 20 and video decoder 30 can both be included in one or more encoders or decoders, which can be integrated in the respective device as part of a combined encoder / decoder (CODEC).

[0107] Source device 12 and destination device 14 can include any of a variety of devices, including any type of handheld or fixed device, such as laptops, mobile phones, smartphones, tablets or tablets, cameras, desktop computers, set-top boxes, televisions, display devices, digital media players, video game consoles, video streaming devices (e.g., content service servers or content distribution servers), broadcast receiver devices, etc., and may or may not use any type of operating system. In some cases, source device 12 and destination device 14 can be used for wireless communication. Therefore, source device 12 and destination device 14 can be wireless communication devices.

[0108] In some cases, Figure 1A The video decoding system 10 shown is merely exemplary, and the technology of this application can be applied to video decoding (e.g., video encoding or video decoding) setups where there is not necessarily any data communication between the encoding and decoding devices. In other examples, data is retrieved from local memory, sent over a network, etc. The video encoding device can encode data and store it in memory, and / or the video decoding device can retrieve data from memory and decode it. In some examples, encoding and decoding are performed by devices that do not communicate with each other but simply encode data into memory and / or retrieve data from memory and decode it.

[0109] Figure 1B A schematic diagram of an exemplary video decoding system 40 provided for an exemplary embodiment. The video decoding system 40 includes... Figure 2 Encoder 20 and / or Figure 3 The decoder 30 is located in the system 40. The system 40 can implement the techniques provided in the various examples described in this application. In the illustrated implementation, the video decoding system 40 may include one or more imaging devices 41, a video encoder 100, a video decoder 30 (and / or a video decoder implemented by logic circuitry 47 in one or more processing units 46), an antenna 42, one or more processors 43, one or more memories 44, and / or a display device 45.

[0110] As shown in the figure, one or more imaging devices 41, antenna 42, one or more processing units 46, logic circuits 47, video encoder 20, video decoder 30, one or more processors 43, one or more memories 44, and / or display device 45 are capable of communicating with each other. As described above, although the video decoding system 40 is shown to include a video encoder 20 and a video decoder 30, in various examples, the video decoding system 40 may include only the video encoder 20 or only the video decoder 30.

[0111] As shown in the figure, in some examples, the video decoding system 40 may include an antenna 42. For example, the antenna 42 may be used to transmit or receive encoded streams of video data. Furthermore, in some examples, the video decoding system 40 may include a display device 45. The display device 45 may be used to present the video data. As shown in the figure, in some examples, the logic circuitry 47 may be implemented by one or more processing units 46. The one or more processing units 46 may include application-specific integrated circuit (ASIC) logic, one or more graphics processors, one or more general-purpose processors, etc. The video decoding system 40 may also include one or more optional processors 43, which may similarly include application-specific integrated circuit (ASIC) logic, one or more graphics processors, one or more general-purpose processors, etc. In some examples, the logic circuitry 47 may be implemented by hardware or dedicated video decoding hardware, and the one or more processors 43 may be implemented by general-purpose software or an operating system, etc. Additionally, one or more memories 44 can be any type of memory, such as volatile memory (e.g., Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), etc.) or non-volatile memory (e.g., flash memory), etc. In a non-limiting example, one or more memories 44 can be implemented using cache memory. In some examples, logic circuitry 47 can access one or more memories 44 (for implementing image buffers, etc.). In other examples, logic circuitry 47 and / or one or more processing units 46 may include memory (e.g., cache, etc.) for implementing image buffers, etc.

[0112] In some examples, the video encoder 20 implemented via logic circuitry may include an image buffer (e.g., implemented via one or more processing units 46 or one or more memories 44) and a graphics processing unit (e.g., implemented via one or more processing units 46). The graphics processing unit may be communicatively coupled to the image buffer. The graphics processing unit may include the video encoder 20 implemented via logic circuitry 47 to contain references... Figure 2 The various modules described herein and / or any other encoder systems or subsystems described herein. Logic circuits may be used to perform the various operations described herein.

[0113] The video decoder 30 can be similarly implemented via logic circuit 47 to include a reference. Figure 3The decoder 30 described herein includes various modules and / or any other decoder systems or subsystems described herein. In some examples, the video decoder 30, implemented via logic circuitry, may include (e.g., implemented via one or more processing units 420 or one or more memories 44) an image buffer and (e.g., implemented via one or more processing units 46) a graphics processing unit. The graphics processing unit may be communicatively coupled to the image buffer. The graphics processing unit may include the video decoder 30 implemented via logic circuitry 47 to contain references to... Figure 3 The various modules described herein and / or any other decoder systems or subsystems described herein.

[0114] In some examples, antenna 42 in video decoding system 40 can be used to receive an encoded bitstream of video data. As described above, the encoded bitstream may include data related to video frame encoding as described herein, indicators, index values, mode selection data, etc., such as data related to decoding segmentation (e.g., transform coefficients or quantization transform coefficients, optional indicators (as described above), and / or data defining decoding segmentation). Video decoding system 40 may also include video decoder 30 coupled to antenna 42 and used for decoding the encoded bitstream. Display device 45 is used to display video frames.

[0115] For ease of description, this document (for example) refers to, on the example, the High-Efficiency Video Coding (HEVC) or Versatile Video Coding (VVC) reference software developed by the Joint Collaboration Team on Video Coding (JCT-VC) of the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Moving Picture Experts Group (MPEG). Those skilled in the art will understand that the embodiments of the present invention are not limited to HEVC or VVC.

[0116] Encoders and Encoding Methods

[0117] Figure 2 This is a schematic block diagram of an exemplary video encoder 20 used to implement the technology of this application. Figure 2In the example, the video encoder 20 includes an input terminal 201 (or input interface 201), a residual calculation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, a reconstruction unit 214, a loop filter unit 220, a decoded picture buffer (DPB) 230, a mode selection unit 260, an entropy coding unit 270, and an output terminal 272 (or output interface 272). The mode selection unit 260 may include an inter-frame prediction unit 244, an intra-frame prediction unit 254, and a segmentation unit 262. The inter-frame prediction unit 244 may include a motion estimation unit and a motion compensation unit (not shown). Figure 2 The video encoder 20 shown can also be called a hybrid video encoder or a video encoder based on a hybrid video codec.

[0118] The residual calculation unit 204, transform processing unit 206, quantization unit 208, and mode selection unit 260 can form the forward signal path of the encoder 20, while the inverse quantization unit 210, inverse transform processing unit 212, reconstruction unit 214, buffer 216, loop filter 220, decoded picture buffer (DPB) 230, inter-frame prediction unit 244, and intra-frame prediction unit 254 can form the backward signal path of the video encoder 20. The backward signal path of the video encoder 20 corresponds to the decoder (see [link to decoder]). Figure 3 The signal path of the video decoder 30 in the video encoder 20. The inverse quantization unit 210, inverse transform processing unit 212, reconstruction unit 214, loop filter 220, decoded picture buffer (DPB) 230, inter-frame prediction unit 244 and intra-frame prediction unit 254 also constitute the "built-in decoder" of the video encoder 20.

[0119] Image and image segmentation (images and patches)

[0120] Encoder 20 can be used to receive image 17 (or image data 17) via input terminal 201, etc. Image 17 can be an image in an image sequence that forms a video or video sequence. The received image or image data can also be a pre-processed image 19 (or pre-processed image data 19). For simplicity, the following description uses image 17. Image 17 can also be referred to as the current image or the image to be decoded (especially in video decoding, to distinguish the current image from other images (e.g., previously encoded and / or decoded images) in the same video sequence (i.e., a video sequence that also includes the current image).

[0121] A digital image is, or can be viewed as, a two-dimensional array or matrix of pixels with intensity values. Pixels in an array are also called pixels (short for image element). The number of pixels in the horizontal and vertical directions (or axes) of the array or image defines the image size and / or resolution. To represent color, three color components are typically used, meaning an image can be represented as or comprise an array of three pixels. In RGB format or color space, an image includes corresponding arrays of red, green, and blue pixels. However, in video decoding, each pixel is typically represented in a luminance and chrominance format or color space, such as YCbCr, which includes the luminance component represented by Y (sometimes also L) and two chrominance components represented by Cb and Cr. The luminance component Y represents the brightness or grayscale intensity (e.g., both are the same in a grayscale image), while the two chrominance components Cb and Cr represent the chrominance or color information components. Accordingly, a YCbCr format image consists of a luminance pixel array for the luminance pixel value (Y) and two chrominance pixel arrays for the chrominance values ​​(Cb and Cr). An RGB format image can be converted or transformed to YCbCr format, and vice versa. This process is also known as color transformation or conversion. If the image is black and white, it may only include the luminance pixel array. Accordingly, the image can be, for example, a black and white format luminance pixel array or a 4:2:0, 4:2:2, and 4:4:4 color format luminance pixel array and two corresponding chrominance pixel arrays.

[0122] In an embodiment, the video encoder 20 may include an image segmentation unit ( Figure 2 (Not shown in the image) is used to segment image 17 into multiple (typically non-overlapping) image blocks 203. These blocks may also be referred to as root blocks, macroblocks (H.264 / AVC), coding tree blocks (CTBs), or coding tree units (CTUs) (H.265 / HEVC and VVC). The image segmentation unit can be used to apply the same block size and a corresponding grid with a defined block size to all images in a video sequence, or to vary the block size between images, subsets of images, or groups of images, and segment each image into corresponding blocks.

[0123] In other embodiments, the video encoder may be used to directly receive blocks 203 in image 17, such as one, several, or all of the blocks that make up image 17. Image block 203 may also be referred to as the current image block or the image block to be decoded.

[0124] Similar to image 17, image block 203 is also a two-dimensional array or matrix composed of pixels with intensity values ​​(pixel values), but the size of image block 203 is smaller than that of image 17. In other words, block 203 may include, for example, a single pixel array (e.g., a luminance array in the case of black and white image 17, or a luminance or chrominance array in the case of a color image), or a three-pixel array (e.g., a luminance array and two chrominance arrays in the case of color image 17), or any other number and / or type of array depending on the color format used. The number of pixels in the horizontal and vertical directions (or axes) of block 203 defines the size of block 203. Accordingly, a block may be an M×N (M columns × N rows) pixel array, or an M×N transform coefficient array, etc.

[0125] In an embodiment, Figure 2 The video encoder 20 shown can be used to encode the image 17 block by block, for example, to perform encoding and prediction for each block 203.

[0126] Residual calculation

[0127] The residual calculation unit 204 is used to calculate the residual block 205 (also called residual 205) in the pixel domain based on the image block 203 and the prediction block 265 (which is described in detail later). For example, the pixel values ​​of the prediction block 265 are subtracted from the pixel values ​​of the image block 203 pixel by pixel.

[0128] Transformation

[0129] The transformation processing unit 206 can be used to perform discrete cosine transform (DCT) or discrete sine transform (DST) on the pixel values ​​of the residual block 205 to obtain the transformation coefficients 207 in the transform domain. The transformation coefficients 207 can also be called transformation residual coefficients, representing the residual block 205 in the transform domain.

[0130] Transform processing unit 206 can be used to apply integer approximations of DCT / DST, such as those specified for H.265 / HEVC. Compared to orthogonal DCT transforms, this integer approximation is typically scaled by a certain factor. To maintain the norm of the residual block after both forward and inverse transforms, other scaling factors are used as part of the transform process. These scaling factors are usually selected based on certain constraints, such as powers of 2 used for shift operations, the bit depth of the transform coefficients, and a trade-off between accuracy and implementation cost. For example, a specific scaling factor is specified for the inverse transform (and the corresponding inverse transform on the video decoder 30 side) via inverse transform processing unit 212, etc.; correspondingly, a corresponding scaling factor can be specified for the forward transform on the encoder 20 side via transform processing unit 206, etc.

[0131] In an embodiment, the video encoder 20 (correspondingly, the transform processing unit 206) can be used to output transform parameters such as the type of one or more transforms, for example, directly output or output after being encoded or compressed by the entropy encoding unit 270, so that the video decoder 30 can receive and use the transform parameters for decoding.

[0132] Quantification

[0133] Quantization unit 208 is used to quantize the transform coefficients 207 using scalar quantization or vector quantization, etc., to obtain quantized transform coefficients 209. Quantization coefficients 209 can also be called quantized transform coefficients 209 or quantized residual coefficients 209.

[0134] The quantization process can reduce the bit depth associated with some or all of the transform coefficients 207. For example, n-bit transform coefficients can be rounded down to m-bit transform coefficients during quantization, where n is greater than m. The degree of quantization can be modified by adjusting the quantization parameter (QP). For example, for scalar quantization, different degrees of scaling can be applied to achieve finer or coarser quantization. Smaller quantization steps correspond to finer quantization, while larger quantization steps correspond to coarser quantization. The appropriate quantization step size can be represented by the quantization parameter (QP). For example, the quantization parameter can be an index of a predefined set of appropriate quantization steps. For example, a smaller quantization parameter can correspond to fine quantization (smaller quantization step size), and a larger quantization parameter can correspond to coarse quantization (larger quantization step size), and vice versa. Quantization may include division by the quantization step size, while corresponding and / or dequantization performed by the dequantization unit 210, etc., may include multiplication by the quantization step size. Quantization parameters can be used to determine the quantization step size according to embodiments of some standards such as HEVC. Generally, the quantization step size can be calculated using a fixed-point approximation of an equation involving division based on the quantization parameter. Additional scaling factors can be introduced for quantization and dequantization to recover the norm of the residual block, which may have been modified by scaling used in the fixed-point approximation of the equations for the quantization step size and quantization parameters. In one exemplary implementation, scaling for inverse transform and dequantization can be combined. Alternatively, a custom quantization table can be used and transmitted from the encoder to the decoder via a signal in the bitstream, etc. Quantization is a lossy operation, where the loss increases with the quantization step size.

[0135] In an embodiment, the video encoder 20 (correspondingly, the quantization unit 208) can be used to output quantization parameters (QP), for example, directly or after being encoded by the entropy coding unit 270, so that the video decoder 30 can receive and use the quantization parameters for decoding.

[0136] Inverse Quantization

[0137] The dequantization unit 210 is used to dequantize the quantization coefficients using the quantization unit 208, resulting in dequantized coefficients 211. For example, it executes a dequantization scheme that is the opposite of the quantization scheme executed by the quantization unit 208, using the same quantization step size as the quantization unit 208. The dequantization coefficients 211 can also be called dequantization residual coefficients 211, corresponding to the transform coefficients 207. However, due to the loss caused by quantization, the dequantization coefficients 211 are usually different from the transform coefficients.

[0138] Inverse Transformation

[0139] The inverse transform processing unit 212 performs the inverse transform of the transform processing unit 206, such as the inverse discrete cosine transform (DCT) or the inverse discrete sine transform (DST), to obtain the reconstructed residual block 213 (or the corresponding dequantization coefficients 213) in the pixel domain. The reconstructed residual block 213 can also be called the transform block 213.

[0140] reconstruction

[0141] The reconstruction unit 214 (e.g., adder or summer 214) is used to add the transform block 213 (i.e., reconstruction residual block 213) to the prediction block 265 to obtain the reconstruction block 215 in the pixel domain by, for example, adding the pixel values ​​of the reconstruction residual block 213 and the pixel values ​​of the prediction block 265 pixel by pixel.

[0142] Filtering

[0143] Loop filter unit 220 (or simply "loop filter" 220) is used to filter the reconstructed block 215 to obtain the filtered block 221, or generally to filter the reconstructed pixels to obtain the filtered pixels. For example, the loop filter unit is used to smoothly perform pixel transformation or improve video quality. Loop filter unit 220 may include one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or one or more other filters, such as a bilateral filter, an adaptive loop filter (ALF), a sharpening or smoothing filter, a co-filter, or any combination thereof. Although loop filter unit 220 is used in... Figure 2 The loop filter unit 220 is shown as an in-loop filter, but in other configurations, it can be implemented as a post-loop filter. The filtered block 221 can also be called the filtered reconstructed block 221. After the loop filter unit 220 performs filtering on the reconstructed decoded block, the decoded image buffer 230 can store the reconstructed decoded block.

[0144] In an embodiment, the video encoder 20 (correspondingly, the loop filter unit 220) can be used to output loop filter parameters (e.g., pixel adaptive offset information), for example, directly output or after being encoded by the entropy coding unit 270, so that the decoder 30 can receive and use the same loop filter parameters or the corresponding loop filter for decoding.

[0145] Decoding image buffer

[0146] The decoded picture buffer (DPB) 230 can be a memory that stores reference images or generally stores reference image data for use by the video encoder 20 when encoding video data. The DPB 230 can be formed from any of a variety of memory devices, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. The decoded picture buffer (DPB) 230 can be used to store one or more filtered blocks 221. The decoded picture buffer 230 can also be used to store other previously filtered blocks (e.g., previously filtered reconstructed blocks 221) in the same current image or different images (e.g., previous reconstructed images), and can provide previously fully reconstructed (i.e., decoded) images (and corresponding reference blocks and pixels) and / or partially reconstructed current images (and corresponding reference blocks and pixels) for inter-frame prediction, etc. If the reconstructed block 215 is not filtered by the loop filter unit 220, the decoded picture buffer (DPB) 230 can also be used to store one or more unfiltered reconstructed blocks 215, or generally store unfiltered reconstructed pixels, or reconstructed blocks or reconstructed pixels that have not undergone any other processing.

[0147] Pattern selection (segmentation and prediction)

[0148] The mode selection unit 260 includes a segmentation unit 262, an inter-frame prediction unit 244, and an intra-frame prediction unit 254, and is used to receive or acquire raw image data such as the original block 203 (the current block 203 in the current image 17) and reconstructed image data (e.g., filtered and / or unfiltered reconstructed pixels or blocks from the same (current) image and / or one or more previous decoded images) from the decoded image buffer 230 or other buffers (e.g., a line buffer, not shown in the figure). The reconstructed image data is used as reference image data for prediction such as inter-frame prediction or intra-frame prediction to obtain prediction block 265 or prediction value 265.

[0149] The mode selection unit 260 can be used to determine or select a segmentation for the current block prediction mode (including no segmentation) and to determine or select a prediction mode (e.g., intra-frame prediction mode or inter-frame prediction mode) to generate the corresponding prediction block 265 for calculating the residual block 205 and reconstructing the reconstructed block 215.

[0150] In embodiments, the mode selection unit 260 can be used to select a segmentation and prediction mode (e.g., from prediction modes supported or available by the mode selection unit 260), which provides the best match or minimum residual (minimum residual refers to better compression in transmission or storage), or provides minimum signaling overhead (minimum signaling overhead refers to better compression in transmission or storage), or considers or balances both. The mode selection unit 260 can be used to determine the segmentation and prediction mode based on rate distortion optimization (RDO), i.e., selecting the prediction mode that provides minimum rate distortion. The terms "best," "minimum," and "optimal" used herein do not necessarily refer to "best," "minimum," or "optimal" overall, but can also refer to situations that meet termination or selection criteria. For example, values ​​exceeding or falling below a threshold or other constraints may lead to a "suboptimal choice," but will reduce complexity and processing time.

[0151] In other words, the segmentation unit 262 can be used to segment block 203 into smaller block segments or sub-blocks (reforming blocks) in such a way as: for example, by iteratively using quad-tree (QT) segmentation, binary-tree (BT) segmentation, or triple-tree (TT) segmentation, or any combination thereof, and is used to (for example) perform a prediction on each of the block segments or sub-blocks, wherein the mode selection includes selecting the tree structure of the segmented block 203 and selecting the prediction mode used for each of the block segments or sub-blocks.

[0152] The segmentation (e.g., by segmentation unit 260) and prediction processing (performed by inter-frame prediction unit 244 and intra-frame prediction unit 254) performed by the exemplary video encoder 20 will be described in detail below.

[0153] segmentation

[0154] Segmentation unit 262 can divide (or partition) the current block 203 into smaller segments, such as smaller blocks the size of a square or rectangle. These smaller blocks (also called sub-blocks) can be further divided into even smaller segments. This is also called tree segmentation or hierarchical tree segmentation. A root block at root level 0 (level 0, depth 0), etc., can be recursively segmented into two or more blocks at the next lower tree level, such as nodes at tree level 1 (level 1, depth 1). These blocks can then be further segmented into two or more blocks at the next lower level, such as tree level 2 (level 2, depth 2), etc., until the segmentation ends (because the termination criterion is met, such as reaching the maximum tree depth or minimum block size). Blocks that are not further segmented are also called leaf blocks or leaf nodes of the tree. A tree divided into two segments is called a binary-tree (BT), a tree divided into three segments is called a ternary-tree (TT), and a tree divided into four segments is called a quad-tree (QT).

[0155] As stated above, the term "block" as used herein can be a portion of an image, particularly a square or rectangular portion. For example, referring to HEVC and VVC, a block can be or may correspond to a coding tree unit (CTU), a coding unit (CU), a prediction unit (PU), and a transform unit (TU) and / or corresponding blocks, such as a coding tree block (CTB), a coding block (CB), a transform block (TB), or a prediction block (PB).

[0156] For example, a coding tree unit (CTU) can be or may include a CTB for luma pixels in an image with a 3-pixel array, two corresponding CTBs for chroma pixels in the same image, or a CTB for pixels in a black-and-white image or an image decoded using 3 separate color planes and syntax structures. These syntax structures are used to decode the pixels. Correspondingly, a coding tree block (CTB) can be an N×N pixel block, where N can be set to a value such that a component is divided into CTBs; this is called partitioning. A coding unit (CU) can be or may include a coding block for luma pixels in an image with a 3-pixel array, two corresponding coding blocks for chroma pixels in the same image, or a coding block for pixels in a black-and-white image or an image decoded using 3 separate color planes and syntax structures. These syntax structures are used to decode the pixels. Correspondingly, a coding block (CB) can be an M×N pixel block, where M and N can be set to a certain value so that a CTB is divided into a coding block, which is called segmentation.

[0157] In an embodiment, for example according to HEVC, a coding tree unit (CTU) can be divided into multiple CUs using a quadtree structure represented as a decoding tree. At the CU level, it is determined whether to use inter-frame (temporal) prediction or intra-frame (spatial) prediction to decode the image region. Each CU can be further divided into one, two, or four PUs depending on the PU partitioning type. The same prediction process is performed within a PU, and relevant information is sent to the decoder on a PU-by-PU basis. After obtaining residual blocks by performing the prediction process according to the PU partitioning type, the CU can be further divided into transform units (TUs) according to other quadtree structures similar to those used for the CU.

[0158] In an embodiment, for example, according to the latest currently developed video decoding standard (called Versatile Video Coding (VVC)), quad-tree and binary-tree (QTBT) segmentation is used to segment the decoding blocks. In the QTBT block structure, a CU can be a square or a rectangle. For example, the coding tree unit (CTU) is first segmented using a quad-tree structure. The leaf nodes of the quad-tree are further segmented using a binary or ternary / triple tree structure. The segmented leaf nodes are called coding units (CUs), and such segments are used for prediction and transform processing without any further segmentation. This means that in the QTBT decoding block structure, the block sizes of CUs, PUs, and TUs are the same. Meanwhile, it is also proposed to use multiple segmentations, such as ternary tree segmentation, with the QTBT block structure.

[0159] In one example, the mode selection unit 260 in the video encoder 20 can be used to perform any combination of the segmentation techniques described herein.

[0160] As described above, the video encoder 20 is used to determine or select the best or optimal prediction mode from a (predetermined) prediction mode set. The prediction mode set may include intra-frame prediction modes and / or inter-frame prediction modes, etc.

[0161] Intra-frame prediction

[0162] The intra-prediction mode set can include 35 different intra-prediction modes, such as non-directional modes like DC (or mean) mode and planar mode, or directional modes as defined in HEVC, or it can include 67 different intra-prediction modes, such as non-directional modes like DC (or mean) mode and planar mode, or directional modes as defined in VVC.

[0163] Intra-prediction unit 254 is used to generate intra-prediction block 265 by using the reconstructed pixels of adjacent blocks in the same current image according to the intra-prediction mode in the intra-prediction mode set.

[0164] Intra-prediction unit 254 (or collectively referred to as mode selection unit 260) is also used to output intra-prediction parameters (or collectively referred to as information representing the selected intra-prediction mode of the block) to entropy coding unit 270 in the form of syntax element 266, so as to be included in the encoded image data 21, so that, for example, video decoder 30 can receive and use the prediction parameters for decoding.

[0165] Inter-frame prediction

[0166] The set of possible inter-frame prediction modes depends on the available reference image (i.e., at least partially decoded image stored in the DPB 230 as described above) and other inter-frame prediction parameters, such as whether the entire reference image or only a portion of the reference image (e.g., the search window region around the current block region) is used to search for the best matching reference block, and / or, for example, whether pixel interpolation (e.g., half-pixel interpolation and / or quarter-pixel interpolation) is performed.

[0167] In addition to the prediction modes mentioned above, skip mode and / or direct mode can also be used.

[0168] Inter-frame prediction unit 244 may include a motion estimation (ME) unit and a motion compensation (MC) unit (both in... Figure 2 (Not shown in the image). The motion estimation unit can be used to receive or acquire image block 203 (current image block 203 in current image 17) and decoded image 231, or at least one or more previous reconstructed blocks (e.g., reconstructed blocks in one or more other / different previous decoded images 231) for motion estimation. For example, the video sequence may include the current image and the previously decoded image 231, or in other words, the current image and the previously decoded image 231 may be part of or constitute the image sequence that makes up the video sequence.

[0169] For example, encoder 20 can be used to select a reference block from multiple reference blocks in the same or different images in multiple other images, and provide the offset (spatial offset) between the position (x-coordinate, y-coordinate) of the reference image (or reference image index) and / or the reference block and the position of the current block as an inter-frame prediction parameter to the motion estimation unit. This offset is also called a motion vector (MV).

[0170] The motion compensation unit is used to acquire (e.g., receive) inter-frame prediction parameters and perform inter-frame prediction based on or using the inter-frame prediction parameters to obtain inter-frame prediction blocks 265. Motion compensation performed by the motion compensation unit may include extracting or generating prediction blocks based on motion / block vectors determined by motion estimation, and may also include performing interpolation with sub-pixel precision. Interpolation filtering can generate pixels of other pixels based on pixels of known pixels, thereby potentially increasing the number of candidate prediction blocks that can be used to decode image blocks. Once the motion vector of the PU corresponding to the current image block is received, the motion compensation unit can locate the prediction block pointed to by the motion vector in one of the reference image lists.

[0171] The motion compensation unit can also generate syntax elements related to blocks and video slices for the video decoder 30 to use when decoding image blocks of video slices.

[0172] Entropy decoding

[0173] Entropy coding unit 270 is used to apply or not apply entropy coding algorithms or schemes (such as variable length coding (VLC) scheme, context adaptive VLC scheme (CAVLC) scheme, arithmetic decoding scheme, binarization, context adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) decoding, or other entropy coding methods or techniques) to (uncompressed) quantization coefficients 209, inter-frame prediction parameters, intra-frame prediction parameters, loop filter parameters, and / or other syntax elements to obtain encoded image data 21 that can be output by output terminal 272 in the form of encoded bitstream 21, so that (for example) video decoder 30 can receive and use these parameters for decoding. The encoded bitstream 21 can be sent to the video decoder 30, or stored in memory for later transmission or retrieval by the video decoder 30.

[0174] Other structural variations of the video encoder 20 can be used to encode the video stream. For example, a non-transform-based encoder 20 can directly quantize the residual signal in some blocks or frames where there is no transform processing unit 206. In another implementation, the encoder 20 may include a quantization unit 208 and an inverse quantization unit 210 combined into a single unit.

[0175] Decoder and Decoding Method

[0176] Figure 3 An example of a video decoder 30 used to implement the technology of this application is shown. The video decoder 30 is used to receive, for example, encoded image data 21 (e.g., encoded bitstream 21) encoded by encoder 20, to obtain a decoded image 331. The encoded image data or bitstream includes information for decoding the encoded image data, such as data representing image blocks in the encoded video strip and associated syntax elements.

[0177] exist Figure 3 In the example, decoder 30 includes an entropy decoding unit 304, an inverse quantization unit 310, an inverse transform processing unit 312, a reconstruction unit 314 (e.g., a summer 314), a loop filter 320, a decoded picture buffer (DPB) 330, an inter-frame prediction unit 344, and an intra-frame prediction unit 354. The inter-frame prediction unit 344 may be or may include a motion compensation unit. In some examples, video decoder 30 may perform substantially the same functions as the referenced unit. Figure 2 The video encoder 100 describes the encoding round as the inverse of the decoding round.

[0178] As described with reference to encoder 20, the inverse quantization unit 210, inverse transform processing unit 212, reconstruction unit 214, loop filter 220, decoded picture buffer (DPB) 230, inter-frame prediction unit 344, and intra-frame prediction unit 354 also constitute the "built-in decoder" of video encoder 20. Accordingly, inverse quantization unit 310 can be functionally identical to inverse quantization unit 110, inverse transform processing unit 312 can be functionally identical to inverse transform processing unit 212, reconstruction unit 314 can be functionally identical to reconstruction unit 214, loop filter 320 can be functionally identical to loop filter 220, and decoded picture buffer 330 can be functionally identical to decoded picture buffer 230. Therefore, the explanation of the corresponding units and functions of video encoder 20 is correspondingly applicable to the corresponding units and functions of video decoder 30.

[0179] Entropy Decoding

[0180] The entropy decoding unit 304 is used to parse the bitstream 21 (or generally the encoded image data 21) and perform entropy decoding on the encoded image data 21 to obtain the quantization coefficients 309 and / or decoding parameters. Figure 3 (Not shown in the image) Examples of parameters include inter-frame prediction parameters (e.g., reference image index and motion vector), intra-frame prediction parameters (e.g., intra-frame prediction mode or index), transform parameters, quantization parameters, loop filter parameters, and / or other syntax elements. The entropy decoding unit 304 can be used to apply a decoding algorithm or scheme corresponding to the encoding scheme described by the entropy coding unit 270 in the reference encoder 20. The entropy decoding unit 304 can also be used to provide inter-frame prediction parameters, intra-frame prediction parameters, and / or other syntax elements to the mode selection unit 360, and to provide other parameters to other units of the decoder 30. The video decoder 30 can receive video strip-level and / or video block-level syntax elements.

[0181] Inverse Quantization

[0182] The dequantization unit 310 can be used to receive quantization parameters (QP) (or generally information related to dequantization) and quantization coefficients from the encoded image data 21 (e.g., parsed and / or decoded by the entropy decoding unit 304, etc.), and dequantize the decoded quantization coefficients 309 according to these quantization parameters to obtain dequantization coefficients 311. Dequantization coefficients 311 can also be called transform coefficients 311. The dequantization process may include determining the degree of quantization using the quantization parameters determined by the video encoder 20 for each video block in the video strip, and also determining the degree of dequantization to be performed.

[0183] Inverse Transformation

[0184] The inverse transform processing unit 312 can be used to receive the dequantized coefficients 311 (also called transform coefficients 311) and transform the dequantized coefficients 311 to obtain the reconstructed residual block 213 in the pixel domain. The reconstructed residual block 213 can also be called the transform block 313. The transform can be an inverse transform, such as inverse DCT, inverse DST, inverse integer transform, or a conceptually similar inverse transform process. The inverse transform processing unit 312 can also be used to receive transform parameters or corresponding information from the encoded image data 21 (e.g., parsed and / or decoded by the entropy decoding unit 304, etc.) to determine the transform to be performed on the dequantized coefficients 311.

[0185] reconstruction

[0186] The reconstruction unit 314 (e.g., adder or summer 314) can be used to add the reconstruction residual block 313 to the prediction block 365 to obtain the reconstruction block 315 in the pixel domain by, for example, adding the pixel values ​​in the reconstruction residual block 313 to the pixel values ​​in the prediction block 365.

[0187] Filtering

[0188] Loop filter unit 320 (in or after the decoding loop) is used to filter the reconstructed block 315 to obtain the filtered block 321, thereby facilitating pixel transformation or improving video quality. Loop filter unit 320 may include one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or one or more other filters, such as a bilateral filter, an adaptive loop filter (ALF), a sharpening or smoothing filter, a co-filter, or any combination thereof. Although loop filter unit 320 is in... Figure 3 The loop filter unit 320 is shown as an in-loop filter, but in other configurations, it can be implemented as a post-loop filter.

[0189] Decoding image buffer

[0190] Subsequently, the decoded video block 321 in one image is stored in the decoded image buffer 330, which stores the decoded image 331 as a reference image. Motion compensation and / or output or display are then performed on other images.

[0191] The decoder 30 is used to output the decoded image 311 through the output terminal 312, etc., for display to the user or for the user to view.

[0192] predict

[0193] Inter-frame prediction unit 344 may be identical to inter-frame prediction unit 244 (particularly to motion compensation unit), and intra-frame prediction unit 354 may be functionally identical to intra-frame prediction unit 254, determining segmentation or partitioning and performing prediction based on segmentation and / or prediction parameters or corresponding information received from encoded image data 21 (e.g., parsed and / or decoded by entropy decoding unit 304, etc.). Mode selection unit 360 may be used to perform prediction (intra-frame prediction or inter-frame prediction) for each block based on the reconstructed image, block, or corresponding pixel (filtered or unfiltered), resulting in prediction block 365.

[0194] When a video stripe is decoded into an intra-decoded (I) stripe, the intra-prediction unit 354 in the mode selection unit 360 generates a prediction block 365 for the current video stripe based on the signaled intra-prediction mode and data from the previous decoded block in the current image. When a video image is decoded into an inter-decoded (B or P) stripe, the inter-prediction unit 344 (e.g., a motion compensation unit) in the mode selection unit 360 generates a prediction block 365 for the current video stripe based on motion vectors and other syntax elements received from the entropy decoding unit 304. For inter-prediction, these prediction blocks can be generated based on one of the reference images in one of the reference image lists. The video decoder 30 can construct reference frame lists 0 and 1 using a default construction technique based on the reference images stored in the DPB 330.

[0195] The mode selection unit 360 is used to determine the prediction information of video blocks in the current video strip by parsing motion vectors and other syntax elements, and to generate prediction blocks for the current video block being decoded using the prediction information. For example, the mode selection unit 360 uses some received syntax elements to determine the prediction mode (e.g., intra-frame prediction or inter-frame prediction) for decoding video blocks in the video strip, the inter-frame prediction stripe type (e.g., B stripe, P stripe, or GPB stripe), the construction information for one or more reference image lists for the stripe, the motion vectors for each inter-frame encoded video block in the stripe, the inter-frame prediction state for each inter-frame decoded video block in the stripe, and other information to decode video blocks in the current video stripe.

[0196] Other variations of the video decoder 30 can be used to decode the encoded image data 21. For example, the decoder 30 can generate an output video stream without the loop filter unit 320. For example, the non-transform-based decoder 30 can directly dequantize the residual signal in certain blocks or frames without the inverse transform processing unit 312. In another implementation, the video decoder 30 may include a dequantization unit 310 and an inverse transform processing unit 312 combined into a single unit.

[0197] Figure 4 This is a schematic diagram of a video decoding device 400 provided for an embodiment of the present invention. The video decoding device 400 is suitable for implementing the disclosed embodiments described herein. In one embodiment, the video decoding device 400 may be a decoder (e.g., Figure 1A The video decoder 30 or encoder (e.g.) in the video decoder 30) or encoder Figure 1A (Video encoder 20 in the middle).

[0198] The video decoding device 400 includes: an input port 410 and a receiving unit (Rx) 420 for receiving data; a processor, logic unit, or central processing unit (CPU) 430 for processing the data; a transmitting unit (Tx) 440 and an output port 450 for transmitting the data; and a memory 460 for storing the data. The video decoding device 400 may also include optical-to-electrical (OE) components and electro-optical (EO) components coupled to the input port 410, the receiving unit 420, the transmitting unit 440, and the output port 450, serving as input or output points for optical or electrical signals.

[0199] Processor 430 is implemented in both hardware and software. Processor 430 can be implemented as one or more CPU chips, one or more cores (e.g., a multi-core processor), one or more FPGAs, one or more ASICs, and one or more DSPs. Processor 430 communicates with ingress port 410, receiver unit 420, transmitter unit 440, egress port 450, and memory 460. Processor 430 includes decoding module 470. Decoding module 470 implements the disclosed embodiments described above. For example, decoding module 470 performs, processes, prepares, or provides various decoding operations. Therefore, including decoding module 470 provides a substantial improvement to the functionality of video decoding device 400 and affects the transitions of video decoding device 400 to different states. Alternatively, decoding module 470 can be implemented with instructions stored in memory 460 and executed by processor 430.

[0200] Memory 460 may include one or more disks, tape drives, and solid-state drives, and may be used as an overflow data storage device to store programs as selected for execution, as well as instructions and data read during program execution. For example, memory 460 may be volatile and / or non-volatile, and may be read-only memory (ROM), random-access memory (RAM), ternary content-addressable memory (TCAM), and / or static random-access memory (SRAM).

[0201] Figure 5 A simplified block diagram of the apparatus 500 provided for an exemplary embodiment. The apparatus 500 can be used as... Figure 1A The device 500 can be any one or both of the source device 12 and the destination device 14. The device 500 can implement the technology described above. The device 500 can be a computing system including multiple computing devices, or a single computing device, such as a mobile phone, tablet computer, laptop computer, desktop computer, etc.

[0202] The processor 502 in device 500 may be a central processing unit. Alternatively, processor 502 may be any other type of device or multiple devices, existing or to be developed in the future, capable of manipulating or processing information. While the disclosed implementation may be implemented using a single processor, such as processor 502 as shown, using multiple processors can improve speed and efficiency.

[0203] In one implementation, the memory 504 in device 500 may be a read-only memory (ROM) device or a random access memory (RAM) device. Any other suitable type of storage device may be used as memory 504. Memory 504 may include code and data 506 accessed by processor 502 via bus 512. Memory 504 may also include an operating system 508 and an application program 510, which includes at least one program that causes processor 502 to perform the methods described herein. For example, application program 510 may include application 1 through application N, and also includes a video decoding application that performs the methods described herein. Device 500 may also include other memory in the form of auxiliary memory 514, which may be a memory card or the like used with a mobile computing device. Since video communication sessions may include a large amount of information, it may be stored, in whole or in part, in auxiliary memory 514 and loaded into memory 504 for processing as needed.

[0204] Device 500 may also include one or more output devices, such as display 518. In one example, display 518 may be a touch-sensitive display combining a display with a touch-sensitive element capable of sensing touch input. Display 518 may be coupled to processor 502 via bus 512. In addition to display 518, other output devices may be provided to allow a user to program or otherwise use device 500. When the output device is a display or includes a display, the display may be a liquid crystal display (LCD), a cathode ray tube (CRT) display, a plasma display, or a light-emitting diode (LED) display (e.g., an organic LED (OLED) display).

[0205] The device 500 may also include an image sensing device 520 or communicate with an image sensing device 520. The image sensing device 520 may be a camera, or any other existing or future image sensing device 520 capable of sensing images (e.g., images of a user operating the device 500). The image sensing device 520 may be positioned facing the user operating the device 500. In one example, the position and optical axis of the image sensing device 520 may be configured such that the field of view includes an area immediately adjacent to the display 518, from which the display 518 can be seen.

[0206] The device 500 may also include or communicate with a sound sensing device 522. The sound sensing device 522 may be a microphone, or any other existing or future sound sensing device capable of sensing sounds near the device 500. The sound sensing device 522 may be placed towards the user operating the device 500. The sound sensing device 522 may be used to receive voice or other sounds emitted by the user while operating the device 500.

[0207] Although Figure 5 The processor 502 and memory 504 in device 500 are described as being integrated into a single device, but other configurations can be used. The operation of processor 502 can be distributed across multiple machines (each machine including one or more processors), which can be directly coupled or coupled via a local area network or other network. Memory 504 can be distributed across multiple machines, such as network-based memory or memory across multiple machines performing the operations of device 500. Although bus 512 of device 500 is shown herein as a single bus, there can be multiple buses 512. Furthermore, auxiliary memory 514 can be directly coupled to other components in device 500 or accessed via a network, and can include a single integrated unit (e.g., a memory card) or multiple units (e.g., multiple memory cards). Therefore, device 500 can have a wide variety of configurations.

[0208] Position-dependent prediction combination (PDPC)

[0209] Recent developments in video decoding have led to the emergence of more sophisticated prediction techniques and solutions.

[0210] One such technique is position-dependent prediction combination (PDPC). PDPC is a scheme designed to address certain problems and improve intra-frame prediction. In PDPC, an image or video decoder determines the value of a predicted pixel based on a filtered reference pixel, an unfiltered reference pixel, and the predicted pixel's position within the current block. PDPC may be associated with decoding efficiency gains. For example, fewer bits can be used to encode the same amount of video data.

[0211] Video decoding schemes such as H.264 / AVC and HEVC are designed based on the successful principles of block-based hybrid video decoding. This principle involves first segmenting the image into blocks, and then predicting each block using intra-frame or inter-frame prediction.

[0212] Several video decoding standards after H.261 belong to the "lossy hybrid video codec" group (combining spatial and temporal prediction in the pixel domain with 2D transform decoding in the transform domain for applying quantization). Each image in a video sequence is typically segmented into a set of non-overlapping blocks, and decoding is usually performed at the block level. In other words, on the encoder side, the video is typically processed (i.e., encoded) at the block (image block) level, for example, by generating prediction blocks through spatial (intra-frame) prediction and temporal (inter-frame) prediction; the prediction blocks are subtracted from the current block (the currently processed block / the block to be processed) to obtain residual blocks; the residual blocks are transformed and quantized in the transform domain to reduce the amount of data to be sent (compressed), while on the decoder side, the encoded or compressed block portions are processed inversely relative to the encoder to reconstruct the representation of the current block. Furthermore, the encoder and decoder processing steps are identical, such that the encoder and decoder generate the same prediction blocks (e.g., intra-frame and inter-frame prediction blocks) and / or reconstructed blocks for processing (i.e., decoding) of subsequent blocks.

[0213] As used herein, the term "block" can refer to a portion of an image or frame. For ease of description, this document refers to High-Efficiency Video Coding (HEVC) or Versatile Video Coding (VVC) reference software developed by the Joint Collaboration Team on Video Coding (JCT-VC) of the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Motion Picture Experts Group (MPEG). Those skilled in the art will understand that embodiments of the invention are not limited to HEVC or VVC. A block can refer to a CU, PU, ​​and TU. In HEVC, a CTU can be divided into multiple CUs using a quadtree structure represented as a decoding tree. At the CU level, it is determined whether to use inter-frame (temporal) prediction or intra-frame (spatial) prediction to decode an image region. Each CU can be further divided into one, two, or four PUs depending on the PU partitioning type. The same prediction process is performed within a PU, and relevant information is sent to the decoder on a PU-by-PU basis. After obtaining residual blocks by performing a prediction process based on the PU partitioning type, the CU can be segmented into transform units (TUs) according to other quadtree structures similar to those used for the CU. In recent developments in video compression technology, quadtree and binary-tree (QTBT) segmentation is used to segment the decoded blocks. In a QTBT block structure, a CU can be square or rectangular. For example, the coding tree unit (CTU) is first segmented using a quadtree structure. The leaf nodes of the quadtree are further segmented using a binary tree structure. The leaf nodes of the binary tree are called coding units (CUs), and such segments are used for prediction and transform processing without any further segmentation. This means that in the QTBT decoded block structure, the block sizes of CUs, PUs, and TUs are the same. Meanwhile, it has also been proposed to use multiple segmentations, such as ternary tree segmentation, with the QTBT block structure.

[0214] ITU-T VCEG (Q6 / 16) and ISO / IEC MPEG (JTC 1 / SC 29 / WG 11) are studying the potential need for standardization of future video decoding technologies, where the compression capabilities will significantly exceed those of the current HEVC standard (including current and recent extensions for screen content decoding and high dynamic range decoding). These two expert groups are jointly conducting this exploration, known as the Joint Video Exploration Team (JVET), to evaluate compression technology designs proposed by their experts in this area.

[0215] In one example, directional intra-frame prediction can employ an intra-frame prediction mode, representing different prediction angles from the top diagonal to the bottom diagonal. To define the prediction angle, an offset value p is defined on a 32-pixel grid. ang . Figure 6 p is shown in the vertical prediction mode ang The relationship between the prediction mode and the corresponding intra-prediction mode. The scheme for horizontal prediction mode is to flip the image to the vertical direction and then assign a corresponding value to p. ang As mentioned above, all angle prediction modes can be used for all appropriate intra-frame prediction block sizes. All angle prediction modes use the same 32-pixel grid to define the prediction angle. Figure 6 p shown ang The distribution of values ​​across a 32-pixel grid indicates that the resolution of the predicted angles increases around the vertical direction, while the resolution of the predicted angles in the diagonal direction is lower. The same applies to the horizontal direction, where the resolution of the predicted angles increases around the horizontal direction, while the resolution of the predicted angles in the diagonal direction is lower. This design is based on the observation that in much video content, approximately horizontal and vertical structures play a significant role compared to diagonal structures.

[0216] In one example, while selecting the pixels to use for prediction in the horizontal and vertical prediction directions is straightforward, angular prediction requires more complex operations. For modes 11 through 25, when based on the set of predicted pixels p in the angular direction... ref When predicting the current block Bc (also known as the primary reference edge), p ref Both the vertical and horizontal portions of the image are involved. This is because determining each pixel's position in the image... ref The position on any branch requires certain computational operations; therefore, a unified 1D prediction reference is designed for HEVC intra-frame prediction. This scheme is as follows: Figure 7 As shown. Before performing the actual prediction operation, the reference pixel set p is... ref Mapping to a 1-dimensional vector p 1,refThe projection for such mapping is determined by the direction indicated by the intra-prediction angle of the corresponding intra-frame prediction mode. Only the part of the reference pixel points in p ref that is predicted to be used is mapped to p 1,ref . Figure 8 And Figure 9 respectively show the actual mapping of the reference pixel points to p 1,ref under each angle prediction mode in the horizontal and vertical angle prediction directions. The set of reference pixel points p 1,ref is constructed once for the predicted pixel block. Then the prediction is derived based on two adjacent reference pixel points in this set, as detailed below. From Figure 8 and Figure 9 it can be seen that in all intra-frame prediction modes, the 1D set of reference pixel points is not fully filled. Only the positions within the projection range in the corresponding intra-frame prediction direction are included in the set.

[0217] The prediction in the horizontal and vertical prediction modes is performed in the same way, except that the x and y coordinates of the block are swapped. The prediction based on p 1,ref is performed with a precision of 1 / 32 pixel precision. According to the value of the angle parameter p ang , the pixel offset i 1,ref of p idx and the weighting factor i fact of the pixel at the position (x, y) are determined. The derivation method in the vertical mode is provided here. The derivation method in the horizontal mode can be deduced by analogy, that is, swapping x and y.

[0218]

[0219] [[ID=--35]]If i fact is not equal to 0, that is, the position for prediction is exactly not the full pixel position in p 1,ref , then the linear weighting between two adjacent pixel positions in p[[ID=--40]] 1,ref is performed as follows: <00006<>05>

[0221] where 0 ≤ x, y < Nc. It should be noted that the values of i idx and i fact only depend on y, so only need to be calculated once per row (in the vertical prediction mode).

[0222] The Versatile Test Model (VTM)-1.0 uses 35 intra-frame modes, while the Benchmark Set (BMS) uses 67 intra-frame modes. Intra-frame prediction is used in many video coding frameworks to improve the compression efficiency when only a certain frame is involved.

[0223] Figure 10A An example of the 67 intra-prediction modes proposed in VVC is shown. Several of the 67 intra-prediction modes include: planar mode (index 0), DC mode (index 1), and angle modes (indexes 2 to 66). Figure 10A The lower left angle pattern refers to index 2, and these indices are numbered sequentially up to index 66, which means... Figure 10A The top right angle mode in the image.

[0224] like Figure 10A As shown, the latest version of JEM has several modes corresponding to skewed intra-prediction directions. In any of these modes, to predict pixels within a block, interpolation is performed on the set of adjacent reference pixels if the corresponding position on the block edge is a fractional position. HEVC and VVC perform linear interpolation between two adjacent reference pixels. JEM uses a more complex 4-tap interpolation filter. The filter coefficients are selected as Gaussian or Cubic filter coefficients based on the width or height value. The determination of whether to use width or height is consistent with the selection of the primary reference edge: when the intra-prediction mode is greater than or equal to the diagonal mode, the top edge of the reference pixel is selected as the primary reference edge, and the width value is chosen to determine the interpolation filter used. Otherwise, the primary edge reference is selected from the left side of the block, and the filter selection process is controlled by the height. Specifically, if the selected edge length is less than or equal to 8 pixels, a cubic interpolation 4-tap filter is used. Otherwise, a 4-tap Gaussian filter is used.

[0225] The specific filter coefficients used in JEM are shown in Table 1. Based on the sub-pixel offset and filter type, the predicted pixel points are calculated by convolving with the coefficients selected from Table 1, as shown below:

[0226]

[0227] In this equation, ">>" indicates a bitwise right shift operation.

[0228] If a cubic filter is selected, the predicted pixels are further clipped to a range of allowed values. This range is defined in the sequence parameter set (SPS) or derived from the bit depth of the selected component.

[0229] SPS can specify the features and activation tools used in the decoded video sequence. Unlike the video parameter set (VPS), which applies to the entire bitstream, SPS information applies only to the layers represented by layer identifiers. For example, features specified in the SPS include color format, bit depth, and pixel resolution of the decoded image.

[0230] Table 1: Intra-prediction interpolation filters used in JEM

[0231]

[0232]

[0233] Table 2 shows another interpolation filter bank with 6-bit precision.

[0234] Table 2: Interpolation filter bank with 6-bit precision

[0235]

[0236]

[0237] Based on the sub-pixel offset and filter type, intra-frame predicted pixels are calculated by convolving with coefficients selected from Table 2, as shown below:

[0238]

[0239] In this equation, ">>" indicates a bitwise right shift operation.

[0240] Table 3 shows another interpolation filter bank with 6-bit precision.

[0241] Table 3: Interpolation filter bank with 6-bit precision

[0242]

[0243]

[0244] Figure 11This diagram illustrates various intra-prediction modes used in the HEVC UIP scheme. For a luma block, the intra-prediction modes can include up to 36 modes, which can include 3 non-directional modes and 33 directional modes. Non-directional modes can include planar prediction modes, mean (DC) prediction modes, and luma-based chromaticity (LM) prediction modes. Planar prediction modes perform prediction by assuming the block amplitude surface has horizontal and vertical slopes derived from the block boundaries. DC prediction modes perform prediction by assuming the flat block surface has values ​​consistent with the mean of the block boundaries. LM prediction modes perform prediction by assuming the block's chromaticity value is consistent with the block's luma value. Directional modes can perform prediction based on adjacent blocks, such as... Figure 11 As shown.

[0245] H.264 / AVC and HEVC specify that reference pixels can be low-pass filtered before being used in intra-prediction. Whether to use reference pixel filtering depends on the intra-prediction mode and block size. This mechanism is called Mode Dependent Intra Smoothing (MDIS). Several methods related to MDIS also exist. For example, Adaptive Reference Sample Smoothing (ARSS) can explicitly (i.e., include a flag in the bitstream) or implicitly (i.e., avoid including the flag in the bitstream, for example, through data hiding to reduce indication overhead) indicate whether to filter the predicted pixels. In this case, the encoder can determine whether to perform smoothing by testing the rate-distortion (RD) cost of all potential intra-prediction modes.

[0246] Figure 10A An example of the 67 intra-prediction modes proposed in VVC is shown. Several of the 67 intra-prediction modes include: planar mode (index 0), DC mode (index 1), and angle modes (indexes 2 to 66). Figure 10A The lower left angle pattern refers to index 2, and these indices are numbered sequentially up to index 66, which means... Figure 10A The top right angle mode in the image.

[0247] like Figure 10B and Figure 10CAs shown, VVC, starting from version 2, has several modes corresponding to the tilted intra-prediction direction, including a wide-angle mode (shown by dashed lines). In any of these modes, to predict pixels within a block, interpolation is performed on the set of adjacent reference pixels if the corresponding position on the block edge is a fractional position. HEVC and VVC perform linear interpolation between two adjacent reference pixels. JEM uses a more complex 4-tap interpolation filter. The filter coefficients are selected as Gaussian or Cubic filter coefficients based on the width or height value. The determination of whether to use width or height is consistent with the selection of the primary reference edge: when the intra-prediction mode is greater than or equal to the diagonal mode, the top edge of the reference pixel is selected as the primary reference edge, and the width value is chosen to determine the interpolation filter used. Otherwise, the primary edge reference is selected from the left side of the block, and the filter selection process is controlled by the height. Specifically, if the selected edge length is less than or equal to 8 pixels, a cubic interpolation 4-tap filter is used. Otherwise, a 4-tap Gaussian filter is used.

[0248] VVC uses a partitioning mechanism based on quadtrees and binary trees, called QTBT. For example... Figure 12 As shown, QTBT segmentation can produce not only square blocks but also rectangular blocks. However, compared to the traditional quadtree-based segmentation used in the HEVC / H.265 standard, QTBT segmentation incurs some indication overhead on the encoder side and increases computational complexity. Nevertheless, compared to traditional quadtree-based segmentation, QTBT-based segmentation offers better segmentation characteristics, resulting in significantly higher decoding efficiency.

[0249] The leaf nodes of the tree used for segmentation are processed in a Z-shaped scan order, so that the current block corresponding to the current leaf node has the left and upper neighbor blocks that have been reconstructed during encoding or decoding (unless the current block is located at the boundary of the stripe). Figure 12 This also demonstrates that. Figure 12 The leaf nodes of the tree shown on the right side of the image are scanned from left to right, corresponding to... Figure 12 The left side of the diagram shows the spatial Z-shaped scan order of the blocks. Quadtrees or multi-type trees use the same scan method.

[0250] For directional intra-frame prediction, reference pixels are obtained based on pixels from previously reconstructed neighboring blocks. Depending on the block size and intra-frame prediction mode, filters can be applied to these reference pixels before using them to obtain the values ​​of the predicted pixels.

[0251] In the case of boundary smoothing and PDPC, the first few columns or rows of the predicted block are combined with other predicted signals generated based on neighboring pixels.

[0252] Depending on the intra-frame prediction mode, the simplified PDPC can be implemented in different ways:

[0253] For planar, DC, HOR / VER (horizontal / vertical) intra-frame prediction modes (in... Figure 10B and Figure 10C If the values ​​are 0, 1, 18, and 50 respectively, perform the following steps:

[0254] Predicted pixel located at (x, y) The calculation is as follows:

[0255]

[0256] Among them, R x,–1 and R –1,y R represents the reference pixel located above and to the left of the current pixel (x, y). –1,–1 This represents the reference pixel located at the top-left corner of the current block. P(x,y) represents the value of the predicted pixel when using planar, DC, or HOR / VER intra-prediction modes, as shown above. The function clip1Cmp is set as follows:

[0257] - If the parameter cIdx, which represents the color component of the current block, is equal to 0, then set clip1Cmp to Clip1. Y ;

[0258] - Otherwise, set clip1Cmp to Clip1 C .

[0259] Clip1 Y (x) = Clip3(0, (1< <BitDepth Y )–1,x)

[0260] Clip1 C (x) = Clip3(0, (1< <BitDepth C )–1,x)

[0261]

[0262] BitDepth Y This indicates the bit depth of the brightness pixel.

[0263] BitDepth C This represents the bit depth of a chroma pixel.

[0264] BitDepth Y and BitDepth C It can be indicated in the sequence parameter set (SPS) of the bitstream.

[0265] Clip1 Y (x) and Clip1 C (x) can also have other definitions. Specifically, as described by F. Galpin, P. Bordes, and F. LeLannec in JVET-C0040 “Adaptive Clipping in JEM2.0”, Clip1Cmp(x) = Clip3(min C ,max C ,x),

[0266] Where, min C This indicates the lower bound of the correction used for component ID C in the current stripe.

[0267] max C This indicates the upper limit of the correction used for component ID C in the current stripe.

[0268] C represents the color component (e.g., Y represents lightness, and Cb and Cr represent chromaticity).

[0269] "x >> y" means that the two's complement integer representation of x is arithmetically shifted y bits to the right. This function is defined only when y is a non-negative integer value. The result of the right shift is that the most significant bit (MSB) is shifted in equal to the MSB of x before the shift operation.

[0270] The DC mode weights are calculated as follows:

[0271] wT=32>>((y<<1)>>shift), wL=32>>((x<<1)>>shift),

[0272] wTL=(wL>>4)+(wT>>4),

[0273] Where shift = (log2(width) + log2(height) + 2) >> 2.

[0274] For planar mode, wTL = 0; for horizontal mode, wTL = wT; and for vertical mode, wTL = wL. For example... Figure 13 The DC mode PDPC weights (wL, wT, wTL) for positions (0, 0) and (1, 0) within a 4×4 block are shown. Figure 13 It is evident that the correction operation defined in equation (1) must be performed. However, a problem may exist in the latest PDPC implementation: the result of the correction operation below may exceed the result defined by BitDepth. Y Or BitDepthC The following example illustrates the defined range:

[0275] Assume R -1,y =0, R x,-1 =0, R -1,-1 =100, P(x,y)=0, from equation (1) we can conclude that for the position (0,0) within the 4×4 prediction block,

[0276] Where wTL = 4, as Figure 13 As shown.

[0277] As seen in the example above, arithmetic bit shifting is used to shift the negative value "-4×100+32=-368" to the right. Depending on the implementation, shifting a negative value to the right arithmetically may produce different outputs (e.g., in the case of C / C++ programming languages). Therefore, it cannot be guaranteed that the output of Clip1Cmp() will always be 0, because the result of shifting a negative value to the right can have a negative sign and a non-zero value in specific implementations.

[0278] While the correction-related issues in PDPC are described for planar intra-prediction, similar issues exist for PDPC using DC intra-prediction.

[0279] For diagonal pattern (in) Figure 10B and Figure 10C (represented as 2 and 66) and adjacent patterns ( Figure 10B or Figure 10C (For directional patterns of not less than 58 or not greater than 10), the following processing is performed according to the same formula (1).

[0280] Figure 14 A shows the reference pixel R of the PDPC extending diagonally to the upper right. x,–1 R –1,y and R –1,–1 Definition: The predicted pixel pred(x',y') is located at (x',y') within the prediction block. Reference pixel R. x,–1 The coordinates x are given by the following equation: x = x' + y' + 1. Similarly, the reference pixel R... –1,y The coordinates y are given by the following equation: y = x' + y' + 1.

[0281] The PDPC weights for the top right diagonal pattern are:

[0282] wT=16>>((y'<<1)>>shift), wL=16>>((x'<<1)>>shift), wTL=0.

[0283] Similarly, Figure 14 B shows the reference pixel R of the PDPC extending to the lower left diagonal pattern. x,–1 R –1,y and R –1,–1 Definition of reference pixel R. x,–1 The coordinates x are given by the following equation: x = x' + y' + 1, with reference pixel R. –1,y The coordinates y are: y = x' + y' + 1. The PDPC weights for the upper right diagonal pattern are: wT = 16 >> ((y' << 1) >> shift), wL = 16 >> ((x' << 1) >> shift), wTL = 0. Figure 14 Figure 14C illustrates the case of the upper right diagonal pattern. The PDPC weights for the upper right diagonal pattern are: wT = 32 >> ((y' << 1) >> shift), wL = 0, wTL = 0. Similarly, Figure 14D illustrates the case of the lower left diagonal pattern. The PDPC weights for the lower left diagonal pattern are: wL = 32 >> ((x' << 1) >> shift), wT = 0, wTL = 0. The reference pixel coordinates for the latter two cases are calculated using the table already used for intra-frame prediction in the angled patterns. If fractional reference pixel coordinates are calculated, linear interpolation is performed on the reference pixels.

[0284] The simplified PDPC can be implemented according to the VVC specification. Furthermore, it uses the following representation:

[0285] Indicates the value of the opposite angle.

[0286] Round(x)=Sign(x)×Floor(Abs(x)+0.5),

[0287]

[0288] Floor(x) represents the largest integer less than or equal to x.

[0289] Log2(x) represents the base-2 logarithm of x.

[0290] intraPredAngle indicates the angle parameters detailed in Table 4.

[0291] The expression A = C? B:D is a ternary assignment operation. If condition C is true, then A is set to B; otherwise, if condition C is false, then A is set to D.

[0292] INTRA_PLANAR indicates the intra-planar prediction mode.

[0293] INTRA_DC indicates DC intra-frame prediction mode.

[0294] INTRA_ANGULARXX represents one of the directional intra-prediction modes, where XX represents the mode number and corresponding direction, such as... Figure 10B or Figure 10C As shown.

[0295] If a term is not described here, it should be understood that the definition of the term can be found in the VVC specification or the HEVC / H.265 standard specification.

[0296] Based on the above description, the steps of the simplified PDPC can be defined as follows:

[0297] The inputs to this process include:

[0298] –Intra-frame prediction mode predModeIntra;

[0299] – The variable nTbW represents the transform block width;

[0300] – The variable nTbH represents the height of the transform block;

[0301] – The variable refW represents the width of the reference pixel;

[0302] – The variable refH represents the height of the reference pixel;

[0303] – Predict pixel points predSamples[x][y], where x = 0..nTbW–1, y = 0..nTbH–1;

[0304] – Adjacent pixels p[x][y], where x = –1, y = –1..refH –1; x = 0..refW –1, y = –1;

[0305] – The variable cIdx represents the color component of the current block.

[0306] The output of this process is the modified predicted pixel points predSamples[x][y], where x = 0..nTbW–1 and y = 0..nTbH–1.

[0307] Based on the value of cIdx, the function clip1Cmp is set as follows:

[0308] – If cIdx equals 0, then set clip1Cmp to Clip1. Y ;

[0309] Otherwise, set clip1Cmp to Clip1 C .

[0310] Set the variable nScale to ((Log2(nTbW)+Log2(nTbH)–2)>>2).

[0311] The derivation of the predicted pixel arrays mainRef[x] and sideRef[y] (where x = 0..nTbW–1, y = 0..nTbH–1) is as follows:

[0312] mainRef[x] = p[x][–1]

[0313] sideRef[y] = p[–1][y]

[0314] The derivation of variables refL[x][y], refT[x][y], wT[y], wL[x], and wTL[x][y] (where x = 0..nTbW–1, y = 0..nTbH–1) is as follows:

[0315] – If predModeIntra equals INTRA_PLANAR, INTRA_DC, INTRA_ANGULAR18, or INTRA_ANGULAR50, then

[0316] refL[x][y]=p[–1][y]

[0317] refT[x][y]=p[x][–1]

[0318] wT[y]=32>>((y<<1)>>nScale)

[0319] wL[x]=32>>((x<<1)>>nScale)

[0320] wTL[x][y]=(predModeIntra==INTRA_DC)? ((wL[x]>>4)+(wT[y]>>4)):0

[0321] Otherwise, if predModeIntra equals INTRA_ANGULAR2 or INTRA_ANGULAR66, then

[0322] refL[x][y]=p[–1][x+y+1]

[0323] refT[x][y]=p[x+y+1][–1]

[0324] wT[y]=(32>>1)>>((y<<1)>>nScale)

[0325] wL[x]=(32>>1)>>((x<<1)>>nScale)

[0326] wTL[x][y]=0

[0327] Otherwise, if predModeIntra is less than or equal to INTRA_ANGULAR10, the following steps are executed sequentially:

[0328] 1. The variables dXPos[y], dXFrac[y], dXInt[y], and dX[x][y] are derived using invAngle as follows:

[0329] dXPos[y]=((y+1)×invAngle+2)>>2

[0330] dXFrac[y]=dXPos[y]&63

[0331] dXInt[y]=dXPos[y]>>6

[0332] dX[x][y]=x+dXInt[y]

[0333] 2. The derivation of variables refL[x][y], refT[x][y], wT[y], wL[x], and wTL[x][y] is as follows:

[0334] refL[x][y]=0

[0335] refT[x][y]=(dX[x][y] <refW–1)?((64–

[0336] dXFrac[y])×mainRef[dX[x][y]]+dXFrac[y]×mainRef[dX[x][y]+1]+32)>>6:0

[0337] (Equation 1)

[0338] wT[y]=(dX[x][y]<refW–1)?32> >((y<<1)>>nScale):0

[0339] wL[x]=0

[0340] wTL[x][y]=0

[0341] Otherwise, if predModeIntra is greater than or equal to INTRA_ANGULAR58 (see...) Figure 10B If (or as shown in Figure 10C), then perform the following steps in sequence:

[0342] 1. The variables dYPos[x], dYFrac[x], dYInt[x], and dY[x][y] are derived using invAngle (as detailed below based on intraPredMode):

[0343] dYPos[x]=((x+1)×invAngle+2)>>2

[0344] dYFrac[x] = dYPos[x] & 63

[0345] dYInt[x] = dYPos[x] >> 6

[0346] dY[x][y] = y + dYInt[x]

[0347] 2. The derivation of variables refL[x][y], refT[x][y], wT[y], wL[x], and wTL[x][y] is as follows:

[0348] refL[x][y]=(dY[x][y]<refH–1)?((64– dYFrac[x])×sideRef[dY[x][y]]+dYFrac[x]×sideRef[dY[x][y]+1]+32)> >6:0

[0349] (Equation 2)

[0350] refT[x][y]=0

[0351] wT[y]=0

[0352] wL[x]=(dY[x][y]<refH–1)?32> >((x<<1)>>nScale):0

[0353] wTL[x][y]=0

[0354] Otherwise, set refL[x][y], refT[x][y], wT[y], wL[x], and wTL[x][y] to 0.

[0355] The modified values ​​of the predicted pixels predSamples[x][y] (where x = 0..nTbW–1, y = 0..nTbH–1) are derived as follows:

[0356] predSamples[x][y]=clip1Cmp((refL[x][y]×wL[x]+refT[x][y]×wT[y]–p[–1][– 1]×wTL[x][y]+(64–wL[x]–wT[y]+wTL[x][y])×predSamples[x][y]+32)>>6)

[0357] In the assignment of Equation 1 above, the simplified PDPC can use nearest neighbor interpolation instead of linear interpolation:

[0358] refT[x][y]=(dX[x][y] <refW–1)?mainRef[dX[x][y]]:0

[0359] Similarly, in the assignment of Equation 2 above, the simplified PDPC can also use nearest neighbor interpolation:

[0360] refL[x][y]=(dY[x][y] <refH–1)?sideRef[dY[x][y]]:0

[0361] Therefore, the above method uses the following as input data on both the encoder and decoder sides:

[0362] Directional intra-prediction mode (also referred to as predModeIntra, such as...) Figure 10B and Figure 10C (as shown);

[0363] The block size parameter nTbS is set to (log2(nTbW)+Log2(nTbH))>>1, where nTbW and nTbH represent the width and height of the prediction block, respectively, and ">>" indicates a right shift operation.

[0364] The VVC specification can be modified to use the above method. The modification may include replacing "adjacent pixel p[x][y]" with "reference pixel p[x][y]" in the section describing the simplified PDPC.

[0365] The angle parameter `intraPredAngle` represents the sub-pixel offset between two adjacent rows of predicted pixels in the fixed-point representation, and its fractional part is 5 bits long. This parameter (`intraPredAngle`) can be derived from the intra-prediction mode (`predModeIntra`). An exemplary derivation of `intraPredAngle` based on `predModeIntra` can be defined using a lookup table (LUT), as shown in Table 4.

[0366] Table 4: Exemplary LUTs for deriving intraPredAngle from predModeIntra

[0367]

[0368]

[0369] Based on the current HEVC and VVC draft specifications, a planar intra-frame prediction method is adopted. This part of VVC draft 3 is included below for reference:

[0370] 8.2.4.2.5 Detailed description of INTRA_PLANAR intra-prediction mode

[0371] The inputs to this process include:

[0372] – The variable nTbW represents the transform block width;

[0373] – The variable nTbH represents the height of the transform block;

[0374] – Adjacent pixels p[x][y], where x = –1, y = –1..nTbH; x = 0..nTbW, y = –1.

[0375] The output of this process is the predicted pixel points predSamples[x][y], where x = 0..nTbW–1 and y = 0..nTbH–1.

[0376] The derivation of the values ​​of the predicted pixels predSamples[x][y] (where x = 0..nTbW–1, y = 0..nTbH–1) is as follows:

[0377] predV[x][y]=((nTbH–1–y)×p[x][–1]+(y+1)×p[–1][nTbH])< <Log2(nTbW)(8-82)

[0378] predH[x][y]=((nTbW–1–x)×p[–1][y]+(x+1)×p[nTbW][–1])< <Log2(nTbH)(8-83)

[0379] predSamples[x][y]=(predV[x][y]+predH[x][y]+nTbW×nTbH)>>(Log2(nTbW)+Log2(nTbH)+ 1) (8-84)

[0380] The present invention solves the above-mentioned related problems (which may generate incorrect predicted pixel point values) caused by performing a shift operation on negative values during DC intra prediction using PDPC.

[0381] The provided technical solution relates to an alternative PDPC method that does not have the problem of equation (1).

[0382] Specifically, the method includes the following steps, as Figure 15 shown in the flowchart. The method for performing intra prediction on a block in an image is executed for each pixel among multiple pixel points of the block.

[0383] In step S100, intra prediction is performed using the DC intra prediction mode, and a predicted pixel point value is obtained based on one or more reference pixel point values.

[0384] In addition, in step S110, the predicted pixel point value is multiplied by a pixel point weighting factor to obtain a weighted predicted pixel point value. Specifically, the pixel point weighting factor is calculated as ((2 << p) – wL – wT), where p is the precision of the pixel point weighting factor, wL is the horizontal weighting factor, and wT is the vertical weighting factor.

[0385] In step S120, the weighted predicted pixel point value is added to other values to obtain an unnormalized predicted pixel point value.

[0386] In step S130, the unnormalized predicted pixel point value is normalized by performing an arithmetic right shift on the integer representation of the unnormalized predicted pixel point value.

[0387] Therefore, through the shown method, DC intra prediction can be performed within the framework of PDPC to determine a normalized predicted pixel point value, and at the same time, by executing a determination method that does not necessarily require a correction process, incorrect predicted values can be avoided.

[0388] Note that further processing can be performed before and after the shown method steps.

[0389] For example, the normalized predicted pixel point value can be added to a residual value to obtain a reconstructed pixel point value.

[0390] In one embodiment, the other values may include a rounding offset. The arithmetic right shift operation corresponds to division by a power of 2 and generally gives a rounded value. To ensure correct rounding results, a rounding offset can be added before performing the right shift operation. After performing the right shift operation, the rounding offset value corresponds to half of the integer value. Therefore, adding the rounding offset ensures the correct rounding result value. For example, in the case of shifting 6 bits to the right (corresponding to division by 2 6 = 64), the rounding offset is 32.

[0391] In another embodiment, a method is provided for intra-frame prediction of a first block and a second block in an image. For each pixel of a plurality of pixels in the first block and each pixel of a plurality of pixels in the second block, the following steps are performed: Figure 15 The difference between the steps shown and those described above is that the intra-prediction mode used to obtain the predicted pixel values ​​of the first block is the DC intra-prediction mode, and the intra-prediction mode used to obtain the predicted pixel values ​​of the second block is the planar intra-prediction mode.

[0392] Specifically, for each pixel in the first block, in step S100, intra-frame prediction is performed using the DC intra-prediction mode, and the predicted pixel value is obtained based on one or more reference pixel values. Furthermore, for each pixel in the second block, in step S100, intra-frame prediction is performed using the planar intra-prediction mode, and the predicted pixel value is obtained based on one or more reference pixel values.

[0393] In one embodiment, the method may include a method for encoding or decoding an image. Specifically, predicted pixel values ​​can be obtained by performing the steps in any of the above methods. Furthermore, the reconstructed pixel values ​​can be obtained by adding the obtained normalized predicted pixel values ​​to the residual values.

[0394] The embodiments of this invention can achieve the following technical advantages: simplified hardware design and reduced number of condition checks. The steps performed to obtain the predicted pixel values ​​in planar and DC intra-prediction modes differ from those in directional intra-prediction. An exemplary hardware implementation of intra-prediction includes at least two modules:

[0395] – Directional intra-frame prediction module,

[0396] – Non-directional intra-frame prediction module.

[0397] For this reason, it is first necessary to coordinate PDPC filtering within these intra-prediction mode groups. Considering the fact that PDPC complexity in DC mode exceeds that in planar mode, this invention simplifies PDPC processing in DC mode.

[0398] According to embodiments of the present invention, these two modules implement their own PDPC filtering processing. Existing solutions require additional checks on whether the intra-prediction mode used by the non-directional intra-prediction module for PDPC filtering is DC mode or planar mode, or require different PDPC filtering modules for DC and planar intra-prediction modes (increasing hardware complexity and power consumption). Therefore, coordinating PDPC in DC and planar modes allows for the same PDPC processing (corresponding hardware modules) in both DC and planar intra-prediction modes.

[0399] The embodiments of the present invention offer another technical advantage when used in intra-prediction designs for simplified encoders. DC intra-prediction modes are typically the simplest intra-prediction modes to compute; therefore, in the worst case, the complexity of PDPC filtering can exceed the complexity of the DC mode itself. By reducing the number of condition checks and / or operations in the DC mode, the present invention can reduce the overall complexity of intra-prediction, especially in simplified coding scenarios implemented in so-called "lazy" encoders that operate using a constrained set of intra-prediction modes.

[0400] The provided method can be implemented by defining the module in the non-directional intra-prediction method to perform the same PDPC processing in both intra-prediction modes. In other words, an embodiment of the present invention can be achieved by introducing a transformation of the non-directional intra-prediction module, that is, simply making the PDPC processing methods of the DC and planar intra-prediction modes the same.

[0401] The weighting factors wL and wT can both be powers of 2, allowing them to be multiplied using shift operators. This facilitates hardware implementation and improves processing efficiency. Therefore, coordinating PDPC processing when using DC intra-prediction mode and PDPC processing when using planar intra-prediction mode simplifies PDPC intra-prediction, and so on. The above method can be performed by the predictor device, encoder device, or decoder device shown in Figure 16, etc.

[0402] The predictor, encoder, or decoder device 1000 includes an acquirer 1010, a multiplier 1020, an adder 1030, and a normalizer 1040. The acquirer 1010 is used to perform intra-frame prediction using the DC intra-prediction mode, acquiring predicted pixel values ​​based on one or more reference pixel values. The multiplier 1020 is used to multiply the predicted pixel value by a pixel weighting factor to obtain a weighted predicted pixel value. Specifically, the weighting factor is ((2 << p) - wL - wT), where p is a parameter of the pixel weighting factor (e.g., precision), wL is a horizontal weighting factor, and wT is a vertical weighting factor. The adder 1030 is used to add the weighted predicted pixel value to other values ​​to obtain a non-normalized predicted pixel value. The normalizer 1040 is used to normalize the non-normalized predicted pixel value by performing an arithmetic right shift on the integer representation of the non-normalized predicted pixel value.

[0403] Similarly, in one embodiment, the acquirer 1010 in the predictor, encoder, or decoder device 1000 can be used to: perform intra-frame prediction using an intra-frame prediction mode for each pixel in a first block and each pixel in a second block, and acquire a predicted pixel value based on one or more reference pixel values. The intra-frame prediction mode used to acquire the predicted value of the pixels in the first block can be a DC intra-frame prediction mode, and the intra-frame prediction mode used to acquire the predicted value of the pixels in the second block can be a planar intra-frame prediction mode. The multiplier 1020 is used to multiply the predicted pixel value by a pixel weighting factor to obtain a weighted predicted pixel value. Specifically, the weighting factor is ((2 << p) - wL - wT), where p is a parameter of the pixel weighting factor (e.g., precision), wL is a horizontal weighting factor, and wT is a vertical weighting factor. Adder 1030 is used to add the weighted predicted pixel value to other values ​​to obtain the non-normalized predicted pixel value; normalizer 1040 is used to normalize the non-normalized predicted pixel value by performing an arithmetic right shift on the integer representation of the non-normalized predicted pixel value.

[0404] In one embodiment, the predictor, encoder, or decoder device 1000 may also add the normalized predicted pixel value to the residual value to obtain the reconstructed pixel value.

[0405] In one embodiment of the method, the horizontal weighting factor wL can be applied to the value of the left reference pixel to obtain the left weighted reference pixel value. Furthermore, the vertical weighting factor wT can be applied to the value of the upper reference pixel to obtain the upper weighted reference pixel value.

[0406] Specifically, for example, the left and top reference pixel values ​​can be obtained by multiplying them by the corresponding horizontal and vertical weighting factors wL and wT, respectively.

[0407] In addition, in one embodiment, the other values ​​may include the sum of the upper weighted reference pixel value and the left weighted reference pixel value.

[0408] The functional components in the encoder or decoder device 1000 can be implemented by the processing circuit 1040, which is used to perform any of the methods described above, such as... Figure 16B As shown.

[0409] In one embodiment, the processing circuitry 1040 may include one or more processors 1050 and a non-transitory computer-readable storage medium 1060 connected to the one or more processors 1050. The storage medium includes program code. When the processor executes the program code, the processor performs any of the methods described above.

[0410] In one specific embodiment, the method may include the following steps:

[0411] Normalized predicted pixel located at (x, y) The following can be calculated:

[0412]

[0413] Among them, R x,–1 and R –1,y These represent the reference pixels located above (above) and to the left of the current pixel (x, y), respectively. P(x, y) represents the value of the predicted pixel using the DC intra-frame prediction mode. In the above formula, the weighted predicted pixel is represented as (64 - wL - wT) × P(x, y), and the other values ​​are represented as wL × R. -1,y +wT× R x,-1 +32. Normalized representation is a bitwise right shift operation >> 6. However, the invention is not limited to the specific definition of the other values, i.e., a shift operation of 6 bits.

[0414] It is worth noting that the function clip1Cmp is not used in the embodiment using equation (2) because the value of the predicted pixel point P(x,y) is always within the range of valid values, that is, between the minimum and maximum pixel values. However, the present invention is not limited to not performing correction operations; corrections can still be performed.

[0415] For example, a normalized predicted pixel value can be calculated based on the predicted pixel value, including calculating...

[0416] (wL×R-1,y +wT×R x,-1 +(64-wL-wT)×P(x,y)+32)>>6.

[0417] In this implementation, normalization is achieved using the right shift operator >> 6. This invention is not limited to the specific calculations given above; it can also perform mathematically equivalent calculations.

[0418] Therefore, by performing the above calculation on the predicted pixel value, the problem of erroneous non-zero positive predicted pixel value caused by the correction operation for negative values ​​can be avoided.

[0419] In the formula above, "x >> y" means that the two's complement integer representation of x is arithmetically shifted y bits to the right. This function is defined only when y is a non-negative integer value. The result of the right shift is that the most significant bit (MSB) is shifted in equal to the MSB of x before the shift operation.

[0420] The DC mode weights can be calculated as follows:

[0421] wT=32>>((y<<1)>>shift), wL=32>>((x<<1)>>shift),

[0422] Where shift = (log2(width) + log2(height) + 2) >> 2.

[0423] For example, Figure 17 The DC mode PDPC weights (wL, wT) for positions (0, 0) and (1, 0) within a 4×4 block are shown. In this example, for the predicted pixel at coordinates (0, 0), the weights wL and wT are both 32. Furthermore, in this example, for the predicted pixel at coordinates (1, 0), the weight wL is 8 and the weight wT is 32. It can be noted that... Figure 13 The weights of the predicted pixels at coordinates (0, 0) and (1, 0) are compared, but the top-left reference pixel is not used, so the weight of that pixel (top-left reference pixel) is not specified. However, this invention is not limited to the described DC mode PDPC weight calculation process; the DC mode PDPC weights can also be determined in different ways or using different formulas.

[0424] The provided method can be represented by a portion of the VVC specification:

[0425] Location-related intra-frame prediction pixel filtering process

[0426] The inputs to this process include:

[0427] –Intra-frame prediction mode predModeIntra;

[0428] – The variable nTbW represents the transform block width;

[0429] – The variable nTbH represents the height of the transform block;

[0430] – The variable refW represents the width of the reference pixel;

[0431] – The variable refH represents the height of the reference pixel;

[0432] – Predict pixel points predSamples[x][y], where x = 0..nTbW–1, y = 0..nTbH–1;

[0433] – Adjacent pixels p[x][y], where x = –1, y = –1..refH –1; x = 0..refW –1, y = –1;

[0434] – The variable cIdx represents the color component of the current block.

[0435] The output of this process is the modified predicted pixel points predSamples[x][y], where x = 0..nTbW–1 and y = 0..nTbH–1.

[0436] Based on the value of cIdx, the function clip1Cmp is set as follows:

[0437] – If cIdx equals 0, then set clip1Cmp to Clip1. Y .

[0438] Otherwise, set clip1Cmp to Clip1 C .

[0439] Set the variable nScale to ((Log2(nTbW)+Log2(nTbH)–2)>>2).

[0440] The derivation of the predicted pixel arrays mainRef[x] and sideRef[y] (where x = 0..nTbW–1, y = 0..nTbH–1) is as follows:

[0441] mainRef[x] = p[x][–1]

[0442] sideRef[y] = p[–1][y]

[0443] The derivation of variables refL[x][y], refT[x][y], wT[y], and wL[x] (where x = 0..nTbW–1, y = 0..nTbH–1) is as follows:

[0444] – If predModeIntra equals INTRA_PLANAR, INTRA_DC, INTRA_ANGULAR18, or INTRA_ANGULAR50, then

[0445] refL[x][y]=p[–1][y]

[0446] refT[x][y]=p[x][–1]

[0447] wT[y]=32>>((y<<1)>>nScale)

[0448] wL[x]=32>>((x<<1)>>nScale)

[0449] Otherwise, if predModeIntra equals INTRA_ANGULAR2 or INTRA_ANGULAR66, then

[0450] refL[x][y]=p[–1][x+y+1]

[0451] refT[x][y]=p[x+y+1][–1]

[0452] wT[y]=(32>>1)>>((y<<1)>>nScale)

[0453] wL[x]=(32>>1)>>((x<<1)>>nScale)

[0454] Otherwise, if predModeIntra is less than or equal to INTRA_ANGULAR10, the following steps are executed sequentially:

[0455] 1. The variables dXPos[y], dXFrac[y], dXInt[y], and dX[x][y] are derived using invAngle (detailed in section 8.2.4.2.7 according to intraPredMode):

[0456] dXPos[y]=((y+1)×invAngle+2)>>2

[0457] dXFrac[y]=dXPos[y]&63

[0458] dXInt[y]=dXPos[y]>>6

[0459] dX[x][y]=x+dXInt[y]

[0460] 2. The derivation of variables refL[x][y], refT[x][y], wT[y], and wL[x] is as follows:

[0461] refL[x][y]=0

[0462] refT[x][y]=(dX[x][y] <refW–1)?((64–

[0463] dXFrac[y])×mainRef[dX[x][y]]+dXFrac[y]×mainRef[dX[x][y]+1]+32)>>6:0

[0464] wT[y]=(dX[x][y]<refW–1)?32> >(y<<1)>>nScale):0

[0465] wL[x]=0

[0466] Otherwise, if predModeIntra is greater than or equal to INTRA_ANGULAR58, the following steps are executed sequentially:

[0467] 1. The variables dYPos[x], dYFrac[x], dYInt[x], and dY[x][y] are derived as follows using invAngle (detailed in section 8.2.4.2.7 based on intraPredMode):

[0468] dYPos[x]=((x+1)×invAngle+2)>>2

[0469] dYFrac[x] = dYPos[x] & 63

[0470] dYInt[x] = dYPos[x] >> 6

[0471] dY[x][y] = y + dYInt[x]

[0472] 2. The derivation of variables refL[x][y], refT[x][y], wT[y], and wL[x] is as follows:

[0473] refL[x][y]=(dY[x][y] <refH–1)?((64–

[0474] dYFrac[x])×sideRef[dY[x][y]]+dYFrac[x]×sideRef[dY[x][y]+1]+32)>>6:0

[0475] refT[x][y]=0

[0476] wT[y]=0

[0477] wL[x]=(dY[x][y]<refH–1)?32> >((x<<1)>>nScale):0

[0478] Otherwise, set refL[x][y], refT[x][y], wT[y], and wL[x] to 0. The derivation of the modified predicted pixel values ​​predSamples[x][y] (where x = 0..nTbW–1, y = 0..nTbH–1) is as follows:

[0479] predSamples[x][y]=(refL[x][y]×wL[x]+refT[x][y]×wT[y]+ (64–wL[x]–wT[y])×predSamples[x][y]+32)>>6)

[0480] Here, (64–wL[x]–wT[y]) represents the pixel weighting factor.

[0481] Figure 18 The method described above is illustrated. The dashed line indicates the correction step, which is performed in the latest PDPC, but is not required since the coefficient wTL is not used, so it is not performed in the provided method.

[0482] The input to the method shown is a reference pixel value, represented as ref[] in the figure. Intra-frame prediction is performed on the current pixel using the reference pixel. Specifically, intra-frame prediction can be performed on the current pixel value using the DC intra-frame prediction mode. Furthermore, other values ​​are calculated using the left and top reference pixel values ​​refL[x][y] and refT[x][y], and the corresponding left weights wL[x] and wT[y], respectively. The intra-frame predicted pixel value is multiplied by a weighting factor (64 – wL[x] – wT[y]). Then, the weighted predicted pixel value is added to the other values. Next, since the weighted predicted pixel value added to the other values ​​does not necessarily represent a normalized value, a normalization process is performed by a bitwise right shift operation (represented as ">>" in the figure). The resulting normalized predicted pixel value does not require further correction operations (in the figure, redundant correction operations are represented by dashed lines).

[0483] Although in the described example, the weighting factor is (64–wL[x]–wT[x]), the present invention is not limited thereto. Specifically, the pixel weighting factor may have different precisions. In other words, the pixel weighting factor may be expressed as ((2<<p)–wL–wT), where p is its precision. In addition, although a correction process is not required, it may still be performed.

[0484] In Figure 14 the intra-frame prediction process uses reference pixels to generate predicted pixels. In addition, each predicted pixel is weighted using a pixel weighting factor. The pixel weighting factor may be equal to (64–wL[x]–wT[y]), etc. Other values of each predicted pixel are calculated according to x and y using the same reference pixels, where x and y define the position of the predicted pixel in the prediction block. These other values are added to the corresponding weighted predicted pixels. Then, according to the predetermined precision of the pixel weighting factor, each pixel obtained by this operation is normalized by right shift. For example, if the pixel weighting factor is defined as (64–wL[x]–wT[y]), the precision is 6 bits. Therefore, in this step, shift 6 bits to the right to ensure that the possible minimum and maximum values of the output are the same as the possible minimum and maximum values of the reference pixels. However, the present invention is not limited to 6-bit precision, and any other precision may also be used.

[0485] One beneficial effect of the provided technical solution is that the planar intra-frame prediction mechanism can be reused to calculate other values. Specifically, the planar intra-frame prediction can use the following equations to derive the horizontal and vertical predicted pixel values:

[0486]

[0487]

[0488] It can be seen from the above two equations (8-82) and (8-83) that predV[x][y] uses the reference pixel p[x][–1] in the same column as predV[x][y], and predH[x][y] uses the reference pixel p[–1][y] in the same row as predH[x][y]. In addition, the last step is to perform a left shift operation, which can be skipped because the left shift operation does not affect the reused intermediate calculations.

[0489] In formulas (8-82) and (8-83) above, predV represents the vertical predicted pixel value determined by intra-planar prediction, and predH represents the horizontal predicted pixel value determined by intra-planar prediction. Furthermore, nTbW and nTbH represent the width and height of the current block when performing intra-planar prediction, respectively. However, nTbW, nTbH, and variables x and y are inputs to the intra-planar prediction method and can therefore be adjusted accordingly. Thus, input variable Dx can be used instead of (nTbW–1–x), and input variable D can be used... y Replace (nTbH–1–y). Since the lower left and upper right reference pixels are unused parameters, they can be set to 0.

[0490] Based on the above observations, equations (8-82) and (8-83) can be rewritten according to predetermined inputs:

[0491] V y =D y ×p[x][–1]

[0492] V x =D x ×p[–1][y]

[0493] Therefore, in order to determine other values ​​to be added to the weighted predicted pixel values, the following steps can be performed uniformly:

[0494] – Other values ​​in the horizontal mode (mode 18) can be calculated as V y =D y ×p[x][–1], where D y Set to wT[y];

[0495] – Other values ​​in the vertical mode (mode 50) case can be calculated as V x =Dx×p[–1][y], where Dx is set to wL[y];

[0496] Other values ​​in the DC mode (Mode 1) case can be calculated as V. y +V x In this case, similar to the previous two, D will be... y Set to wT[y], and set D x Set to wL[y].

[0497] By changing the selection of the reference pixel, it can be shown that all intra-prediction modes specified for PDPC can perform uniform processing.

[0498] In another embodiment, the PDPC process can be detailed as follows:

[0499] Location-related intra-frame prediction pixel filtering process

[0500] The inputs to this process include:

[0501] –Intra-frame prediction mode predModeIntra;

[0502] – The variable nTbW represents the transform block width;

[0503] – The variable nTbH represents the height of the transform block;

[0504] – The variable refW represents the width of the reference pixel;

[0505] – The variable refH represents the height of the reference pixel;

[0506] – Predict pixel points predSamples[x][y], where x = 0..nTbW–1, y = 0..nTbH–1;

[0507] – Adjacent pixels p[x][y], where x = –1, y = –1..refH –1; x = 0..refW –1, y = –1.

[0508] The output of this process is the modified predicted pixel points predSamples[x][y], where x = 0..nTbW–1 and y = 0..nTbH–1.

[0509] The derivation of the variable nScale is as follows:

[0510] – If predModeIntra is greater than INTRA_ANGULAR50, then nScale is set to Min(2,Log2(nTbH)–Floor(Log2(3×invAngle–2))+8) via invAngle (as detailed in section 8.4.5.2.12).

[0511] Otherwise, if predModeIntra is less than INTRA_ANGULAR18, not equal to INTRA_PLANAR, and not equal to INTRA_DC, then nScale is set to Min(2, Log2(nTbW) – Floor(Log2(3×invAngle – 2)) + 8) via invAngle (as detailed in section 8.4.5.2.12).

[0512] Otherwise, set nScale to ((Log2(nTbW)+Log2(nTbH)–2)>>2).

[0513] The derivation of the predicted pixel arrays mainRef[x] and sideRef[y] (where x = 0..nTbW–1, y = 0..nTbH–1) is as follows:

[0514]

[0515] The derivation of variables refL[x][y], refT[x][y], wT[y], and wL[x] (where x = 0..nTbW–1, y = 0..nTbH–1) is as follows:

[0516] – If predModeIntra equals INTRA_PLANAR or INTRA_DC, then

[0517] refL[x][y]=p[–1][y] (8-230)

[0518] refT[x][y]=p[x][–1] (8-231)

[0519] wT[y]=32>>((y<<1)>>nScale) (8-232)

[0520] wL[x]=32>>((x<<1)>>nScale) (8-233)

[0521] Otherwise, if predModeIntra equals INTRA_ANGULAR18 or INTRA_ANGULAR50, then refL[x][y] = p[–1][y] – p[–1][–1] + predSamples[x][y] (8-234)

[0522] refT[x][y]=p[x][–1]–p[–1][–1]+predSamples[x][y] (8-235)

[0523] wT[y]=(predModeIntra==INTRA_ANGULAR18)? 32>>((y<<1)>>nScale):0 (8-236)

[0524] wL[x]=(predModeIntra==INTRA_ANGULAR50)? 32>>((x<<1)>>nScale):0 (8-237)

[0525] Otherwise, if predModeIntra is less than INTRA_ANGULAR18 and nScale is greater than or equal to 0, the following steps are executed sequentially:

[0526] 1. The variables dXInt[y] and dX[x][y] are derived using invAngle (detailed in section 8.4.5.2.12 according to intraPredMode) as follows:

[0527]

[0528] 2. The derivation of variables refL[x][y], refT[x][y], wT[y], and wL[x] is as follows:

[0529] refL[x][y]=0 (8-239)

[0530] refT[x][y]=(y<(3< <nScale))?mainRef[dX[x][y]]:0 (8-240)

[0531] wT[y]=32>>((y<<1)>>nScale) (8-241)

[0532] wL[x]=0 (8-242)

[0533] Otherwise, if predModeIntra is greater than INTRA_ANGULAR50 and nScale is greater than or equal to 0, then perform the following steps in sequence:

[0534] 1. The variables dYInt[x] and dY[x][y] are derived using invAngle (detailed in section 8.4.5.2.12 according to intraPredMode) as follows:

[0535]

[0536] 2. The derivation of variables refL[x][y], refT[x][y], wT[y], and wL[x] is as follows:

[0537] refL[x][y]=(x<(3< <nScale))?sideRef[dY[x][y]]:0 (8-244)

[0538] refT[x][y]=0 (8-245)

[0539] wT[y]=0 (8-246)

[0540] wL[x]=32>>((x<<1)>>nScale) (8-247)

[0541] Otherwise, set refL[x][y], refT[x][y], wT[y], and wL[x] to 0.

[0542] The derivation of the modified predicted pixel values ​​predSamples[x][y] (where x = 0..nTbW–1, y = 0..nTbH–1) is as follows:

[0543]

[0544] In the above text, the function Clip1 can be further defined as Clip1Cmp, etc.

[0545] Although embodiments of the present invention are primarily described in relation to video decoding, it should be noted that embodiments of the decoding system 10, encoder 20, and decoder 30 (correspondingly, system 10), as well as other embodiments described herein, can also be used for still image processing or decoding, i.e., processing or decoding a single image in video decoding independent of any previous or consecutive images. Generally, if image processing decoding is limited to a single image 17, only the inter-frame prediction units 244 (encoder) and 344 (decoder) are unavailable. All other functions (also referred to as tools or techniques) of the video encoder 20 and video decoder 30 can also be used for still image processing, such as residual calculation 204 / 304, transform 206, quantization 208, inverse quantization 210 / 310, (inverse) transform 212 / 312, segmentation 262 / 362, intra-frame prediction 254 / 354 and / or loop filtering 220 / 320, entropy decoding 270, and entropy decoding 304.

[0546] For example, embodiments of encoder 20 and decoder 30, as well as the functions described herein (e.g., with reference to encoder 20 and decoder 30), can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the various functions can be stored as one or more instructions or code in a computer-readable medium or transmitted via a communication medium and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium, corresponding to a tangible medium (e.g., a data storage medium), or may include any communication medium that facilitates the transfer of a computer program from one place to another according to a communication protocol, etc. In this way, a computer-readable medium can generally correspond to (1) a non-transitory tangible computer-readable storage medium or (2) a communication medium such as a signal or carrier wave. A data storage medium may be any available medium accessible by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementing the techniques described herein. A computer program product may include a computer-readable medium.

[0547] By way of example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Furthermore, any connection may be appropriately referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but rather refer to non-transient tangible storage media. The disks and optical discs used herein include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), and Blu-ray discs, wherein disks typically reproduce data magnetically, while optical discs reproduce data optically using lasers. Combinations of the above items should also be included within the scope of computer-readable media.

[0548] Instructions can be executed by one or more processors, such as one or more digital signal processors (DSPs), one or more general-purpose microprocessors, one or more application-specific integrated circuits (ASICs), one or more field-programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, the term "processor" as used herein can refer to any of the above-described structures or any other structure suitable for implementing the techniques described herein. Furthermore, in some aspects, the various functions described herein can be provided within dedicated hardware and / or software modules for encoding and decoding, or incorporated into a combinational decoder. Moreover, these techniques can be fully implemented in one or more circuit or logic elements. The processing circuitry mentioned in this invention can include both hardware and software.

[0549] The technology of the present invention can be implemented in various devices or apparatuses, including wireless mobile phones, integrated circuits (ICs), or a group of ICs (such as a chipset). The present invention describes various components, modules, or units to emphasize the functional aspects of the devices for performing the disclosed technology, but they do not necessarily need to be implemented by different hardware units. Instead, as described above, various units can be combined in a decoder hardware unit with appropriate software and / or firmware, or provided by a set of interoperating hardware units including one or more processors as described above.

[0550] Other embodiments are summarized as follows.

[0551] A method for intra prediction of blocks in an image is provided. The method includes: for each pixel point in a plurality of pixel points of the block, performing intra prediction using one of a DC intra prediction mode, a planar intra prediction mode, and an angular intra prediction mode, and obtaining a predicted pixel point value according to one or more reference pixel point values; multiplying the predicted pixel point value by a pixel point weighting factor to obtain a weighted predicted pixel point value; adding the weighted predicted pixel point value to other values to obtain an unnormalized predicted pixel point value; and normalizing the unnormalized predicted pixel point value by performing an arithmetic right shift on the integer representation of the unnormalized predicted pixel point value to obtain a normalized predicted pixel point value.

[0552] In one embodiment, the image is part of a video sequence.

[0553] In one embodiment, the pixel point weighting factor is ((2 << p) – wL – wT), where

[0554] p is the precision of the pixel point weighting factor,

[0555] wL is a horizontal weighting factor,

[0556] wT is a vertical weighting factor.

[0557] In one embodiment, the horizontal weighting factor is wL = (2 << (p – 1)) >> ((x << 1) >> nScale), where x is the abscissa of the pixel point; the vertical weighting factor is wT = (2 << (p – 1)) >> ((y << 1) >> nScale), where y is the ordinate of the pixel point; and nScale is a scaling parameter.

[0558] In one embodiment, the scaling parameter nScale is derived according to the size of the block.

[0559] In one embodiment, the scaling parameter nScale is determined to be ((Log2(nTbW)+Log2(nTbH)–2)>>2), where nTbW is the width of the block and nTbH is the height of the block.

[0560] In one embodiment, the normalized predicted pixel value is calculated based on the predicted pixel value as follows:

[0561]

[0562] in,

[0563] The normalized pixel prediction value,

[0564] P(x,y) is the predicted pixel value.

[0565] Rx,–1 and R–1,y represent the values ​​of the nearest reference pixel located above and to the left of the predicted pixel.

[0566] wL is the level weighting factor.

[0567] wT is the vertical weighting factor.

[0568] In one embodiment, the horizontal weighting factor is wL = (2 << (p – 1)) >> ((x << 1) >> nScale), where x is the horizontal coordinate of the pixel; the vertical weighting factor is wT = (2 << (p – 1)) >> ((y << 1) >> nScale), where y is the vertical coordinate of the pixel; and nScale is the scaling parameter.

[0569] In one embodiment, the scaling parameter nScale is derived based on the size of the block.

[0570] In one embodiment, the scaling parameter nScale is determined to be ((Log2(nTbW)+Log2(nTbH)–2)>>2), where nTbW is the width of the block and nTbH is the height of the block.

[0571] In one embodiment, the plurality of pixels in the block includes each pixel in the block.

[0572] An apparatus for encoding or decoding images is also provided. The apparatus includes processing circuitry for performing any of the methods described above.

[0573] In one embodiment, the processing circuitry includes one or more processors and a non-transitory computer-readable medium connected to the one or more processors. The non-transitory computer-readable medium includes program code; when the one or more processors execute the program code, the device performs the method.

[0574] A non-transitory computer-readable medium comprising program code is also provided. When a computer device executes the program code, the computer device performs any of the methods described above.

[0575] Abbreviations and Terminology Definitions

[0576] JEM Joint Exploration Model (a software codebase for future video decoding exploration)

[0577] JVET Joint Video Experts Team

[0578] LUT Look-Up Table

[0579] PDPC Position-dependent prediction combination

[0580] PPS Picture parameter set

[0581] QT QuadTree

[0582] QTBT QuadTree plus Binary Tree (quadtree combined with binary tree)

[0583] Rate-distortion optimization (RDO)

[0584] ROM (Read-Only Memory)

[0585] SPS Sequence parameter set

[0586] VTM VVC Test Model

[0587] VVC (Versatile Video Coding) is a standardized video decoding project developed by JVET.

[0588] CTU / CTB Coding Tree Unit / Coding Tree Block

[0589] CU / CB Coding Unit / Coding Block

[0590] PU / PB Prediction Unit / Prediction Block

[0591] TU / TB Transform Unit / Transform Block

[0592] HEVC High Efficiency Video Coding.

Claims

1. A method for decoding blocks in an image, characterized in that, The method includes: For the pixels in the block, intra-prediction mode is used to perform intra-prediction on the pixels, and the first predicted pixel value of the pixels is obtained based on one or more reference pixel values. The intra-prediction mode includes DC intra-prediction mode or planar intra-prediction mode. Multiply the first predicted pixel value by the pixel weighting factor to obtain the second predicted pixel value; The pixel weighting factor is ,in, The parameters of the weighting factor for the pixel are... As a level weighting factor, It is a vertical weighting factor. Indicates an arithmetic left shift; The integer representation of the result of adding the second predicted pixel value to other values ​​is arithmetically right-shifted to obtain the third predicted pixel value of the pixel, wherein the other values ​​are the sum of one or more addends, the one or more addends including addends determined according to the one or more reference pixels, the addends determined according to the one or more reference pixels being wL×R_(-1,y)+wT×R_(x,-1), wherein R_(x,-1) and R_(-1,y) represent the values ​​of the nearest reference pixels located above and to the left of the pixel, respectively; The reconstructed pixel value is obtained by adding the third predicted pixel value to the residual of the pixel. The intra-frame prediction mode and the residual of the pixel are both obtained from the bitstream of the image.

2. The method according to claim 1, characterized in that, The pixel weighting factor is .

3. The method according to claim 1, characterized in that, The horizontal weighting factor wL and / or the vertical weighting factor wT are powers of 2.

4. The method according to claim 1, characterized in that, The horizontal weighting factor is wL=(2 (p-1)) ((x 1) nScale), where x is the x-coordinate of the pixel; The vertical weighting factor is wT=(2 (p-1)) ((y 1) nScale), where y is the ordinate of the pixel; nScale is the scaling parameter. This indicates an arithmetic right shift.

5. The method according to claim 4, characterized in that, Also includes: The scaling parameter nScale is derived based on the size of the block.

6. The method according to claim 5, characterized in that, The step of deriving the scaling parameter nScale based on the size of the block includes: determining the scaling parameter nScale as... Where nTbW is the width of the block and nTbH is the height of the block. It is a logarithmic function with base 2.

7. The method according to claim 1, characterized in that, The step of arithmetically right-shifting the integer representation of the result of adding the second predicted pixel value to other values ​​to obtain the third predicted pixel value includes the following calculations: (wL×R_(-1,y)+wT×R_(x,-1)+(64-wL-wT)×P(x,y)+32) 6 in, Indicates arithmetic right shift, P(x,y) is the value of the first predicted pixel. R_(x,-1) and R_(-1,y) represent the values ​​of the nearest reference pixel located above and to the left of the pixel, respectively.

8. The method according to claim 1, characterized in that, The other values ​​are calculated using either DC intra-frame prediction mode or planar intra-frame prediction mode.

9. The method according to claim 1, characterized in that, The image blocks include a first block and a second block. For pixels in the first block and pixels in the second block, the intra-prediction mode used to obtain the predicted pixel value of the pixels in the first block is the DC intra-prediction mode, and the intra-prediction mode used to obtain the predicted pixel value of the pixels in the second block is the planar intra-prediction mode.

10. A device for decoding images, characterized in that, The device includes a processing circuit for performing the method according to any one of claims 1 to 9.

11. A device for decoding images, characterized in that, The device includes processing circuitry, the processing circuitry including one or more processors and a non-transitory computer-readable medium connected to the one or more processors, wherein the non-transitory computer-readable medium includes program code; when the one or more processors execute the program code, the device performs the method according to any one of claims 1 to 9.

12. A non-transitory computer-readable storage medium comprising program code, characterized in that, When the computer device executes the program code, the computer device performs the method according to any one of claims 1 to 9.

13. An apparatus for decoding blocks in an image, characterized in that, The device includes: The acquirer is used to perform intra-prediction using an intra-prediction mode for pixels in the block, and to acquire a first predicted pixel value of the pixel based on one or more reference pixel values, wherein the intra-prediction mode includes a DC intra-prediction mode or a planar intra-prediction mode. A multiplier is used to multiply the first predicted pixel value by the pixel weighting factor to obtain the second predicted pixel value. An adder is used to add the second predicted pixel value to other values; A normalizer is configured to arithmetically right-shift the integer representation of the result obtained by adding the second predicted pixel value to the other values ​​in the adder to obtain a third predicted pixel value; and The reconstruction unit is used to add the third predicted pixel value to the residual of the pixel to obtain the reconstructed pixel value of the pixel; in, The pixel weighting factor is ,in, The parameters of the weighting factor for the pixel are... As a level weighting factor, It is a vertical weighting factor. Indicates an arithmetic left shift; The other value is the sum of one or more addends, the one or more addends including the addends determined according to the one or more reference pixels, the addends determined according to the one or more reference pixels being wL×R_(-1,y)+wT×R_(x,-1), where R_(x,-1) and R_(-1,y) represent the values ​​of the nearest reference pixels located above and to the left of the pixel, respectively; Both the intra-frame prediction mode and the residual of the pixel are obtained from the bitstream of the image.

14. The device according to claim 13, characterized in that, The pixel weighting factor is (64-wL-wT).

15. The device according to claim 13, characterized in that, The horizontal weighting factor wL and / or the vertical weighting factor wT are powers of 2.

16. The device according to claim 13, characterized in that, The horizontal weighting factor is wL=(2 (p-1)) ((x 1) nScale), where x is the x-coordinate of the pixel; The vertical weighting factor is wT=(2 (p-1)) ((y 1) nScale), where y is the ordinate of the pixel; nScale is the scaling parameter. This indicates an arithmetic right shift.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program instructions, wherein when executed by the device or one or more processors, the program instructions cause the device or one or more processors to perform the method according to any one of claims 1 to 9.

Images