Warping mode for intra-frame prediction

Abstract

The present disclosure describes a set of advanced video coding techniques, in particular to intra warpage prediction of samples in a video block, where the samples are intra-predicted using a weighted sum of primary functions for pixel coordinate positions of the samples. A set of weighting coefficients for the weighted sum may be predefined, or derived by an encoder, and signaled in a bitstream containing video blocks at different coding levels.

Description

Incorporated herein by reference

[0001] This application is based on and claims the priority of the U.S. non-provisional patent application with application number 18 / 497,726, filed on October 30, 2023, and entitled “Warp Mode for Intra-Frame Prediction”, which is based on and claims the priority of the U.S. provisional patent application with application number 63 / 443,901, filed on February 7, 2023, and entitled “Intra-Frame Warp Mode”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] The present disclosure describes a set of advanced video coding techniques and is particularly directed to intra-frame warp prediction of samples in a video block, where the samples are intra-predicted using a weighted sum of basis functions for pixel coordinate positions of the samples. Background Art

[0003] Uncompressed digital video may include a series of pictures and may have specific bit rate requirements for transmission bandwidth in storage, data processing, and streaming applications. One purpose of video encoding and decoding may be to reduce redundancy in the uncompressed input video signal through various compression techniques. Summary of the invention

[0004] The present disclosure describes a set of advanced video coding techniques, particularly relating to intra-frame warp prediction of samples in a video block, wherein the samples are intra-predicted using a weighted sum of basis functions for their pixel coordinate positions.

[0005] In some example embodiments, a method for decoding a block in a video bitstream is disclosed. The method may include determining to apply intra-frame warp prediction to the block based on syntax elements in the video bitstream; determining a set of weighting coefficients to be applied to a set of basis functions used in the intra-frame warp prediction; predicting a value of the sample based on a weighted sum of the set of basis functions at pixel coordinate positions of the sample of the block, wherein the set of basis functions is weighted according to the set of weighting coefficients respectively; and reconstructing the block including the sample based on the predicted value.

[0006] In the above example implementation, the basis function set includes power functions of the pixel coordinate positions of the samples up to order N, N being an integer; and the weighted sum of the basis functions includes polynomials of the pixel coordinate positions of the samples up to order N.

[0007] In any of the above-mentioned example embodiments, N=2, and the set of weighting coefficients includes: 6 coefficients for two-dimensional polynomial terms of the horizontal and vertical pixel coordinates of the sample; 5 coefficients for the two-dimensional polynomial terms of the horizontal and vertical pixel coordinates of the sample excluding one cross term; or 4 coefficients for the two-dimensional polynomial terms that depend on the horizontal and vertical pixel coordinates of the sample excluding one cross term, and further include at least one coordinate-independent term.

[0008] In any of the above example embodiments, the at least one coordinate-independent term comprises an offset term comprising a DC value derived from a reconstructed neighborhood of the sample.

[0009] In any of the above-mentioned example embodiments, the set of weighting coefficients further includes an offset coefficient, and the at least one coordinate-independent term includes an offset term, the offset term being the product of the offset coefficient and a reconstructed value of an upper left corner sample of the block; or the set of weighting coefficients further includes a first offset coefficient and a second offset coefficient, and the at least one coordinate-independent term includes the first offset term and the second offset term, the first offset term being the product of the first offset coefficient and a top reconstructed neighbor of the sample, and the second offset term being the product of the second offset coefficient and a left reconstructed neighbor of the sample; or the set of weighting coefficients further includes a first offset coefficient, a second offset coefficient, a third offset coefficient, and a fourth offset coefficient, and the at least one coordinate-independent term includes the first offset term, the second offset term, the third offset term, and the fourth offset term, the first offset term being the product of the first offset coefficient and a top reconstructed neighbor of the sample, the second offset term being the product of the second offset coefficient and a left reconstructed neighbor of the sample, the third offset term being the product of the third offset coefficient and an upper right reconstructed neighbor of the sample, and the fourth offset term being the product of the fourth offset coefficient and a lower left reconstructed neighbor of the sample.

[0010] In any of the above example embodiments, N=2, and the weighted sum of the basis functions includes: p(x, y)=a·x 2 +b·y 2 +c·x·y+d·x+e·y+f; or p(x,y)=a·x 2 +b·y 2 +d·x+e·y+f, wherein a, b, c, d, e and f represent polynomial coefficients, (x, y) represent sample coordinates, and p(x, y) represents the weighted sum.

[0011] In any of the above-mentioned example embodiments, the basis function set includes up to N triangular basis functions of the pixel coordinate position of the sample, where N is an integer, and wherein the N triangular basis functions include: the first N type-2 discrete cosine transform bases; the first N type-7 discrete sine transform bases; or any N type-1 to 8 discrete sine transform bases and type-1 to 8 discrete cosine transform bases.

[0012] In any of the above example embodiments, the set of basis functions includes up to N Karhunen Lòeve Transform (KLT) bases as a function of the pixel coordinate position of the sample, N being an integer.

[0013] In any of the above example embodiments, the N KLT bases comprise eigenvectors of a covariance matrix derived from a reconstructed neighborhood of the sample, or are predefined eigenvectors known to both the encoder and the decoder.

[0014] In any of the above example implementations, the set of basis functions includes a subset of power functions and a subset of trigonometric functions of the pixel coordinate positions of the samples.

[0015] In any of the above example implementations, the method further comprises parsing the video bitstream to determine the set of weighting coefficients signaled in the video bitstream.

[0016] In any of the above example embodiments, the set of weighting coefficients is derived by an encoder using a plurality of rows of templates above and to the left of the block.

[0017] In any of the above-mentioned example embodiments, parsing the video bitstream to determine the set of weighting coefficients includes: parsing the video bitstream to obtain an index of the set of weighting coefficients from the video bitstream; and identifying the set of weighting coefficients in multiple sets of weighting coefficients according to the index.

[0018] In any of the above example implementations, the set of weighting coefficients is clipped to a predefined range of values.

[0019] In any of the above example implementations, the predicted value of the sample is clipped to a predefined value range, and the clipped minimum value and the clipped maximum value of the predicted value are determined by the internal bit depth range.

[0020] In any of the above example embodiments, the set of weighting coefficients is quantized according to one or more predefined precisions.

[0021] In any of the above example embodiments, at least two of the sets of weighting coefficients are quantized to different precisions.

[0022] In any of the above example implementations, terms in the weighted sum of the set of basis functions associated with the same precision are calculated together and rounded.

[0023] In any of the above example embodiments, the precision of the set of weighting coefficients is signaled in the video bitstream using a high level syntax.

[0024] In some other embodiments, a device for processing video information is disclosed, which may include a circuit configured to perform any of the above method embodiments.

[0025] Aspects of the present disclosure also provide a non-transitory computer-readable medium for storing instructions, which, when executed by a computer for video decoding and / or encoding, causes the computer to perform the video decoding and / or encoding method. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Other features, properties and various advantages of the disclosed subject matter will become further apparent from the following detailed description and accompanying drawings, in which:

[0027] Figure 1 is a schematic diagram of a simplified block diagram of a communication system (100) according to an exemplary embodiment;

[0028] Figure 2 is a schematic diagram of a simplified block diagram of a communication system (200) according to an exemplary embodiment;

[0029] Figure 3 is a schematic diagram of a simplified block diagram of a video decoder according to an exemplary embodiment;

[0030] Figure 4 is a schematic diagram of a simplified block diagram of a video encoder according to an exemplary embodiment;

[0031] Figure 5 is a block diagram of a video encoder according to another exemplary embodiment;

[0032] Figure 6 is a block diagram of a video decoder according to another exemplary embodiment;

[0033] Figure 7 A schematic diagram showing an exemplary subset of intra-prediction directional modes;

[0034] Figure 8 The nominal angle in directional intra prediction is shown;

[0035] Fig. 9 A schematic diagram showing an exemplary intra prediction direction;

[0036] Fig.10The top, left and upper left positions for PAETH mode on the coded block are shown;

[0037] Fig.11 An example recursive intra-frame filtering mode is shown;

[0038] Fig.12 shows a flow chart for performing an example intra prediction in intra warping mode; and

[0039] Fig.13 A schematic diagram of a computer system according to an exemplary embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0040] Throughout the specification and claims, terms may have subtle meanings suggested or implied by the context, not just the express meaning. The phrases "in one embodiment / implementation" or "in some embodiments / implementations" as used herein do not necessarily refer to the same embodiment / implementation, and the phrases "in another embodiment / implementation" or "in other embodiments" as used herein do not necessarily refer to different embodiments. For example, the claimed subject matter is intended to include all or part of a combination of exemplary embodiments / implementations.

[0041] In general, terms can be understood at least in part from usage in context. For example, terms used herein, such as "and", "or", or "and / or", can include various context-dependent meanings. Typically, "or", if used in an association list, such as A, B, or C, is intended to mean A, B, and C, which are used in an inclusive sense herein, and A, B, or C, which are used in an exclusive sense herein. In addition, the terms "one or more", "at least one", "one (a)", "an" or "the" used herein, at least in part depending on the context, can be used in a singular sense or in a plural sense. In addition, the term "based on" or "determined by..." can be understood as not necessarily intended to express a set of exclusive factors, but can allow for the presence of other factors that are not necessarily explicitly described again, which depends at least in part on the context.

[0042] Figure 1 A simplified block diagram of a communication system (100) according to an embodiment of the present disclosure is shown. The communication system (100) includes a plurality of terminal devices, such as 110, 120, 130, and 140, which can communicate with each other via, for example, a network (150). Figure 1In an embodiment, the first pair of terminal devices (110) and (120) can perform unidirectional data transmission. For example, the terminal device (110) can encode video data in the form of one or more encoded bit streams (such as video picture streams collected by the terminal device (110)) for transmission through the network (150). The terminal device (120) can receive the encoded video data from the network (150), decode the encoded video data to restore the video picture, and display the video picture based on the restored video data. The unidirectional data transmission can be implemented in applications such as media services.

[0043] In another embodiment, the second pair of terminal devices (130) and (140) can perform bidirectional transmission of encoded video data, such as during a video conferencing application. For bidirectional data transmission, in one example, each of the terminal devices (130) and (140) can encode video data (such as a video picture stream collected by the terminal device) for transmission to the other terminal device (130) and (140) and can also receive the encoded video data from the other terminal device to restore and display the video picture.

[0044] exist Figure 1 In the embodiments of the present disclosure, the terminal device may be implemented as a server, a personal computer, and a smart phone, but the applicability of the basic principles of the present disclosure may not be limited thereto. The embodiments of the present disclosure may be implemented in a desktop computer, a laptop computer, a tablet computer, a media player, a wearable computer, a dedicated video conferencing device, etc. The network (150) represents any number or type of network that transmits encoded video data between terminal devices, including, for example, wired (wired) and / or wireless communication networks. The communication network (150) may exchange data in circuit switching, packet switching, and / or other types of channels. Representative networks may include telecommunication networks, local area networks, wide area networks, and / or the Internet.

[0045] As an example of application of the disclosed subject matter, Figure 2 The placement of the video encoder and video decoder in a video streaming environment is shown. The disclosed subject matter is equally applicable to other video applications, including, for example, video conferencing, digital TV broadcasting, gaming, virtual reality, storage of compressed video on digital media including CDs, DVDs, memory sticks, etc., and the like.

[0046] like Figure 2As shown, the video streaming system may include a video acquisition subsystem (213), which may include a video source (201), such as a digital camera, for creating an uncompressed video picture or image stream (202). In an embodiment, the video picture stream (202) includes samples recorded by the digital camera of the video source (201). Compared to the encoded video data (204) (or the encoded video bitstream), the video picture stream (202) is depicted as a thick line to emphasize the high data volume of the video picture stream, and the video picture stream (202) can be processed by an electronic device (220), which includes a video encoder (203) coupled to the video source (201). The video encoder (203) may include hardware, software, or a combination of hardware and software to implement or implement various aspects of the disclosed subject matter as described in more detail below. Compared to the uncompressed video picture stream (202), the encoded video data (204) (or the encoded video bitstream (204)) is depicted as a thin line to emphasize the lower amount of data of the encoded video data (204) (or the encoded video bitstream (204)), which can be stored on the streaming server (205) for future use, or directly stored to a downstream video device (not shown). One or more streaming client subsystems, such as Figure 2 The client subsystem (206) and the client subsystem (208) in the video transmission server (205) can access the streaming server (205) to retrieve the copy (207) and the copy (209) of the encoded video data (204). The client subsystem (206) can include, for example, a video decoder (210) in the electronic device (230). The video decoder (210) decodes the incoming copy (207) of the encoded video data and produces an output video picture stream (211) that is decompressed and can be presented on a display (212) (such as a display screen) or other presentation device (not depicted).

[0047] Figure 3 A block diagram of a video decoder (310) of an electronic device (330) according to any of the following embodiments of the present disclosure is shown. The electronic device (330) may include a receiver (331) (e.g., a receiving circuit). The video decoder (310) may be used to replace Figure 2 A video decoder (210) of an embodiment.

[0048] like Figure 3As shown, a receiver (331) may receive one or more encoded video sequences from a channel (301). To prevent network jitter and / or process playback timing, a buffer memory (315) may be provided between the receiver (331) and an entropy decoder / parser (320) (hereinafter referred to as "parser (320)"). The parser (320) may reconstruct symbols (321) based on the encoded video sequence. The categories of these symbols include information for managing the operation of the video decoder (310) and potential information for controlling a presentation device such as a display (312) (e.g., a display screen). The parser (320) may parse / entropy decode the encoded video sequence. The parser (320) may extract a subgroup parameter set for at least one of the subgroups of pixels in the video decoder from the encoded video sequence. The subgroup may include a group of pictures (Group of Pictures, GOP), a picture, a tile, a slice, a macroblock, a coding unit (Coding Unit, CU), a block, a transform unit (Transform Unit, TU), a prediction unit (Prediction Unit, PU), and the like. The parser (320) may also extract information from the encoded video sequence, such as transform coefficients (e.g., Fourier transform coefficients), quantizer parameter values, motion vectors, etc. The reconstruction of the symbols (321) may involve a number of different processing or functional units. Which units are involved and how they are involved may be controlled by subgroup control information parsed by the parser (320) from the encoded video sequence.

[0049] The first unit may include a scaler / inverse transform unit (351). The scaler / inverse transform unit (351) may receive quantized transform coefficients as symbols (321) and control information from the parser (320), including information indicating which type of inverse transform to use, block size, quantization factors / parameters, quantization scaling matrices, etc. The scaler / inverse transform unit (351) may output a block including sample values, which may be input into an aggregator (355).

[0050] In some cases, the output samples of the scaler / inverse transform unit (351) may belong to an intra-coded block, i.e., a block that does not use predictive information from a previously reconstructed picture, but may use predictive information from a previously reconstructed portion of the current picture. Such predictive information may be provided by an intra-picture prediction unit (352). In some cases, the intra-picture prediction unit (352) may use surrounding block information that has already been reconstructed and stored in a current picture buffer (358) to generate a block of the same size and shape as the block being reconstructed. For example, the current picture buffer (358) buffers a partially reconstructed current picture and / or a fully reconstructed current picture. In some embodiments, the aggregator (455) may add the prediction information generated by the intra-prediction unit (352) to the output sample information provided by the scaler / inverse transform unit (351) on a per-sample basis.

[0051] In other cases, the output samples of the scaler / inverse transform unit (351) may belong to an inter-coded and potentially motion compensated block. In this case, the motion compensated prediction unit (353) may access the reference picture memory (357) based on the motion vector to extract samples for inter-picture prediction. After motion compensation of the extracted reference samples according to the block-related symbols (321), these samples may be added to the output of the scaler / inverse transform unit (351) by an aggregator (355) (the output of unit 351 is called residual samples or residual signal), thereby generating output sample information.

[0052] The output samples of the aggregator (355) can be used by various loop filtering techniques in a loop filter unit (356) including a plurality of loop filters. The output of the loop filter unit (356) can be a sample stream, which can be output to a rendering device (312) and stored in a reference picture memory (357) for subsequent inter-frame picture prediction.

[0053] Figure 4 A block diagram of a video encoder (403) according to an exemplary embodiment of the present disclosure is shown. The video encoder (403) may be provided in an electronic device (420). The electronic device (420) may further include a transmitter (440) (e.g., a transmission circuit). The video encoder (403) may be used to replace Figure 4 A video encoder (403) in an embodiment.

[0054] The video encoder (403) may receive video samples from a video source (401). According to some exemplary embodiments, the video encoder (403) may encode and compress pictures of a source video sequence into an encoded video sequence (443) in real time or under any other time constraints required by an application. Implementing an appropriate encoding speed constitutes a function of a controller (450). In some embodiments, the controller (450) may be functionally coupled to and control other functional units as described below. Parameters set by the controller (450) may include rate control related parameters (picture skipping, quantizer, lambda value of rate-distortion optimization techniques, etc.), picture size, group of pictures (GOP) layout, maximum motion vector search range, etc.

[0055] In some exemplary embodiments, the video encoder (403) may operate in an encoding loop. The encoding loop may include a source encoder (430) and a (local) decoder (433) embedded in the video encoder (403). Even if the embedded decoder (433) processes the encoded video stream by the source encoder (430) without entropy coding, the decoder (433) reconstructs the symbols to create sample data in a manner similar to that of the (remote) decoder to create sample data (because in the video compression technology considered in the disclosed subject matter, any compression between the symbols in the entropy coding and the encoded video code stream is lossless). At this point, it can be observed that any decoder technology other than parsing / entropy decoding that may only exist in the decoder may also necessarily need to exist in the corresponding encoder in substantially the same functional form. For this reason, the disclosed subject matter may sometimes focus on the decoder operation, which is associated with the decoding portion of the encoder. Therefore, the description of the encoder technology can be simplified because the encoder technology is mutually inverse to the decoder technology described in detail. A more detailed description of the encoder is provided below only in certain areas or aspects.

[0056] During operation in some exemplary embodiments, the source encoder (430) may perform motion compensated predictive coding, which predictively encodes an input picture with reference to one or more previously encoded pictures in a video sequence designated as "reference pictures."

[0057] The local video decoder (433) can decode the encoded video data of a picture that can be designated as a reference picture. The local video decoder (433) replicates the decoding process that can be performed by the video decoder on the reference picture and can cause the reconstructed reference picture to be stored in the reference picture memory (434). In this way, the video encoder (403) can locally store a copy of the reconstructed reference picture that has common content (absent transmission errors) with the reconstructed reference picture to be obtained by the far-end (remote) video decoder.

[0058] The predictor (435) may perform prediction search for the encoding engine (432). That is, for a new picture to be encoded, the predictor (435) may search the reference picture memory (434) for sample data (as a candidate reference pixel block) or certain metadata, such as reference picture motion vector, block shape, etc., that may serve as a suitable prediction reference for the new picture.

[0059] The controller (450) may manage encoding operations of the source encoder (430), including, for example, setting parameters and subgroup parameters for encoding video data.

[0060] The outputs of all of the above functional units may be entropy encoded in an entropy encoder (445). The transmitter (440) may buffer the encoded video sequence created by the entropy encoder (445) in preparation for transmission over a communication channel (460), which may be a hardware / software link to a storage device where the encoded video data will be stored. The transmitter (440) may combine the encoded video data from the video encoder (403) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (source not shown).

[0061] The controller (450) may manage the operation of the video encoder (403). During encoding, the controller (450) may assign a certain coded picture type to each coded picture, but this may affect the coding techniques that may be applied to the corresponding picture. For example, a picture may generally be assigned to any of the following picture types: intra picture (I picture), predicted picture (P picture), bidirectional predicted picture (B picture), multi-predicted picture. A source picture may generally be spatially subdivided into a plurality of sample coding blocks, as described in further detail below.

[0062] Figure 5 A schematic diagram of a video encoder (503) according to another exemplary embodiment of the present disclosure is shown. The video encoder (503) is configured to receive a processed block (e.g., a prediction block) of sample values ​​within a current video picture in a video picture sequence, and encode the processed block into an encoded picture that is part of an encoded video sequence. The exemplary video encoder (503) may be used to replace Figure 4 A video encoder (403) in an embodiment.

[0063] For example, the video encoder (503) receives a matrix of sample values ​​for a processing block. The video encoder (503) then uses, for example, rate-distortion optimization (RDO) to determine whether to use intra mode, inter mode, or bi-prediction mode to encode the processing block.

[0064] exist Figure 5 In an embodiment of the present invention, the video encoder (503) includes an inter-frame encoder (530), an intra-frame encoder (522), a residual calculator (523), a switch (526), ​​a residual encoder (524), a general controller (521) and an entropy encoder (525) coupled together, such as Figure 5 An example arrangement is shown in FIG.

[0065] The inter-frame encoder (530) is used to receive a sample of a current block (e.g., a processing block), compare the block with one or more reference blocks in a reference picture (e.g., blocks in a previous picture and a subsequent picture in a display order), generate inter-frame prediction information (e.g., redundant information description, motion vector, merge mode information according to the inter-frame coding technology), and calculate an inter-frame prediction result (e.g., a predicted block) based on the inter-frame prediction information using any suitable technology.

[0066] The intra-frame encoder (522) is used to receive samples of a current block (e.g., a processing block), compare the block with encoded blocks in the same picture, generate quantization coefficients after transformation, and in some cases also generate intra-frame prediction information (e.g., intra-frame prediction direction information based on one or more intra-frame coding techniques).

[0067] The general controller (521) may be used to determine general control data and control other components of the video encoder (503) based on the general control data, for example, determining a prediction mode for a block and providing a control signal to a switch (526) based on the prediction mode.

[0068] The residual calculator (523) may be used to calculate the difference (residual data) between the received block and the prediction result of the block selected from the intra encoder (522) or the inter encoder (530). The residual encoder (524) may be used to encode the residual data to generate transform coefficients. The transform coefficients are then quantized to obtain quantized transform coefficients. In various embodiments, the video encoder (503) further includes a residual decoder (528). The residual decoder (528) is used to perform an inverse transform and generate decoded residual data. The entropy encoder (525) may be used to format the bitstream to include the encoded blocks and perform entropy encoding.

[0069] Figure 6 FIG. 6 is a schematic diagram of an exemplary video decoder (610) according to another embodiment of the present disclosure. The video decoder (610) is used to receive an encoded image as part of an encoded video sequence, and decode the encoded image to generate a reconstructed picture. In an embodiment, the video decoder (610) may be used to replace Figure 4 A video decoder (410) of an embodiment.

[0070] exist Figure 6In an embodiment, the video decoder (610) includes an entropy decoder (671), an inter-frame decoder (680), a residual decoder (673), a reconstruction module (674) and an intra-frame decoder (672) coupled together, such as Figure 6 An example arrangement is shown in FIG.

[0071] The entropy decoder (671) may be used to reconstruct certain symbols from an encoded picture, the symbols representing syntax elements constituting the encoded picture. The inter-frame decoder (680) may be used to receive inter-frame prediction information and generate an inter-frame prediction result based on the inter-frame prediction information. The intra-frame decoder (672) may be used to receive intra-frame prediction information and generate a prediction result based on the intra-frame prediction information. The residual decoder (673) may be used to perform inverse quantization to extract dequantized transform coefficients, and process the dequantized transform coefficients to convert the residual from the frequency domain to the spatial domain. The reconstruction module (674) may be used to combine the residual output by the residual decoder (673) with the prediction result (which may be output by the inter-frame prediction module or the intra-frame prediction module) in the spatial domain to form a reconstructed block, the reconstructed block forming part of the reconstructed picture, and the reconstructed picture may then be part of the reconstructed video.

[0072] It should be noted that the video encoders (203), (403) and (503) and the video decoders (210), (310) and (610) may be implemented using any suitable technology. In some exemplary embodiments, the video encoders (203), (403) and (503) and the video decoders (210), (310) and (610) may be implemented using one or more integrated circuits. In another embodiment, the video encoders (203), (403) and (503) and the video decoders (210), (310) and (610) may be implemented using one or more processors that execute software instructions.

[0073] Turning to the block partitioning for encoding and decoding, the general partitioning can start from the basic block and can follow a predefined set of rules, a specific pattern, a partition tree, or any partition structure or scheme. Partitioning can be hierarchical and recursive. After following any of the example partitioning processes described below or other processes or combinations thereof to partition or partition the basic block, a final set of partitions or coding blocks can be obtained. Each of these partitions can be in one of the various partition levels in the partition hierarchy and can have various shapes. Each partition can be referred to as a coding block (CB). For the various example partitioning implementations further described below, each generated CB can have any allowed size and partition level. Such partitions are called coding blocks because they can form units to which some basic encoding / decoding decisions can be made, and the encoding / decoding parameters can be optimized, determined, and signaled in the encoded video bitstream. The highest or deepest level in the final partition represents the depth of the coding block partition structure of the tree. The coding block can be a luminance coding block or a chrominance coding block. The CB tree structure for each color can be referred to as a coding block tree (CBT). The coding blocks of all color channels can be collectively referred to as coding units (CUs). The hierarchical structure of all color channels may be collectively referred to as a coding tree unit (CTU). The partitioning modes or structures of various color channels in a CTU may be the same or different.

[0074] In some embodiments, the partition tree schemes or structures for luma and chroma channels may not have to be the same. In other words, luma and chroma channels may have separate coding tree structures or modes. In addition, whether the luma and chroma channels use the same or different coding partition tree structures, and the actual coding partition tree structures to be used may depend on whether the slice being encoded is a P slice, a B slice, or an I slice. For example, for an I slice, the chroma channel and the luma channel may have separate coding partition tree structures or coding partition tree structure modes, while for a P slice or a B slice, the luma and chroma channels may share the same coding partition tree scheme. When a separate coding partition tree structure or mode is applied, the luma channel may be divided into multiple CBs by one coding partition tree structure, and the chroma channel may be divided into multiple chroma CBs by another coding partition tree structure.

[0075] A video block (PB or CB, also referred to as a PB when not further divided into multiple prediction blocks) can be predicted in various ways, rather than directly encoded, thereby taking advantage of various correlations and redundancies in the video data to improve compression efficiency. Accordingly, this prediction can be performed in various modes. For example, a video block can be predicted by intra-frame prediction or inter-frame prediction. In particular, in the inter-frame prediction mode, a video block can be predicted by one or more other reference blocks or inter-frame predictor blocks from one or more other frames by single reference or composite reference inter-frame prediction. In order to implement inter-frame prediction, a reference block can be specified by its frame identifier (the temporal position of the reference block) and a motion vector indicating the spatial offset between the current block being encoded or decoded and the reference block (the spatial position of the reference block). The reference frame identifier and the motion vector can be signaled in the bitstream. The motion vector as a spatial block offset can be signaled directly, or the motion vector itself can be predicted by another reference motion vector or a predictor motion vector. For example, the current motion vector may be predicted directly by a reference motion vector (e.g., of a candidate neighboring block), or by a combination of a reference motion vector and a motion vector difference (MVD) between the current motion vector and the reference motion vector. The latter may be referred to as a merge mode with motion vector difference (MMVD). The reference motion vector may be identified in the bitstream as a pointer to, for example, a spatial neighboring block of the current block or a temporally adjacent but spatially co-located block.

[0076] Returning to the intra prediction process, samples in a block (e.g., a luma or chroma prediction block, or a coding block if not further divided into prediction blocks) are predicted by samples of an adjacent row, a next adjacent row, or other one or more rows, or a combination thereof, to generate a prediction block. The residual between the actual block being encoded and the prediction block may then be processed by a transform followed by quantization. Various intra prediction modes may be made available, and parameters related to intra mode selection as well as other parameters may be signaled in the bitstream. For example, various intra prediction modes may involve one or more row positions for prediction samples, directions for selecting prediction samples from one or more prediction rows, and other special intra prediction modes.

[0077] For example, a set of intra-prediction modes (interchangeably referred to as "intra-modes") may include a predefined number of directional intra-prediction modes. Figure 7 As described in the example embodiment of FIG. 1 , these intra prediction modes may correspond to a predefined number of directions along which out-of-block samples are selected as predictions for samples being predicted in a particular block. In another specific example embodiment, eight (8) main directional modes may be supported and correspond to predefined angles of 45 to 207 degrees from the horizontal axis, also as shown in FIG. Figure 7 shown.

[0078] In some other embodiments of intra prediction, in order to further exploit more types of spatial redundancy in directional textures, the directional intra mode can be further extended to a set of angles with finer granularity. For example, the above 8-angle embodiment can be configured to provide eight nominal angles, namely V_PRED, H_PRED, D45_PRED, D135_PRED, D113_PRED, D157_PRED, D203_PRED and D67_PRED, as shown in FIG. Figure 8 As shown, and for each nominal angle, a predefined number (e.g., 7) of finer angles can be added. With such an extension, a larger total number (e.g., 56 in this example) of directional angles corresponding to the same number of predefined directional intra modes can be used for intra prediction. The predicted angle can be represented by the nominal intra angle plus the angle increment. For the specific example above, each nominal angle has 7 finer angle directions, and the angle increment can be a step size of -3 to 3 times 3 degrees.

[0079] As another example, some other angular schemes for direction prediction may be used, such as Fig. 9 As shown, there are 65 different prediction angles.

[0080] In some example embodiments, as an alternative or in addition to the above-mentioned directional intra-frame modes, a predetermined number of non-directional intra-frame prediction modes may be predefined and made available. For example, 5 non-directional intra-frame modes may be specified, referred to as smooth intra-frame prediction modes. These non-directional intra-frame prediction modes may be specifically referred to as DC, PAETH, SMOOTH, SMOOTH_V, and SMOOTH_H intra-frame modes. Fig.10 The prediction of samples for a particular block under these example non-directional modes is shown. As an example, Fig.10A 4×4 block 1002 is shown that is predicted by samples from a top neighboring row and / or a left neighboring row. A particular sample 1010 in the block 1002 may correspond to a sample 1004 directly above the sample 1010 in the top neighboring row of the block 1002, a sample 1006 above and to the left of the sample 2011 that is the intersection of the top neighboring row and the left neighboring row, and a sample 1008 directly to the left of the sample 1010 in the left neighboring row of the block 1002. For the example DC intra prediction mode, the average of the left neighboring sample 1008 and the above neighboring sample 1004 may be used as a predictor for the sample 1010. For the example PAETH intra prediction mode, the top reference sample 1004, the left reference sample 1008, and the top left reference sample 1006 may be extracted, and then any value that is closest to (top+left-topleft) among these three reference samples may be set as the predictor for the sample 1010. For the example SMOOTH_V intra prediction mode, sample 1010 may be predicted by quadratic interpolation in the vertical direction of the upper left neighboring sample 1006 and the left neighboring sample 1008. For the example SMOOTH_H intra prediction mode, sample 1010 may be predicted by quadratic interpolation in the horizontal direction of the upper left neighboring sample 2006 and the top neighboring sample 1004. For the example SMOOTH intra prediction mode, sample 1010 may be predicted by the average of the quadratic interpolation in the vertical and horizontal directions. The above-described implementation of the non-directional intra mode is described only as a non-limiting example. Other neighboring rows, other non-directional selections of samples, and other ways of combining prediction samples for predicting a particular sample in a prediction block may also be considered.

[0081] The encoder may be signaled in the bitstream to select a specific intra-frame prediction mode from the above-mentioned directional or non-directional modes at various coding levels (pictures, slices, blocks, units, etc.). In some example embodiments, exemplary 8 nominal directional modes and 5 non-angular smooth modes (a total of 13 options) may be first signaled. Then, if the signaled mode is one of the 8 nominal angle intra-frame modes, an index is further signaled to indicate the selected angle increment relative to the corresponding signaled nominal angle. In some other example embodiments, all intra-frame prediction modes may be indexed together (e.g., 56 directional modes plus 5 non-directional modes to generate 61 intra-frame prediction modes) for signaling.

[0082] In some example embodiments, the example 56 or other number of directional intra prediction modes may be implemented with a unified directional predictor that projects each sample of the block to a reference subsample location and interpolates the reference samples using, for example, a 2-tap bilinear filter.

[0083] In some embodiments, in order to capture the attenuated spatial correlation with the reference on the edge, additional filter modes, called filter intra modes, can be designed. For these modes, in addition to the samples outside the block, the predicted samples within the block can also be used as intra prediction reference samples for some small blocks (patches) within the block. For example, these modes can be predefined and can be used for intra prediction of at least luminance blocks (or only luminance blocks). A predefined number (for example, five) of filter intra modes can be pre-designed, each filter intra mode is represented by a set of n-tap filters (for example, 7-tap filters), reflecting the correlation between the samples in, for example, a 4×2 small block and its n=7 adjacent neighbors. In other words, the weighting factors of the n-tap filter can be position-dependent. Taking 8×8 blocks, 4×2 small blocks and 7-tap filtering as an example, as Fig.11 As shown, the 8×8 block 2002 can be divided into 8 4×2 small blocks. Fig.11 In the example, they are represented by B0, B1, B2, B3, B4, B5, B6, and B7. For each small block, its 7 neighbors (in Fig.11 For block B0, all neighboring samples may have been reconstructed and used to predict block B0. However, for other blocks, some neighboring samples are in the current block and may not have been reconstructed yet, so their predicted values ​​can be used as reference. For example, Fig.11 As shown, all 7 neighboring samples of the small block B7 of the current block have not been reconstructed, so the prediction samples of these neighboring samples are used.

[0084] The intra-frame prediction implementations of the above examples include directional intra-frame prediction modes, various non-directional modes, and recursive filtering intra-frame prediction modes, which generally rely on linear prediction, which relies on linear operations based on other intra-frame reference samples (e.g., adjacent samples) to estimate or predict specific pixels. In some cases, texture patterns in images can be highly dynamic and do not necessarily present linear relationships in adjacent pixels. Therefore, intra-frame prediction based on nonlinear relationships between adjacent samples can be designed to better capture this relationship and provide improved intra-frame prediction coding gain, as described in further detail below.

[0085] The following example methods / implementations / embodiments may be used alone or in combination in any order. In addition, each of these methods / implementations / embodiments (part or all of the encoder / decoder) may be implemented by a processing circuit (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored in a non-volatile computer-readable medium. In the following, the term "block" may be interpreted as a prediction block, a coding block, or a coding unit, i.e., a CU.

[0086] In some example embodiments, a predefined function may be used to derive intra-prediction samples, the predefined function using horizontal and vertical coordinate values ​​and / or surrounding sample values ​​as input, and adjacent reconstructed samples may be used to derive function parameters (or model parameters) or to explicitly signal function parameters. In other words, linear and / or nonlinear relationships among adjacent samples reflecting texture information may be parameterized in a predefined model. Such a function may alternatively be referred to as a prediction function or prediction model. The term "predefined function" may be used to refer to a form of a function. In other words, the form of the prediction function may be predefined, as shown in the following example.

[0087] In this intra prediction approach, because the predictor of a sample depends on the sample pixel coordinates within the coding block of the sample, it is similar to warped inter prediction in a sense, where an affine model can be used to convert a block into a warped reference block, so that the predictor sample in the reference frame of the sample being predicted in the current block depends on the pixel coordinates of the sample being predicted (due to spatial warping). Due to this pixel coordinate dependency of the predictor sample, the above intra prediction and the intra prediction described in further detail below can be referred to as intra warping. The intra prediction mode used for this prediction approach can be correspondingly referred to as intra warping mode. The intra warping mode can be regarded as an additional intra prediction mode to the other intra prediction modes (e.g., directional intra prediction mode, non-directional intra prediction mode, filter intra prediction mode, etc.) mentioned above. In some embodiments, this warped intra prediction mode can be parallel to the non-warped intra prediction mode, and these other intra prediction modes mentioned above exist in the non-warped intra prediction mode. Accordingly, the encoder provides signaling in the bitstream to indicate to the decoder which of the multiple intra prediction modes is being used at a particular coding level. Alternatively, the encoder may provide signaling in the bitstream to indicate to the decoder whether the warped intra prediction mode is being used for a particular coding level, and then send additional signaling if encoding is not performed in the warped intra prediction mode to indicate which of the other non-warped intra prediction modes is being used for the particular coding level.

[0088] In one example, the predefined prediction function may be a polynomial function up to Nth order.

[0089] In one example, the prediction function may be in the form of p(x,y)=a·x 2 +b·y 2 +c·x·y+d·x+e·y+f, of the form with model parameters a, b, c, d, e and f, where (x, y) are the sample coordinates, e.g. relative to the top left corner of the block.

[0090] In one example, the prediction function may be in the form of p(x,y)=a·x 2 +b·y2 +d·x+e·y+f, which has model parameters a, b, c, d, e, and f, where (x, y) are the sample coordinates.

[0091] In one example, the prediction function may be of the form: p(x,y)=a·x 2 +b·y 2 +d·x+e·y+DC, (1) This form has model parameters a, b, d, and e, where (x, y) are the sample coordinates and DC represents the DC prediction value derived from neighboring reconstructed samples.

[0092] In one example, the prediction function may be of the form: p(x,y)=a·x 2 +b·y 2 +d·x+e·y+w TL TL, (2) This form has model parameters a, b, d, e and w TL , where (x, y) are the sample coordinates and TL is the top-left sample of the current block (TL is used in this example instead of DC above to avoid pipeline delays for calculating DC).

[0093] In one example, the prediction function may be of the form: p(x,y)=a·x 2 +b·y 2 +d·x+e·y+w T ·T+w L ·L, (3) This form has model parameters a, b, d, e, w T and w L , where (x, y) represents the sample coordinates, T represents the value of the sample located at the top adjacent position of the current pixel, and L represents the value of the sample located at the left adjacent position of the current pixel.

[0094] In one example, the prediction function may be of the form: p(x,y)=a·x 2 +b·y 2 +d·x+e·y+w T ·T+w L ·L+w TR TR+w BL BL, (4) This form has model parameters a, b, d, e, w T 、w L 、w TR and w BL, where (x, y) represents the sample coordinates, T represents the sample value at the top adjacent position of the current pixel, L represents the sample value at the left adjacent position of the current pixel, TR represents the sample value at the upper right adjacent position of the current pixel, and BL represents the sample value at the lower left adjacent position of the current pixel.

[0095] In the above examples, the order N of the above example polynomial can be 1 (linear), 2 (2nd order), 3, 4, ... Other model parameters can be introduced.

[0096] In some other example embodiments, the predefined prediction function may be in the form of a weighted sum of a plurality of triangular basis functions having up to N bases (each triangular function is referred to as a basis function or basis) rather than a polynomial function. For example, the triangular basis functions or bases may be orthogonal to each other. Each triangular basis function may be a function of (x, y), such as sample coordinates relative to the upper left corner of a block. For example, the triangular basis function may be a 2D triangular basis function, or may be a combination of 2D triangular basis functions.

[0097] In one example, the basis may include the first N discrete cosine transform type 2 (DCT-2) bases. In one example, the basis may include the first N discrete sine transform type 7 (DST-7) bases. In one example, the basis may be from any one of the 1 to 8 type DCT and 1 to 8 type DST bases.

[0098] In some other example embodiments, the predefined prediction function can be in the form of a weighted sum of multiple Karhunen Lòeve transform (KLT) bases up to N KLT bases. For example, the KLT bases can be eigenvectors of a covariance matrix derived from adjacent reconstructed samples. For example, the KLT bases can be predefined eigenvectors known to both the encoder and the decoder. Likewise, each KLT basis can be a function of (x, y), such as sample coordinates relative to the upper left corner of a block.

[0099] In some other example implementations, the predefined prediction function may be a hybrid form of the above-mentioned polynomial function and trigonometric function, and the number of terms with corresponding model parameters may be predefined.

[0100] The various model parameters described above may be predefined or signaled. In some example embodiments, the model parameters may be predefined as a set of indices. For example, the encoder selects a set or index of a particular block in real time and at various levels (picture, frame, slice, macroblock, block, etc.) and signals it in the bitstream. In some example embodiments, the model parameters may be derived by the encoder in real time and signaled in the bitstream at various levels.

[0101] In some example embodiments, model parameters used in predefined prediction functions may be clipped to predefined ranges (so as to limit the number of bits representing the values ​​of these parameters). Model parameters with clipped ranges may help reduce computational burden. Furthermore, where model parameters are signaled, clipping of parameter value ranges may further help reduce the number of bits used to signal the parameters. In some embodiments, one or more ranges of parameter values ​​may be signaled rather than predefined.

[0102] In some example embodiments, the prediction value generated by the predefined prediction function may be clipped to a predefined range in order to reduce the computational burden of the encoder and / or decoder.

[0103] In the above example implementations, the clipped minimum and maximum values ​​of the predicted values ​​of the predefined function may depend on the internal bit depth range. For example, if the internal bit depth is 8 (or 10) for computational purposes, the minimum and maximum ranges may be from 0 (0) to 255 (1023). In some example implementations, the clipped minimum and maximum values ​​or ranges may be signaled using a high-level syntax.

[0104] In some example embodiments, model parameters used in the predefined prediction function may be quantized to a specific precision, such as 1 (integer), 1 / 2, 1 / 4, 1 / 8, 1 / 16, 1 / 32, 1 / 64, 1 / 128, 1 / 256.

[0105] In some example embodiments, different model parameters of the predefined prediction function may be quantized differently. For example, for the example prediction function in equation (1) above, parameters a, b, and c may be quantized to 1 / 16 precision, and parameters d, e, and f may be quantized to 1 / 4 precision. Thus, the prediction function may be implemented as p(x, y) = (a·x 2 +b·y 2 +c·x·y+8)>>4+(d·x+e·y+f+2)>>2, where the model parameters a, b, c, d, e and f are all integers representing shifted (magnified) versions of these parameters according to their precision.

[0106] In some example embodiments, the weighted terms in the above prediction functions associated with the same accuracy (e.g., a·x 2 or b u 2 ) can be calculated and rounded together.

[0107] In some example embodiments, the precision of model parameters may be signaled using a high-level syntax and at various levels. A set of available precisions may be predefined, and the precision selection of a parameter or parameter group may be signaled by an index into the predefined set of available precisions.

[0108] In some example embodiments, the coefficients of the above functions may be derived at the encoder and decoder side according to a predefined or signaled algorithm or optimization process. The derivation may be based on using a single or multiple row template above and to the left of the current block containing the reconstructed samples. The template representing the selected samples used to derive the optimization may be predefined or may be signaled. For example, a set of templates may be predefined, and the template to be used may be signaled by an index within the set of templates at various coding levels in the bitstream.

[0109] In some example embodiments, the current block may be divided into a plurality of sub-blocks (where the pixel width / height is greater than or equal to a predetermined value, such as 2), and the model parameters or coefficients described above may be calculated / derived for each sub-block using available surrounding information of the current sub-block.

[0110] In some example embodiments, the coefficients of the above model prediction functions may be explicitly signaled in the bitstream and parsed at the decoder side to perform reconstruction.

[0111] Fig.12 A flow chart of an example method 1200 for decoding a block in a video bitstream is shown. The method 1200 starts at S1201. In step S1210, based on syntax elements in the video bitstream, it is determined to apply intra-frame warp prediction to the block. In step 1220, a set of weighting coefficients is determined, and the set of weighting coefficients is to be applied to a set of basis functions used in the intra-frame warp prediction. In step S1230, based on the weighted sum of the basis function set at the pixel coordinate positions of the samples of the block, the value of the sample is predicted, wherein the basis function set is weighted according to the set of weighting coefficients respectively. In step S1240, the block including the sample is reconstructed based on the predicted value. The flow of the method 1200 ends at S1299.

[0112] The above operations can be combined or arranged in any number or order as needed. Two or more steps and / or operations can be performed in parallel. The embodiments and implementation methods in the present disclosure can be used alone or in combination in any order. In addition, each method (or embodiment), encoder and decoder can be implemented by a processing circuit (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored in a non-volatile computer-readable medium. The embodiments in the present disclosure can be applied to luminance blocks or chrominance blocks. The term "block" can be interpreted as a prediction block, a coding block, or a coding unit, i.e., a CU. The term "block" here can also be used to refer to a transform block. In the following terms, when referring to a block size, it may refer to the width or height of the block, or the maximum value of the width and height, or the minimum value of the width and height, or the area size of the block (width*height), or the aspect ratio (width:height, or height:width).

[0113] The above techniques may be implemented as computer software via computer-readable instructions and physically stored in one or more computer-readable media. Fig.13 A computer system (1300) is shown that is suitable for implementing certain embodiments of the disclosed subject matter.

[0114] The computer software may be encoded in any suitable machine code or computer language, and may be assembled, compiled, linked, or the like to create a code comprising instructions, which may be directly executed by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc., or executed by decoding, microcode, or the like.

[0115] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, IoT devices, etc.

[0116] Fig.13 The components shown for the computer system (1300) are exemplary in nature and are not intended to impose any limitations on the scope of use or functionality of computer software implementing the disclosed embodiments. Nor should the configuration of the components be interpreted as having any dependency or requirement on any one or combination of components shown in the exemplary embodiment of the computer system (1300).

[0117] The computer system (1300) may include certain human-computer interface input devices. The human-computer interface input devices may include one or more of the following (only one of which is depicted): keyboard (1301), mouse (1302), touch pad (1303), touch screen (1310), data gloves (not shown), joystick (1305), microphone (1306), scanner (1307), camera (1308).

[0118] The computer system (1300) may also include certain human-computer interface output devices. Such human-computer interface output devices may stimulate one or more human user senses through, for example, tactile output, sound, light, and smell / taste. Such human-computer interface output devices may include tactile output devices (e.g., tactile feedback through a touch screen (1310), a data glove (not shown), or a joystick (1305), but there may also be tactile feedback devices that are not used as input devices), audio output devices (e.g., speakers (1309), headphones (not shown)), visual output devices (e.g., screens (1310) including cathode ray tube screens, liquid crystal screens, plasma screens, organic light emitting diode screens, each of which has or does not have a touch screen input function, each of which has or does not have a tactile feedback function - some of which may output two-dimensional visual output or output of more than three dimensions through means such as stereoscopic image output; virtual reality glasses (not shown), holographic displays, and smoke boxes (not shown)) and printers (not shown).

[0119] The computer system (1300) may also include human-accessible storage devices and their associated media, such as optical media including high-density read-only / rewritable optical disks (CD / DVD ROM / RW) (1320) with CD / DVD or similar media (1321), thumb drives (1322), removable hard disk drives or solid state drives (1323), traditional magnetic media such as tapes and floppy disks (not shown), special-purpose ROM / ASIC / PLD-based devices such as security software protectors (not shown), and the like.

[0120] Those skilled in the art should also understand that the term "computer-readable media" used in connection with the disclosed subject matter does not include transmission media, carrier waves, or other transient signals.

[0121] The computer system (1300) may also include an interface (1354) to one or more communication networks (1355). For example, the network may be wireless, wired, or optical. The network may also be a local area network, a wide area network, a metropolitan area network, an in-vehicle network, an industrial network, a real-time network, a delay-tolerant network, and the like. The network also includes local area networks such as Ethernet, wireless local area networks, cellular networks (GSM, 3G, 4G, 5G, LTE, etc.), television wired or wireless wide area digital networks (including cable television, satellite television, and terrestrial broadcast television), in-vehicle and industrial networks (including CANBus), and the like.

[0122] The above-mentioned human-machine interface devices, human-accessible storage devices, and network interfaces may be connected to the core (1340) of the computer system (1300).

[0123] The core (1340) may include one or more central processing units (CPUs) (1341), graphics processing units (GPUs) (1342), dedicated programmable processing units in the form of field programmable gate arrays (FPGAs) (1343), hardware accelerators for specific tasks (1344), graphics adapters (1350), etc. These devices, as well as read-only memory (ROM) (1345), random access memory (1346), internal mass storage (e.g., internal non-user accessible hard disk drives, solid-state drives, etc.) (1347), etc., may be connected via a system bus (1348). In some computer systems, the system bus (1348) may be accessed in the form of one or more physical plugs so that it may be expanded by additional central processing units, graphics processing units, etc. Peripheral devices may be directly attached to the system bus (1348) of the core, or connected via a peripheral bus (1349). In an embodiment, the screen (1310) may be connected to the graphics adapter (1350). The architecture of the peripheral bus includes the external peripheral component interconnect PCI, the universal serial bus USB, etc.

[0124] The computer readable medium may have computer code thereon for performing various computer-implemented operations. The medium and computer code may be specially designed and constructed for the purposes of the present disclosure, or may be the medium and code well known and available to those skilled in the art of computer software.

[0125] Although the present disclosure has described a number of exemplary embodiments, various changes, arrangements and various equivalent substitutions of the embodiments are within the scope of the present disclosure. It should therefore be understood that those skilled in the art are capable of designing a variety of systems and methods, which, although not explicitly shown or described herein, embody the principles of the present disclosure and are therefore within the spirit and scope of the present disclosure.

Claims

1. A method for decoding a block in a video bitstream, characterized in that The method comprises: Determining, based on a syntax element in the video bitstream, to apply intra warp prediction to the block; determining a set of weighting coefficients to be applied to a set of basis functions used in intra warp prediction; predicting the value of the sample based on a weighted sum of the basis function set at the pixel coordinate position of the sample of the block, wherein the basis function set is weighted according to the weighting coefficient set respectively; and The block including the samples is reconstructed based on the prediction values.

2. The method according to claim 1, characterized in that The set of basis functions includes power functions of the pixel coordinate positions of the samples up to order N, N being an integer; and The weighted sum of the basis functions comprises polynomials of the pixel coordinate positions of the samples up to order N.

3. The method according to claim 2, characterized in that N=2, and the weight coefficient set includes: 6 coefficients for the two-dimensional polynomial terms for the horizontal and vertical pixel coordinates of the sample; 5 coefficients for the two-dimensional polynomial terms for the horizontal and vertical pixel coordinates of the sample, excluding one cross term; or 4 coefficients for two-dimensional polynomial terms other than a cross term that depend on the horizontal and vertical pixel coordinates of the sample and further include at least one term that is independent of the coordinates.

4. The method according to claim 3, characterized in that The at least one coordinate-independent term comprises an offset term comprising a DC value derived from a reconstructed neighbor of the sample.

5. The method according to claim 3, characterized in that: The set of weighting coefficients further comprises an offset coefficient, and the at least one coordinate-independent term comprises an offset term, the offset term being a product of the offset coefficient and a reconstructed value of an upper left corner sample of the block; or The set of weighting coefficients further includes a first offset coefficient and a second offset coefficient, and the at least one coordinate-independent term includes a first offset term and a second offset term, wherein the first offset term is a product of the first offset coefficient and a top reconstructed neighbor of the sample, and the second offset term is a product of the second offset coefficient and a left reconstructed neighbor of the sample; or The set of weighted coefficients further includes a first offset coefficient, a second offset coefficient, a third offset coefficient and a fourth offset coefficient, and the at least one coordinate-independent item includes a first offset item, a second offset item, a third offset item and a fourth offset item, wherein the first offset item is the product of the first offset coefficient and the top reconstructed neighbor of the sample, the second offset item is the product of the second offset coefficient and the left reconstructed neighbor of the sample, the third offset item is the product of the third offset coefficient and the upper right reconstructed neighbor of the sample, and the fourth offset item is the product of the fourth offset coefficient and the lower left reconstructed neighbor of the sample.

6. The method according to claim 2, characterized in that N=2, and the weighted sum of the basis functions includes: p(x,y) = a·x 2 + b·y 2 + c·x·y + d·x + e·y + f; or p(x,y)=a·x 2 +b·y 2 +d·x+e·y+f, Wherein a, b, c, d, e and f represent polynomial coefficients, (x, y) represent sample coordinates, and p(x, y) represents the weighted sum.

7. The method according to any one of claims 1 to 6, characterized in that The set of basis functions includes up to N triangular basis functions of the pixel coordinate position of the sample, where N is an integer, and wherein the N triangular basis functions include: The first N type-2 discrete cosine transform bases; The first N discrete sine transform bases of type 7; or Any N discrete sine transform bases of type 1 to 8 and discrete cosine transform bases of type 1 to 8.

8. The method according to any one of claims 1 to 6, characterized in that The set of basis functions includes up to N Karhunen Lòeve Transform (KLT) bases as a function of the pixel coordinate position of the sample, N being an integer.

9. The method according to claim 8, characterized in that The N KLT bases include eigenvectors of the covariance matrix derived from the reconstructed neighborhood of the sample, or are predefined eigenvectors known to both the encoder and the decoder.

10. The method according to any one of claims 1 to 6, characterized in that The basis function set includes a subset of power functions and a subset of trigonometric functions of the pixel coordinate positions of the samples.

11. The method according to claim 1, characterized in that: Further including: The video bitstream is parsed to determine the set of weighting coefficients signaled in the video bitstream.

12. The method according to any one of claims 1 to 6, characterized in that The set of weighting coefficients is derived by the encoder using multiple rows of templates above and to the left of the block.

13. The method according to any one of claims 1 to 6, characterized in that Parsing the video bitstream to determine the set of weighting coefficients includes: Parsing the video bitstream to obtain an index of the weighting coefficient set from the video bitstream; and The weighting coefficient set is identified among a plurality of weighting coefficient sets according to the index.

14. The method according to any one of claims 1 to 6, characterized in that The set of weighting coefficients is clipped to a predefined range of values.

15. The method according to any one of claims 1 to 6, characterized in that The predicted value of the sample is clipped to a predefined value range, and the clipped minimum value and the clipped maximum value of the predicted value are determined by the internal bit depth range.

16. The method according to any one of claims 1 to 6, characterized in that The set of weighting coefficients is quantized according to one or more predefined precisions.

17. The method according to claim 16, characterized in that At least two of the sets of weighting coefficients are quantized to different precisions.

18. The method according to claim 16, characterized in that The terms in the weighted sum of the set of basis functions associated with the same precision are computed together and rounded.

19. The method according to claim 16, characterized in that The precision of the set of weighting coefficients is signaled in the video bitstream using a high-level syntax.

20. An electronic device, comprising: a memory for storing instructions, and a processor for executing the instructions to: Determining, based on a syntax element in the video bitstream, to apply intra warp prediction to the block; determining a set of weighting coefficients to be applied to a set of basis functions used in intra warp prediction; predicting the value of the sample based on a weighted sum of the basis function set at the pixel coordinate position of the sample of the block, wherein the basis function set is weighted according to the weighting coefficient set respectively; and The block including the samples is reconstructed based on the prediction values.

Images