Method and apparatus for video encoding and decoding, storage medium and computer equipment

Abstract

An improved scheme for intra-inter prediction mode and adaptive intra-inter prediction mode for video encoding / decoding, comprising: selecting a weighted set in intra-inter mode, and applying a position-dependent intra prediction combination to modify at least one of an inter-prediction sample and a reconstructed sample of an inter-coded coding unit.

Description

[0001] This application claims priority to U.S. Provisional Application No. 62 / 767,228 filed on November 14, 2018 and U.S. Provisional Application No. 62 / 788,056 filed on January 3, 2019, and U.S. Application No. 16 / 673,097 filed on November 4, 2019. The entire contents of each of the above applications are hereby expressly incorporated by reference into this application.

[0002] This application is a divisional application for the Chinese patent application with application number 201980025045.5, application date November 12, 2019, and invention name “Method and device, storage medium and computer device for video encoding and decoding”. Technical Field

[0003] The present disclosure relates to video processing, and more particularly to a method and apparatus, a storage medium, and a computer device for video encoding and decoding. Background Art

[0004] ITU-T VCEG (Q6 / 16) and ISO / IEC MPEG (JTC 1 / SC 29 / WG 11) published the H.265 / HEVC (High Efficiency Video Coding) standard in 2013 (version 1), 2014 (version 2), 2015 (version 3), and 2016 (version 4). In 2015, the two standards organizations jointly formed the JVET (Joint Video Exploration Team, JVET) to explore the potential of developing the next video coding standard beyond HEVC. In October 2017, the two standards organizations issued a joint call for proposal (CfP) on video compression with capabilities beyond HEVC. As of February 15, 2018, a total of 22 CfP responses were submitted for standard dynamic range (SDR), 12 CfP responses for high dynamic range (HDR), and 12 CfP responses for 360 video categories. In April 2018, all received CfP responses were evaluated in the 122MPEG / 10th JVET meeting. As a result of this meeting, JVET formally launched the standardization process for the next generation of video coding beyond HEVC. The new standard was named Versatile Video Coding (VVC), and JVET was renamed the Joint Video Experts Group.

[0005] The following clause from VVC draft 2 describes PDPC, where nScale is used to specify the weighting factor decrement rate, and "32" in equations 8-77, 8-78, 8-82, and 8-83 specifies the initial weighting factor.

[0006] The input to the position-dependent intra prediction combination process can be

[0007] - Intra prediction mode predModelntra,

[0008] - variable nTbW specifying the transform block width,

[0009] - variable nTbH specifying the transform block height,

[0010] - variable refW specifying the reference sample width,

[0011] - variable refH specifying the reference sample height,

[0012] - prediction samples predSamples[x][y], where x=0..nTbW-1 and y=0..nTbH-1,

[0013] - neighboring samples p[x][y], where x=-1, y=-1..refH-1 and x=0..refW-1, y=-1,

[0014] - Variable cIdx that specifies the color component of the current block.

[0015] The output of this process is the modified prediction samples predSamples[x][y], where x=0..nTbW-1, y=0..nTbH-1.

[0016] According to the value of cIdx, set the function clip1Cmp as follows:

[0017] -If cIdx is equal to 0, set clip1Cmp equal to Clip1Y.

[0018] - Otherwise, set clip1Cmp equal to Clip1C.

[0019] Set the variable nScale to ((Log2(nTbW)+Log2(nTbH)-2)>>2).

[0020] The reference sample arrays mainRef[x] and sideRef[y], where x=0..refW and y=0..refH, are derived as follows:

[0021] mainRef[x]=p[x][-1]

[0022] sideRef[y]=p[-1][y]

[0023] The variables refL[x][y], refT[x][y], wT[y], wL[x] and wTL[x][y], where x=0..nTbW-1 and y=0..nTbH-1, are derived as follows:

[0024] - If predModeIntra is equal to INTRA_PLANAR, INTRA_DC, INTRA_ANGULAR18, or INTRA_ANGULAR50, the following applies:

[0025] refL[x][y]=p[-1][y]

[0026] refT[x][y]=p[x][-1]

[0027] wT[y]=32>>((y<<1)>>nScale)

[0028] wL[x]=32>>((x<<1)>>nScale)

[0029] wTL[x][y]=(predModeIntra==INTRA_DC)? ((wL[x]>>4)+(wT[y]>>4)): 0

[0030] Otherwise, if predModeIntra is equal to INTRA_ANGULAR2 or INTRA_ANGULAR66, the following applies:

[0031] refL[x][y]=p[-1][x+y+1]

[0032] refT[x][y]=p[x+y+1][-1]

[0033] wT[y]=(32>>1)>>((y<<1)>>nScale)

[0034] wL[x]=(32>>1)>>((x<<1)>>nScale)

[0035] wTL[x][y]=0

[0036] Otherwise, if predModeIntra is less than or equal to INTRA_ANGULAR10, the following sequence of steps applies:

[0037] - The variables dXPos[y], dXFrac[y], dXInt[y] and dX[y] are derived from intraPredMode using invAngle as specified in clause 8.2.4.2.7 as follows:

[0038] dXPos[y]=((y+1)*invAngle+2)>>2

[0039] dXFrac[y]=dXPos[y]&63

[0040] dXInt[y]=dXPos[y]>>6

[0041] dX[y]=x+dXInt[y]

[0042] The variables refL[x][y], refT[x][y], wT[y], wL[x] and wTL[x][y] are derived as follows:

[0043] refL[x][y]=0 (8 86)

[0044] refT[x][y]=(dX[y]<refW-1)?

[0045] ((64-dXFrac[y])*mainRef[dX[y]]+dXFrac[y]*mainRef[dX[y]+1]+32)>>6(8 87) :0

[0047] wT[y]=(dX[y]<refW-1)? 32>>((y<<1)>>nScale): 0 (8 88)

[0048] wL[x]=0 (8 89)

[0049] wTL[x][y]=0

[0050] Otherwise, if predModeIntra is greater than or equal to INTRA_ANGULAR58, the following sequence of steps applies:

[0051] - The variables dYPos[x], dYFrac[x], dYInt[x] and dY[x] are derived from intraPredMode using invAngle as specified in clause 8.2.4.2.7 as follows:

[0052] dYPos[x]=((x+1)*invAngle+2)>>2

[0053] dYFrac[x]=dYPos[x]&63

[0054] dYInt[x]=dYPos[x]>>6

[0055] dY[x]=x+dYInt[x]

[0056] - The variables refL[x][y], refT[x][y], wT[y], wL[x] and wTL[x][y] are derived as follows:

[0057] refL[x][y]=(dY[x]<refH-1)?

[0058] ((64-dYFrac[x])*sideRef[dY[x]]+

[0059] dYFrac[x]*sideRef[dY[x]+1]+32)>>6 :0

[0061] refT[x][y]=0

[0062] wT[y]=0

[0063] wL[x]=(dY[x]<refH-1)? 32>>((x<<1)>>nScale): 0

[0064] wTL[x][y]=0

[0065] Otherwise, set refL[x][y], refT[x][y], wT[y], wL[x], and wTL[x][y] to zero.

[0066] The values ​​of the filtered samples filtSamples[x][y], where x=0..nTbW-1 and y=0..nTbH-1, are obtained as follows:

[0067] filtSamples[x][y]=clip1Cmp((refL[x][y]*wL+refT[x][y]*wT-p[-1][-1]*wTL[x][y]+

[0068] (64-wL[x]-wT[y]+wTL[x][y])*predSamples[x][y]+32)>>6)

[0069] For the chroma component of the intra-coded block, the encoder selects the best chroma prediction mode from five modes including plane (mode index 0), DC (mode index 1), horizontal (mode index 18), vertical (mode index 50), diagonal (mode index 66), and selects a direct copy of the intra prediction mode for the associated luminance component (ie, DM mode). The mapping between the number of intra prediction modes for chroma and the intra prediction direction is shown in Table 1:

[0070] Table 1 - Mapping between chroma intra prediction modes and intra prediction directions

[0071]

[0072] In order to avoid repeated modes, four modes other than DM are allocated according to the intra prediction mode of the associated luma component. When the number of intra prediction modes of the chroma component is 4, the intra prediction direction of the luma component is used for intra prediction sample generation of the chroma component. When the number of intra prediction modes of the chroma component is not 4 and it is the same as the number of intra prediction modes of the luma component, 66 intra prediction directions are used for intra prediction sample generation of the chroma component.

[0073] Merge mode for inter-picture prediction is used to indicate to a block that motion data is inferred rather than explicitly signaled. A merge candidate list of candidate motion parameters is first constructed, and then an index identifying the candidate to use is signaled.

[0074] The merge candidate list includes a non-sub-CU merge candidate list and a sub-CU merge candidate list. Non-sub-CU merge candidates are constructed based on spatially adjacent motion vectors, collocated temporal motion vectors, and history-based motion vectors. The sub-CU merge candidate list includes affine merge candidates and ATMVP merge candidates. Sub-CU merge candidates are used to derive multiple MVs for the current CU, and different parts of the samples in the current CU can have different motion vectors.

[0075] Skip mode is used to indicate for a block that motion data is inferred rather than explicitly signaled and that the prediction residual is zero, i.e., no transform coefficients are transmitted. At the beginning of each CU in an inter-picture predicted slice, a skip_flag is signaled, which implies the following:

[0076] Use merge mode to get motion data

[0077] No residual data exists in the bitstream

[0078] Multi-hypothesis intra-inter prediction combines one intra prediction and one merge index prediction, i.e., intra-inter prediction mode. In the merged CU, a flag is signaled for the merge mode to select the intra mode from the intra candidate list when the flag is true. For the luminance component, the intra candidate list is derived from 4 intra prediction modes including DC mode, planar mode, horizontal mode and vertical mode, and the size of the intra candidate list can be 3 or 4 depending on the block shape. When the CU width is greater than twice the CU height, the horizontal mode is removed from the intra mode list, and when the CU height is greater than twice the CU width, the vertical mode is removed from the intra mode list. Weighted averaging is used to combine one intra prediction mode selected by the intra mode index and one merge index prediction selected by the merge index. For chrominance components, DM is always applied without additional signaling.

[0079] The weights used for combined prediction are described as follows. When DC mode or planar mode is selected or the coding block (CodingBlock, CB) width or height is less than 4, equal weights are applied. For those CBs whose CB width and height are greater than or equal to 4, when the horizontal / vertical mode is selected, one CB is first divided vertically / horizontally into four equal-area regions. Each weight set represented as (w_intrai, w_interi) will be applied to the corresponding region, where i is 1 to 4 and (w_intra1, w_inter1) = (6, 2), (w_intra2, w_inter2) = (5, 3), (w_intra3, w_inter3) = (3, 5) and (w_intra4, w_inter4) = (2, 6). (w_intra1, w_inter1) is used for the region closest to the reference sample and (w_intra4, w_inter4) is used for the region farthest from the reference sample. Then, the combined prediction can be calculated by adding the two weighted predictions and shifting right by 3 bits. Furthermore, if the following neighboring CBs are intra-coded, the intra prediction mode of the intra hypothesis used for the predictor may be saved for their intra-mode coding.

[0080] Currently, for the luma component, for the intra-inter prediction mode, up to 4 intra prediction modes are adopted, which introduces too much complexity.

[0081] Currently, for intra-inter mode, one buffer is used to store inter prediction values, and an additional buffer is added to store intra prediction values, which increases the decoder memory cost.

[0082] For the current intra-inter mode, both intra prediction and inter prediction need to be performed, and multiplication is involved in the intra prediction process, which adds extra complexity to the decoder.

[0083] For the current Intra-Inter mode, the weights are fixed values ​​for different block sizes, which may not be optimal and can be improved.

[0084] The weights (w_intra, w_inter) applied in the multiple hypotheses are chosen based on the relative positions of the samples within the block. However, this choice may not be very reliable or flexible.

[0085] Therefore, there is a need for technical solutions to the above problems. Summary of the invention

[0086] A method and an apparatus are included, the apparatus comprising: a memory configured to store a computer program code; and one or more hardware processors, the hardware processor or processors configured to access the computer program code and operate as instructed by the computer program code. The computer program includes: a selection code configured to cause the processor to select a weighting set in an intra-inter mode; and an application code configured to cause the processor to apply a position-dependent intra prediction combination (PDPC) to modify at least one of an inter-prediction sample and a reconstructed sample of an inter-coded coding unit (CU).

[0087] According to an exemplary embodiment, the application code is further configured to cause the processor to apply the PDPC including modifying only the luma component of the inter-coded CU.

[0088] According to an exemplary embodiment, the signaling code is further configured to cause the processor to signal an inter-position dependent intra prediction combination flag interPDPCFlag in response to determining that the coding block (CB) is encoded by the merge mode.

[0089] According to an exemplary embodiment, the application code is further configured to cause the processor to apply PDPC to the plurality of reconstructed samples before applying any of an in-loop deblocking filter including deblocking, sample adaptive offset (SAO), and an adaptive loop filter (ALF).

[0090] According to an exemplary embodiment, the program code also includes an assignment code, which is configured to cause the processor to assign a default intra-frame prediction mode to a current coding block (CB), and the application code is also configured to cause the processor to apply the default intra-frame prediction mode to a CB following the current CB for at least intra-frame mode encoding and most probable mode (MPM) derivation.

[0091] According to an exemplary embodiment, the application code is also configured to cause the processor to apply a PDPC filter set to at least one of inter-frame prediction samples other than the inter-frame prediction samples and reconstructed samples of the inter-frame coded CU other than the reconstructed samples of the inter-frame coded CU, each PDPC filter in the PDPC filter set being defined for a corresponding block size of multiple block sizes.

[0092] According to an exemplary embodiment, the signaling code is also configured to cause the processor to perform the following operations: signaling at least one interPDPCFlag indicating whether PDPC is applied; determining whether interPDPCFlag is set to be equal to true; in response to determining that interPDPCFlag is set to be equal to a first value, signaling a second flag indicating that at least one PDPC filter is to be applied to at least one of the inter-frame predicted samples and the reconstructed samples of the inter-frame coded CU after inter-frame prediction; and in response to determining that interPDPCFlag is set to be equal to the second value, determining to exclude PDPC from at least one of the inter-frame predicted samples and the reconstructed samples of the inter-frame coded CU.

[0093] According to an exemplary embodiment, the program code also includes: a signaling code configured to cause the processor to select whether the weights in the weighting set are equal to each other; and a weighting code configured to cause the processor to weight inter-frame prediction in PDPC filtering according to whether the adjacent blocks are intra-frame encoded, inter-frame encoded, and intra-frame-inter-frame encoded in response to a determination signal indicating that the weights are unequal.

[0094] According to an exemplary embodiment, the intra-inter mode has no intra prediction, and the intra-inter mode includes applying PDPC directly on top of inter prediction, and applying PDPC directly on top of inter prediction includes:

[0095] pred(x,y)=(w_intra*wL×R-1,y+w_intra*wT×Rx,-1-w_intra*wTL×R-1,-1+(N-w_intra*wL-w_intra*wT+w_intra*wTL)×InterPred(x,y)+N / 2)>>log2(N)

[0096] Wherein InterPred indicates input inter-frame prediction, w_intra is selected from a predefined value set according to whether the neighboring block is at least one of intra-frame coded, inter-frame coded, and intra-inter-frame coded, and wherein N indicates one of 8, 16, 32, 64, 128 and 256.

[0097] It can be seen that when performing video decoding, a weighting set is selected in the intra-inter mode; and PDPC is applied to modify at least one of the inter-prediction samples and the reconstructed samples of the inter-coded CU. Thus, the weighting (w_intra, w_inter) applied in multiple hypotheses can be correlated with whether the neighboring block is intra-coded or inter-coded. For example, by considering whether the upper neighboring block or the left neighboring block is intra-coded or inter-coded, the weighting can be selected more efficiently. BRIEF DESCRIPTION OF THE DRAWINGS

[0098] Additional features, properties, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings, in which:

[0099] Figure 1 is a schematic diagram of a simplified block diagram of a communication system according to an embodiment;

[0100] Figure 2 is a schematic diagram of a streaming system according to an embodiment;

[0101] Figure 3 is a schematic diagram of a functional block diagram of a video decoder according to an embodiment;

[0102] Figure 4 is a functional block diagram of a video encoder according to an embodiment;

[0103] Figure 5 It is a schematic diagram of the intra prediction mode used in HEVC and JEM;

[0104] Figure 6 It is a schematic diagram of the DC mode PDPC weighting the (0,0) position and the (1,0) position within a 4×4 block;

[0105] Fig. 7A is a schematic diagram of the intra prediction mode used in HEVC;

[0106] Figure 7B is a schematic diagram of an intra prediction mode in VVC in an implementation manner;

[0107] Figure 8 is a flow chart according to an embodiment;

[0108] Fig. 9 is a flow chart according to another embodiment;

[0109] Fig.10 is a flow chart according to another embodiment;

[0110] Fig.11 is a flow chart according to another embodiment;

[0111] Fig.12 is a schematic diagram of a computer system according to one embodiment. DETAILED DESCRIPTION

[0112] The proposed features discussed below may be used alone or in any order. In addition, embodiments may be implemented by processing circuits (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored in a non-transitory computer-readable medium. In the present disclosure, the most probable mode (MPM) may refer to a primary MPM, a secondary MPM, or both a primary MPM and a secondary MPM.

[0113] Figure 1 A simplified block diagram of a communication system 100 according to an embodiment of the present disclosure is shown. The communication system 100 may include at least two terminals 102 and 103 interconnected via a network 105. For unidirectional transmission of data, a first terminal 103 may encode video data at a local location for transmission to another terminal 102 via the network 105. The second terminal 102 may receive the encoded video data of the other terminal from the network 105, decode the encoded data and display the recovered video data. Unidirectional data transmission may be common in media service applications and the like.

[0114] Figure 1 A second pair of terminals 101 and 104 is shown, which are provided to support bidirectional transmission of encoded video, such as may occur during a video conference. For bidirectional transmission of data, each terminal 101 and 104 can encode video data captured at a local location for transmission to the other terminal via a network 105. Each terminal 101 and 104 can also receive encoded video data transmitted by the other terminal, can decode the encoded data, and can display the recovered video data at a local display device.

[0115] exist Figure 1In the embodiment of the present disclosure, terminals 101, 102, 103 and 104 can be illustrated as servers, personal computers and smart phones, but the principles of the present disclosure are not limited thereto. The embodiments of the present disclosure are applicable to laptop computers, tablet computers, media players and / or dedicated video conferencing equipment. Network 105 represents any number of networks that transmit encoded video data between terminals 101, 102, 103 and 104, including, for example, wired and / or wireless communication networks. Communication network 105 can exchange data in circuit switching and / or packet switching channels. Representative networks include telecommunication networks, local area networks, wide area networks and / or the Internet. For the purpose of this discussion, unless explained below, the architecture and topology of network 105 may be insignificant for the operation of the present disclosure.

[0116] As examples of applications of the disclosed subject matter, Figure 2 The placement of the video encoder and video decoder in a streaming environment is shown. The disclosed subject matter can be equally applicable to other video-enabled applications, including, for example, video conferencing, digital TV, storing compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0117] The streaming system may include an acquisition subsystem 203, which may include a video source 201, such as a digital camera, that creates, for example, an uncompressed video sample stream 213. The sample stream 213, depicted as a thick line to emphasize the high amount of data compared to the encoded video bitstream, may be processed by an encoder 202 coupled to the camera 201. The encoder 202 may include hardware, software, or a combination thereof to implement or implement various aspects of the disclosed subject matter as described in more detail below. The encoded video bitstream 204, depicted as a thin line to emphasize the lower amount of data compared to the sample stream, may be stored on a streaming server 205 for future use. One or more streaming clients 212 and 207 may access the streaming server 205 to retrieve copies 208 and 206 of the encoded video bitstream 204. The client 212 may include a video decoder 211 that decodes an incoming copy 208 of the encoded video bitstream and creates an output video sample stream 210 that may be presented on a display 209 or other presentation device (not depicted). In some streaming systems, the video bitstreams 204, 206, and 208 may be encoded according to certain video encoding / compression standards. Examples of these standards are mentioned above and are further described herein.

[0118] Figure 3 It may be a functional block diagram of the video decoder 300 according to an embodiment of the present invention.

[0119] Receiver 302 may receive one or more codec video sequences to be decoded by decoder 300; in the same or another embodiment, one coded video sequence is received at a time, wherein the decoding of each coded video sequence is independent of the other coded video sequences. The coded video sequence may be received from channel 301, which may be a hardware / software link to a storage device storing the coded video data. Receiver 302 may receive the coded video data as well as other data, such as coded audio data and / or auxiliary data streams that may be forwarded to their respective use entities (not depicted). Receiver 302 may separate the coded video sequence from the other data. To prevent network jitter, buffer memory 303 may be coupled between receiver 302 and entropy decoder / parser 304 (hereinafter referred to as "parser"). Buffer memory 303 may not be needed or may be small when receiver 302 receives data from a storage / forward device with sufficient bandwidth and controllability or from an isochronous network. For use over a traffic packet network such as the Internet, a buffer memory 303 may be required, which may be relatively large and may advantageously have an adaptive size.

[0120] The video decoder 300 may include a parser 304 to reconstruct symbols 313 from an entropy coded video sequence. The categories of these symbols include information for managing the operation of the decoder 300, and potentially information for controlling a rendering device such as a display 312, which is not an integral part of the decoder but may be coupled to the decoder. The control information for the rendering device may be in the form of a supplementary enhancement information (SEI (Supplemental Enhancement Information, SEI) message) or a video usability information (VUI) parameter set fragment (not depicted). The parser 304 may parse / entropy decode the received coded video sequence. The encoding of the coded video sequence may be performed according to a video coding technique or standard, and may follow principles known to those skilled in the art, including: variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser 304 may extract a subgroup parameter set for at least one of the subgroups of pixels in the video decoder from the coded video sequence based on at least one parameter corresponding to the group. Subgroups may include: Groups of Picture (GOP), pictures, tiles, slices, macroblocks, Coding Units (CU), blocks, Transform Units (TU), Prediction Units (PU), etc. The entropy decoder / parser may also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, etc.

[0121] The parser 304 may perform an entropy decoding / parsing operation on the video sequence received from the buffer memory 303, thereby creating the symbol 313. The parser 304 may receive the encoded data and selectively decode a specific symbol 313. In addition, the parser 304 may determine whether the specific symbol 313 is to be provided to the motion compensation prediction unit 306, the scaler / inverse transform unit 305, the intra prediction unit 307, or the loop filter 311.

[0122] Depending on the type of the coded video picture or part of the coded video picture (e.g., inter-frame and intra-frame pictures, inter-frame blocks and intra-frame blocks) and other factors, the reconstruction of the symbol 313 may involve multiple different units. Which units are involved and how they are involved can be controlled by subgroup control information parsed from the coded video sequence by the parser 304. For the sake of brevity, such subgroup control information flow between the parser 304 and the following multiple units is not described.

[0123] In addition to the functional blocks already mentioned, the video decoder 300 can be conceptually subdivided into several functional units as described below. In actual implementations operating under commercial constraints, many of these units interact closely with each other and may be at least partially integrated with each other. However, for the purpose of describing the disclosed subject matter, the conceptual subdivision into the following functional units is appropriate.

[0124] The first unit is the sealer / inverse transform unit 305. The sealer / inverse transform unit 305 receives the quantized transform coefficients as symbols 313 from the parser 304 as well as control information including which transform method to use, block size, quantization factor, quantization scaling matrix, etc. The sealer / inverse transform unit 305 may output a block including sample values ​​that may be input into the aggregator 310.

[0125] In some cases, the output samples of the scaler / inverse transform 305 may belong to an intra-coded block; that is, predictive information from a previously reconstructed picture is not used, but predictive information from a previously reconstructed portion of the current picture may be used. Such predictive information may be provided by the intra-picture prediction unit 307. In some cases, the intra-picture prediction unit 307 uses surrounding already reconstructed information extracted from the current (partially reconstructed) picture 309 to generate a block of the same size and shape as the block being reconstructed. In some cases, the aggregator 310 adds the prediction information already generated by the intra-prediction unit 307 to the output sample information provided by the scaler / inverse transform unit 305 on a per-sample basis.

[0126] In other cases, the output samples of the scaler / inverse transform unit 305 may belong to an inter-coded and potentially motion compensated block. In such a case, the motion compensated prediction unit 306 may access the reference picture buffer 308 to extract samples for prediction. After the extracted samples are motion compensated according to the symbols 313 belonging to the block, these samples may be added to the output of the scaler / inverse transform unit (in this case referred to as residual samples or residual signals) by the aggregator 310, thereby generating output sample information. The address within the reference picture memory from which the motion compensated prediction unit extracts the predicted samples may be controlled by a motion vector, which is provided to the motion compensation unit in the form of a symbol 313, which may have, for example, X, Y and reference picture components. Motion compensation may also include interpolation of sample values ​​extracted from the reference picture memory, motion vector prediction mechanisms, etc. when using sub-sample accurate motion vectors.

[0127] The output samples of aggregator 310 may be employed by various loop filtering techniques in loop filter unit 311. The video compression techniques may include in-loop filter techniques controlled by parameters included in the coded video bitstream and available to loop filter unit 311 as symbols 313 from parser 304, however the video compression techniques may also be responsive to meta-information obtained during decoding of a previous (in decoding order) portion of a coded picture or coded video sequence, and to previously reconstructed and loop filtered sample values.

[0128] The output of the loop filter unit 311 may be a sample stream that may be output to a rendering device 312 as well as stored in the reference picture memory 308 for use in future inter-picture prediction.

[0129] Once fully reconstructed, certain coded pictures may be used as reference pictures for future prediction. Once a coded picture is fully reconstructed and the coded picture is identified as a reference picture (e.g., by parser 304), current reference picture 309 may become part of reference picture buffer 308, and new current picture memory may be reallocated before starting reconstruction of a subsequent coded picture.

[0130] The video decoder 300 can perform decoding operations according to a predetermined video compression technology that can be recorded in a standard such as ITU-T H.265 Recommendation. The encoded video sequence can conform to the syntax specified by the video compression technology or standard being used in the sense that the encoded video sequence follows the syntax of the video compression technology or standard specified in the video compression technology document or standard and specifically in the configuration document therein. For compliance, the complexity of the encoded video sequence is also required to be within the range defined by the hierarchy of the video compression technology or standard. In some cases, the hierarchy limits the maximum picture size, the maximum frame rate, the maximum reconstruction sample rate (measured in, for example, megasamples per second), the maximum reference picture size, etc. In some cases, the limits set by the hierarchy can be further defined by the Hypothetical Reference Decoder (HRD) specification and the metadata of the HRD buffer management signaled in the encoded video sequence.

[0131] In an embodiment, the receiver 302 may receive additional (redundant) data along with the encoded video. The additional data may be included as part of the encoded video sequence. The additional data may be used by the video decoder 300 to correctly decode the data and / or more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0132] Figure 4 It may be a functional block diagram of a video encoder 400 according to an embodiment of the present disclosure.

[0133] The encoder 400 may receive video samples from a video source 401 (which is not part of the encoder) that may capture video images to be encoded by the encoder 400 .

[0134] The video source 401 may provide a source video sequence in the form of a digital video sample stream to be encoded by a source encoder (403), which may have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, ...), any color space (e.g., BT.601 Y CrCb, RGB, ...), and any suitable sampling structure (e.g., Y CrCb 4:2:0, Y CrCb 4:4:4). In a media service system, the video source 401 may be a storage device storing previously prepared videos. In a video conferencing system, the video source 401 may be a camera that collects local image information as a video sequence. The video data may be provided as a plurality of separate pictures that are given motion when viewed in sequence. The pictures themselves may be organized as a spatial pixel array, wherein each pixel may include one or more samples, depending on the sampling structure, color space, etc. used. The relationship between pixels and samples may be easily understood by those skilled in the art. The following description focuses on samples.

[0135] According to an embodiment, the encoder 400 can encode and compress the pictures of the source video sequence into an encoded video sequence 410 in real time or under any other time constraints required by the application. It is a function of the controller 402 to implement the appropriate encoding speed. The controller controls other functional units described below and is functionally coupled to these units. For the sake of brevity, the coupling is not depicted. The parameters set by the controller may include: rate control related parameters (picture skipping, quantizer, lambda value of rate distortion optimization technology, ...), picture size, picture group (group of picture, GOP) layout, maximum motion vector search range, etc. Those skilled in the art can easily identify other functions of the controller 402, as they may belong to the video encoder 400 optimized for a specific system design.

[0136] Some video encoders operate in a manner that is readily recognizable to those skilled in the art as a "coding loop". As a very simplified description, the coding loop may include an encoding portion of an encoder 403 (hereinafter referred to as a "source encoder"), which is responsible for creating symbols based on the input picture to be encoded and the reference picture, and a (local) decoder 406 embedded in the encoder 400, which reconstructs the symbols to create sample data that the (remote) decoder will also create (because in the video compression techniques considered in the disclosed subject matter, any compression between the symbols and the encoded video bitstream is lossless). The reconstructed sample stream is input to a reference picture memory 405. Since the decoding of the symbol stream produces bit-accurate results that are independent of the decoder location (local or remote), the reference picture buffer contents are also bit-accurate between the local encoder and the remote encoder. In other words, the reference picture samples "seen" by the prediction portion of the encoder are exactly the same sample values ​​that the decoder will "see" when using prediction during decoding. This basic principle of reference picture synchronization (and the drift that results when synchronization cannot be maintained, for example due to channel errors) is well known to those skilled in the art.

[0137] The operation of the "local" decoder 406 can be combined with the above Figure 3 The operation of the "remote" decoder 300 described in detail is the same. However, reference is also briefly made to Figure 4 , when symbols are available and can be losslessly encoded / decoded into an encoded video sequence by the entropy encoder 408 and the parser 304, the entropy decoding portion of the decoder 300 including the channel 301, the receiver 302, the buffer memory 303 and the parser 304 may not be fully implemented in the local decoder 406.

[0138] At this point it can be observed that any decoder techniques other than parsing / entropy decoding present in the decoder must also be present in the corresponding encoder in substantially the same functional form. The description of encoder techniques can be simplified since they are inverse to the decoder techniques described comprehensively. A more detailed description is only required in certain areas and is provided below.

[0139] As part of its operation, source encoder 403 may perform motion compensated predictive coding, which predictively encodes an input frame with reference to one or more previously encoded frames from a video sequence designated as "reference frames." In this manner, encoding engine 407 encodes the differences between pixel blocks of an input frame and pixel blocks of a reference frame that may be selected as a prediction reference for the input frame.

[0140] The local video decoder 406 can decode the encoded video data of the frame that can be designated as the reference frame based on the symbol created by the source encoder 403. The operation of the encoding engine 407 can advantageously be lossy processing. When the encoded video data can be decoded at the video decoder ( Figure 4 When decoded at a remote video decoder (not shown), the reconstructed video sequence may typically be a replica of the source video sequence with some errors. The local video decoder 406 replicates the decoding process that may be performed by the video decoder on the reference frame and may cause the reconstructed reference frame to be stored in the reference picture cache 405. In this way, the encoder 400 may locally store a copy of the reconstructed reference frame that has common content (absent transmission errors) with the reconstructed reference frame to be obtained by the remote video decoder.

[0141] The predictor 404 may perform a prediction search for the encoding engine 407. That is, for a new frame to be encoded, the predictor 404 may search the reference picture memory 405 for sample data (as a candidate reference pixel block) or specific metadata, such as reference picture motion vectors, block shapes, etc., that may be used as a suitable prediction reference for the new picture. The predictor 404 may operate on a pixel block-by-pixel block basis based on sample blocks to find a suitable prediction reference. In some cases, as determined by the search results obtained by the predictor 404, the input picture may have a prediction reference extracted from a plurality of reference pictures stored in the reference picture memory 405.

[0142] The controller 402 may manage encoding operations of the source encoder 403 , including, for example, setting parameters and subgroup parameters for encoding video data.

[0143] The outputs of all the above functional units may be subjected to entropy coding in the entropy encoder 408. The entropy encoder converts the symbols generated by the various functional units into a coded video sequence by losslessly compressing the symbols according to techniques known to those skilled in the art (e.g., Huffman coding, variable length coding, arithmetic coding, etc.).

[0144] Transmitter 409 may buffer the encoded video sequence created by entropy encoder 408 in preparation for transmission via communication channel 411, which may be a hardware / software link to a storage device where the encoded video data will be stored. Transmitter 409 may combine the encoded video data from source encoder 403 with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (source not shown).

[0145] The controller 402 may manage the operation of the encoder 400. During encoding, the controller 402 may assign a certain coded picture type to each coded picture, but this may affect the coding techniques that may be applied to the corresponding picture. For example, a picture may generally be assigned to any of the following picture types:

[0146] An intra picture (I picture) may be a picture that can be encoded and decoded without using any other frame in the sequence as a prediction source. Some video codecs allow different types of intra pictures, including, for example, Independent Decoder Refresh pictures. Those skilled in the art are aware of those variations of I pictures and their corresponding applications and features.

[0147] A predictive picture (P picture) may be a picture that may be encoded and decoded using intra prediction or inter prediction, which uses at most one motion vector and a reference index to predict sample values ​​for each block.

[0148] Bidirectional predictive pictures (B pictures), which can be pictures that can be encoded and decoded using intra prediction or inter prediction, which uses up to two motion vectors and reference indices to predict the sample values ​​of each block. Similarly, multiple predictive pictures can use more than two reference pictures and associated metadata for reconstruction of a single block.

[0149] The source picture may typically be spatially subdivided into a number of blocks of samples (e.g., blocks of 4×4, 8×8, 4×8, or 16×16 samples each) and encoded block by block. These blocks may be predictively encoded with reference to other (already encoded) blocks determined by the coding allocation applied to the block's corresponding picture. For example, blocks of an I picture may be non-predictively encoded, or these blocks may be predictively encoded (spatial prediction or intra-frame prediction) with reference to already encoded blocks of the same picture. Pixel blocks of a P picture may be non-predictively encoded with reference to one previously encoded reference picture via spatial prediction or via temporal prediction. Blocks of a B picture may be non-predictively encoded with reference to one or two previously encoded reference pictures via spatial prediction or via temporal prediction.

[0150] The video encoder 400 may perform encoding operations according to a predetermined video encoding technique or standard, such as ITU-T H.265 Recommendation. In the operation of the video encoder 400, the video encoder 400 may perform various compression operations, including predictive encoding operations that exploit temporal and spatial redundancy in an input video sequence. Thus, the encoded video data may conform to the syntax specified by the video encoding technique or standard used.

[0151] In an embodiment, the transmitter 409 may transmit additional data when transmitting the encoded video. The video encoder 400 may include such data as part of the encoded video sequence. The additional data may include time / space / SNR enhancement layers, other forms of redundant data such as redundant pictures and slices, Supplementary Enhancement Information (SEI) messages, Visual Usability Information (VUI) parameter set fragments, etc.

[0152] Figure 5 The intra prediction modes used in HEVC and JEM are shown. To capture arbitrary edge directions present in natural video, the number of directional intra modes is extended from 33 used in HEVC to 65. The additional directional modes in JEM on top of HEVC are depicted as Figure 5 The dashed arrows in the right figure are shown, and the planar mode and DC mode remain unchanged. These more dense directional intra prediction modes are applicable to all block sizes and both luma and chroma intra prediction. Figure 5 As shown, the directional intra prediction mode associated with an odd intra prediction mode index and identified by a dotted arrow is referred to as an odd intra prediction mode. The directional intra prediction mode associated with an even intra prediction mode index and identified by a solid arrow is referred to as an even intra prediction mode. In this document, Figure 5 The directional intra prediction mode indicated by the solid or dashed arrow in is also called angular mode.

[0153] In JEM, a total of 67 intra prediction modes are used for luma intra prediction. To encode in intra mode, an MPM list of size 6 is constructed based on the intra modes of neighboring blocks. If the intra mode is not from the MPM list, a flag is signaled to indicate whether the intra mode belongs to the selected mode. In JEM-3.0, there are 16 selected modes, which are uniformly selected for every four angular modes. In JVET-D0114 and JVET-G0060, 16 secondary MPMs are derived to replace the uniformly selected modes.

[0154] In JEM, for the luminance component, the neighboring samples used for intra prediction sample generation are filtered before the generation process. The filtering is controlled by a given intra prediction mode and a transform block size. If the intra prediction mode is DC or the transform block size is equal to 4×4, the neighboring samples are not filtered. If the distance between a given intra prediction mode and a vertical mode (or horizontal mode) is greater than a predefined threshold, the filtering process is enabled. For neighboring sample filtering, a [1, 2, 1] filter and a bilinear filter are used.

[0155] The position dependent intra prediction combination (PDPC) method is an intra prediction method that calls for a combination of HEVC-style intra prediction using filtered boundary reference samples and unfiltered boundary reference samples. Each prediction sample pred[x][y] at (x, y) is calculated as follows:

[0156] pred[x][y]=(wL*R- 1，y +wT*R x，-1 +wTL*R -1，-1 +(64-wL-wT-wTL)*pred[x][y]+32)>>6

[0157] (Formula 2-1)

[0158] Where Rx,-1, R-1,y represent the unfiltered reference samples located above and to the left of the current sample (x, y), respectively, and R-1,-1 represents the unfiltered reference sample located at the upper left corner of the current block. The weighting is calculated as follows:

[0159] wT=32>>((y<<1)>>shift) (Formula 2-2)

[0160] wL=32>>((x<<1)>>shift) (Formula 2-3)

[0161] wTL=-(wL>>4)-(wT>>4) (Formula 2-4)

[0162] Shift = (log2(width) + log2(height) + 2) >> 2 (Formula 2-5)

[0163] Figure 6 A diagram 700 is shown in which the DC mode PDPC is weighted (wL, wT, wTL) for the (0, 0) position and the (1, 0) position within a 4×4 block. If PDPC is applied to DC intra mode, planar intra mode, horizontal intra mode, and vertical intra mode, no additional boundary filters, such as the HEVC DC mode boundary filter or the horizontal / vertical mode edge filters, are required. Figure 6 The definition of reference samples Rx,-1, R-1,y and R-1,-1 for PDPC applied to the upper right diagonal mode is shown. The prediction sample pred(x',y') is located at (x',y') within the prediction block. The coordinate x of the reference sample Rx,-1 is given by x=x'+y'+1, and the coordinate y of the reference sample R-1,y is similarly given by y=x'+y'+1.

[0164] For local illumination compensation (LIC), it can be based on a linear model for brightness change using a scaling factor a and an offset b. And for each inter-mode coded coding unit (CU), local illumination compensation can be adaptively enabled or disabled.

[0165] When LIC is applied to a CU, the least square error method is adopted to derive parameters a and b by using the neighboring samples of the current CU and their corresponding reference samples. More specifically, the subsampled (2:1 subsampled) neighboring samples of the CU and the corresponding samples in the reference picture (identified by the motion information of the current CU or sub-CU) are used. The IC parameters are derived and applied to each prediction direction separately.

[0166] When a CU is encoded using merge mode, the LIC flag is copied from the neighboring block in a similar manner to motion information copying in merge mode; otherwise, the LIC flag is signaled to the CU to indicate whether LIC is applied.

[0167] Fig. 7A Intra prediction modes used in HEVC are shown 900. In HEVC, there are a total of 35 intra prediction modes, of which mode 10 is a horizontal mode, mode 26 is a vertical mode, and modes 2, 18, and 34 are diagonal modes. Intra prediction modes are signaled by three most probable modes (MPMs) and 32 remaining modes.

[0168] Figure 7B It is shown that there are a total of 87 intra prediction modes in the implementation of VVC, of ​​which mode 18 is a horizontal mode, mode 50 is a vertical mode, and mode 2, mode 34 and mode 66 are diagonal modes. Mode-1 to mode-10 and mode 67 to mode 76 are called wide-angle intra prediction (WAIP) modes.

[0169] According to the PDPC expression, the prediction sample pred(x, y) at position (x, y) is predicted using a linear combination of the intra prediction mode (DC, planar, angular) and the reference samples:

[0170] pred(x,y)=(wL×R-1,y+wT×Rx,-1-wTL×R-1,-1+(64-wL-wT+wTL)×pred(x,y)+32)>>6

[0171] Among them, Rx,-1, R-1,y respectively represent the reference samples located above and to the left of the current sample (x, y), and R-1,-1 represents the reference sample located at the upper left corner of the current block.

[0172] For DC mode, for a block with dimensions width and height, the weights are calculated as follows:

[0173] wT=32>>((y<<1)>>nScale), wL=32>>((x<<1)>>nScale), wTL=(wL>>4)+(wT>>4)

[0174] Wherein, nScale = (log2(width)-2+log2(height)-2+2)>>2, where wT represents the weighting factor of the reference sample with the same horizontal coordinate in the upper reference line, wL represents the weighting factor of the reference sample with the same vertical coordinate in the left reference line, and wTL represents the weighting factor of the upper left reference sample of the current block, nScale specifies the speed at which the weighting factor decreases along the axis (wL decreases from left to right or wT decreases from top to bottom), that is, the weighting factor decreasing rate, and in the current design, the weighting factor decreasing rate is the same along the x-axis (from left to right) and the y-axis (from top to bottom). And 32 represents the initial weighting factor of the neighboring sample, and the initial weighting factor is also the upper (left or upper left) weight assigned to the upper left sample in the current CB, and the weighting factor of the neighboring sample in the PDPC process should be equal to or less than the initial weighting factor.

[0175] For planar mode wTL=0, while for horizontal mode wTL=wT and for vertical mode wTL=wL. The PDPC weights can be calculated using only addition and shifting. The value of pred(x,y) can be calculated in a single step using Equation 1.

[0176] Figure 8 A flowchart 1000 according to an exemplary embodiment is shown, wherein at S1, PDPC is considered to be applied to reconstructed samples or inter-frame predicted samples of an inter-frame coded CU. When PDPC is applied to reconstructed samples or inter-frame predicted samples of an inter-frame coded CU, the reconstructed samples or predicted samples of the inter-frame coded CU are further modified in the PDPC process.

[0177] At S2, a flag such as interPDPCFlag is signaled to indicate whether PDPC is applied. According to an embodiment, interPDPCFlag may be signaled at the CU level to indicate whether PDPC is applied to reconstructed samples or inter-predicted samples of an inter-coded CU, and according to other embodiments, interPDPCFlag may be signaled at the TU level to indicate whether PDPC is applied to reconstructed samples or inter-predicted samples of an inter-coded CU. In addition, interPDPCFlag may be signaled only when it is determined that the current CU is neither an intra-coded CU nor a skipped CU.

[0178] According to an embodiment, at S2, the interPDPCFlag may be signaled only for coding blocks encoded by merge mode, and the flag may be signaled before or after the merge index, so that according to an embodiment, PDPC is not applied to the skipped CU, and / or so that PDPC is applied neither to the skipped CU nor to the sub-block merged CU.

[0179] At S3, consider whether interPDPCFlag is set equal to true after reconstruction or inter prediction of the inter-coded CU. If so, at S4, PDPC is applied to the reconstructed samples or inter-prediction samples of the inter-coded CU to generate reconstructed samples or final prediction values ​​of the inter-coded CU.

[0180] According to an embodiment, at S4, when PDPC is applied to the reconstructed samples, PDPC is applied to the reconstructed samples before applying any in-loop deblocking filter (eg, deblocking, Sample Adaptive Offset (SAO) and Adaptive Loop Filter (ALF), etc.).

[0181] According to an embodiment, when interPDPCflag is set to true at S3, deblocking and SAO are not applied to the upper and left boundaries of the current CU at S4, and according to other embodiments, when interPDPCflag is set to true at S3, any other filter except PDPC is not applied to the upper and left boundaries of the current CU.

[0182] Otherwise, at S5, after reconstruction or inter prediction of the inter-coded CU, PDPC is not applied to the reconstructed samples or inter-predicted samples of the inter-coded CU, but instead, a conventional inter prediction process may be used.

[0183] According to an embodiment, PDPC may be determined and applied only to the luma component of an inter-coded CU.

[0184] According to an embodiment, at S6, when PDPC is applied to the reconstructed sample or inter-prediction sample of the inter-coded CU of the current CB, it can be determined whether to assign the default intra-prediction mode to the current CB at S7, so that the subsequent CB can use the default intra-prediction mode for intra-mode coding and most probable mode (MPM) derivation. For example, the default intra-prediction mode can be any of the planar mode and the DC mode.

[0185] At S8, it is determined whether different initial weighting factors of PDPC are used for the PDPC filtering process applied in the conventional intra prediction mode and the PDPC filtering process at S4. If so, at S9, the weighting factors can be set to 16, 8, 4, 2 or 1 for the PDPC at S4. According to an embodiment, when PDPC is applied to the reconstructed samples or inter-prediction samples of the inter-coded CU of the current CB, the intra prediction mode can be marked as unavailable, so that the current CB will not be regarded as an intra-coded block for subsequent neighboring blocks, and the current CB will not be used for intra-mode coding or MPM derivation for subsequent neighboring blocks.

[0186] According to an embodiment, in the case where PDPC is applied to the reconstructed samples or inter-prediction samples of an inter-coded CU at S4, the weighting factor decrement rate and / or the initial weighting factor may be considered at S10, and the weighting factor decrement rate and / or the initial weighting factor may be set to different values ​​than those used in the original PDPC in intra prediction at S11.

[0187] According to an embodiment, at S11, the weighting factor decreasing rate may be set equal to a fixed value N, which is independent of the block size of the current coding unit, and N may be any integer equal to or greater than 0. For example, N may be set equal to 0, 1, 2, 3, or 4.

[0188] According to an embodiment, at S11, when the block size (width*height) is equal to or less than Thres_1, the weighting factor decreasing rate may be set equal to a fixed value N1. Otherwise, the weighting factor decreasing rate may be set equal to another fixed value N2. N1 and N2 may be any integer greater than or equal to 0. For example, N1 is set equal to 1, and N2 is set equal to 2. Thres_1 may be any positive integer, such as 32, 64, 128, 256, 512, or 1024.

[0189] According to an embodiment, at S11, the weighting factor decreasing rate can be set equal to ((Log2(nTbW)+Log2(nTbH)+S)>>2), where S can be any integer, such as -3, -2, -1, 0, 1, 2 or 3.

[0190] According to an embodiment, at S11, when PDPC is applied to inter-frame prediction samples, the weighting factor decreasing rate may have different values ​​for the x-axis (from left to right) and the y-axis (from top to bottom). For example, the weighting factor decreasing rate along the x-axis may be related to the width of the current CB, such as (Log2(nTbH)+S1)>>1, and the weighting factor decreasing rate along the y-axis may be related to the height of the current CB, such as (Log2(nTbW)+S2)>>1. S1 and S2 may be any integers, such as -3, -2, -1, 0, 1, 2, or 3.

[0191] In addition, regarding Figure 8 In an implementation manner, high-level syntax elements including but not limited to SPS (Sequence Parameter Set), PPS (Picture Parameter Set), slice header, and CTU header may be used to signal whether PDPC is allowed to be applied on an inter-frame prediction block or an inter-frame reconstruction block, and a flag may be sent by signal to indicate whether PDPC is allowed to be applied on a luma inter-frame prediction block or an inter-frame reconstruction block, and another flag may be sent by signal to indicate whether PDPC is allowed to be applied on a chroma inter-frame prediction block or an inter-frame reconstruction block.

[0192] Fig. 9 A flowchart 1100 is shown according to an exemplary embodiment. Figure 8 implementation method or Figure 8 1100 is separated from the implementation of the embodiment, where S1, S2, S3 and S5 can be similar, and at S3, if it is determined that the flag is set to be equal to true, then at S12, a second flag or index can be sent with a signal. Flowchart 1100 involves applying multiple PDPC filter sets to inter-frame prediction samples (or reconstructed samples of an inter-frame coded CU). Multiple PDPC filter sets can be defined for each block size, and when PDPC is applied to inter-frame prediction samples or reconstructed samples of an inter-frame coded CU, the prediction samples in the PDPC process can be modified to inter-frame prediction samples or reconstructed samples of an inter-frame coded CU. The selection of the PDPC filter is signaled for the coding block to which PDPC can be applied.

[0193] At S12, according to an embodiment, a flag, i.e., interPDPCFlag, may be signaled at S2 to indicate whether PDPC is applied. If interPDPCFlag is set equal to true at S3, a second flag or index may be signaled to indicate which PDPC filter is utilized after inter prediction. The specified filter type may be applied to the inter prediction samples or the reconstructed samples of the inter-coded CU to generate the final inter prediction values ​​or the reconstructed samples of the inter-coded CU.

[0194] At S13 , according to an embodiment, there may be N different PDPC filter types, which may be signaled using unary truncation coding or fixed length coding, where N may be any positive integer, such as 2, 3 or 4.

[0195] At S13, according to an embodiment, the number of PDPC filter types may depend on encoding information, including but not limited to block width, block height, block aspect ratio, block area size, whether the current block is uni-predicted or bi-predicted, and whether the current block is encoded using MERGE mode or AMVP mode.

[0196] At S13, according to an embodiment, different PDPC filter types can be specified by a weighting factor decrement rate or a scaling factor, and / or an initial weighting factor weighted above and / or to the left and / or to the upper left sample of the current block to which the weighting is assigned. For example, the weighting factor decrement rate can be set to ((Log2(nTbW)+Log2(nTbH)-2+S)>>2), where S can be any integer, and S has different settings for different PDPC filter types. In an embodiment, the weighting factor decrement rate can be set to N for each PDPC filter type, where N can be any integer, and N can have different settings for different PDPC filter types. In an embodiment, the decrement rate of the weighting factor can be derived using the method described when there are N different PDPC filter types, and the initial value of the weighting factor can be 32, 16, 8, 4, 2, 1 or 0. In other embodiments, only the initial value of the weighting factor may be different for different PDPC filter types, and example values ​​of the initial value of the weighting factor may include, but are not limited to: 32, 16, 8, 4, 2, 1 or 0.

[0197] In addition, regarding S13, similar to S6 and S7, when PDPC is applied to inter-frame prediction samples or reconstructed samples of inter-frame encoded CUs, a default intra-frame prediction mode can be assigned to the current CB so that subsequent CBs can use the default intra-frame prediction mode for MPM derivation, and the default intra-frame prediction mode can be a planar mode or a DC mode.

[0198] Fig.10 A flowchart 1200 is shown relating to multi-hypothesis intra-inter prediction, referred to as intra-inter mode or also referred to as inter-intra mode, or a combined merge and intra prediction mode. Applying the same weights to intra prediction and inter prediction is referred to as equal weighting, e.g., (w_intra, w_inter) = (1, 1) or (w_intra, w_inter) = (4, 4) are both equal weightings, where w_intra is the weight applied to intra prediction and w_inter is the weight applied to inter prediction.

[0199] At S14, for the weights applied by the intra-inter mode, i.e., w_intra and w_inter, a process of determining a set of weights is started, and the selection of weights for blocks encoded by the intra-inter mode is signaled. At S14, parameters of the neighboring blocks may be checked, and at S15, a first flag may be signaled to indicate whether equal weighting is used. According to an embodiment, the context for entropy encoding the first flag depends on whether the neighboring block is intra-coded, inter-coded, or intra-inter-coded. According to an embodiment, the context for entropy encoding the first flag depends on whether one of the upper neighboring block and the left neighboring block is intra-coded. According to an embodiment, the context for entropy encoding the first flag depends on whether one of the upper neighboring block and the left neighboring block is intra-coded or intra-inter-coded. According to an embodiment, the context for entropy encoding the first flag depends on whether one of the upper neighboring block and the left neighboring block is inter-coded.

[0200] For example, at S16 it may be determined whether certain conditions are met.If the first flag is signaled with a value indicating that equal weighting is not to be applied, the weighting is selected depending on whether the neighboring blocks are intra-coded, inter-coded or intra-inter-coded.

[0201] According to an exemplary embodiment, if any of the exemplary conditions described below are met, a weighting is applied at S17, such as the example candidate weighting (w_intra, w_inter) = (3, 1), and if any of the exemplary conditions described below are not met, a weighting is applied at S18, such as the example candidate weighting (w_intra, w_inter) = (1, 3).

[0202] The condition at S16 may be considered according to any one of the following: whether the context for entropy encoding the first flag depends on “whether both the upper neighboring block and the left neighboring block are intra-coded”; whether the context for entropy encoding the first flag depends on “whether both the upper neighboring block and the left neighboring block are intra-coded or intra-inter-coded”; and whether the context for entropy encoding the first flag depends on “whether both the upper neighboring block and the left neighboring block are inter-coded”. However, according to an exemplary embodiment, if the context for entropy encoding the first flag depends on whether both the upper neighboring block and the left neighboring block are inter-coded, then in contrast to the other embodiments described above, the weighting may be set to (w_intra, w_inter)=(1,3).

[0203] At S19, a second flag may be signaled to indicate which weight in the candidate weighting set may be used. According to an embodiment, the context for entropy encoding the second flag depends on whether the neighboring block is intra-coded, inter-coded, or intra-inter-coded. For example, if both the upper neighboring block and the left neighboring block are intra-coded, the context value may be set to a first value, otherwise, if one of the upper neighboring block and the left neighboring block is intra-coded, the context value may be set to a second value, otherwise, if neither the upper neighboring block nor the left neighboring block is intra-coded, the context value may be set to a third value. According to other embodiments, if both the upper neighboring block and the left neighboring block are intra-coded or intra-inter-coded, the context value may be set to a first value, otherwise, if one of the upper neighboring block and the left neighboring block is intra-coded or intra-inter-coded, the context value may be set to a second value, otherwise, if neither the upper neighboring block nor the left neighboring block is intra-coded, the context value may be set to a third value. According to an embodiment, the candidate weighted sets may include (w_intra, w_inter)=(1,3) and (w_intra, w_inter)=(3,1).

[0204] Fig.11 A flow chart 1300 is shown, wherein at S20 intra prediction is not used in intra-inter mode but PDPC is applied directly on top of inter prediction; however, the weighting applied to inter prediction in the PDPC filter depends on whether the neighboring blocks are intra coded, inter coded, or intra-inter coded.

[0205] According to an embodiment, at S21, PDPC is directly applied on top of inter prediction as follows:

[0206] pred(x, y) = (w_intra*wL×R -1，y +w_mtra*wT×Rx，-1 -w_intra*wTL×R -1，-1 +(N-w_intra*wL-w_intra*wT+w_intra*wTL)×InterPred(x,y)+N / 2)>>log2(N)

[0207] Where InterPred is the input inter prediction, and w_intra is selected from a set of predefined values, and the selection depends on whether the neighboring block is intra-coded, inter-coded, or intra-inter-coded. Example values ​​of N include 8, 16, 32, 64, 128, 256. If the neighboring block meets the condition, a weight of 1 can be applied at S22, and if the neighboring block does not meet the condition, a weight of 2 can be applied at S23.

[0208] According to an embodiment, at S21, PDPC is directly applied on top of inter prediction as follows:

[0209] pred(x, y) = (w_intra*wL×R -1，y +w_intra*wT×R x，-1 +(N-w_intra*wL-w_intra*wT)×InterPred(x,y)+(N / 2))>>log2(N)

[0210] Where InterPred is the input inter prediction, and w_intra is selected from a set of predefined values, and the selection depends on whether the neighboring block is intra-coded, inter-coded, or intra-inter-coded. Example values ​​of N include 8, 16, 32, 64, 128, 256. If the neighboring block meets the condition, a weight of 1 can be applied at S22, and if the neighboring block does not meet the condition, a weight of 2 can be applied at S23.

[0211] According to an embodiment, at S21, PDPC is directly applied on top of inter prediction as follows:

[0212] pred(x, y) = (w_intra*wL×R -1，y +w_intra*wT×R x，-1 +(N*(w_intra+w_inter)-w_intra*wL-w_intra*wT)×InterPred(x,y)+(N / 2)*((w_intra+w_inter)))>>(log2(N)+log2(w_intra+w_inter))

[0213] Where InterPred is the input inter prediction, and w_intra is selected from a set of predefined values, and the selection depends on whether the neighboring block is intra-coded, inter-coded, or intra-inter-coded. For example, when w_intra=3 and w_inter=1, the above equation can be rewritten as:

[0214] pred(x,y)=(3*wL×R -1，y +3*wT×R x，-1 +(N*4-3*wL-3*wT)×InterPred(x,y)+2*N)>>(log2(N)+2)

[0215] Example values ​​of N include 8, 16, 32, 64, 128, 256.

[0216] According to an embodiment, at S21, the weighting applied to the upper reference sample (wT) in the PDPC depends on whether the upper neighboring block is intra-coded, inter-coded, or intra-inter-coded, and the weighting applied to the left reference sample (wL) in the PDPC depends on whether the left neighboring block is intra-coded, inter-coded, or intra-inter-coded.

[0217] According to an embodiment, at S21, PDPC is directly applied on top of inter prediction as follows:

[0218] pred(x, y) = (w_intraL*wL×R -1， y+w_intraT*wT×R x，-1 -w_intraTL*wTL×R -1，-1+ (N-w_intraL*wL-w_intraT*wT+w_intraTL*wTL)×InterPred(x,y)+N / 2)>>log2(N)

[0219] Where InterPred is the input inter prediction, and w_intraL is selected from a set of predefined values, and the selection depends on whether the left neighboring block is intra-coded, inter-coded, or intra-inter-coded, w_intraT is selected from a set of predefined values, and the selection depends on whether the upper neighboring block is intra-coded, inter-coded, or intra-inter-coded, and w_intraTL is selected from a set of predefined values, and the selection depends on whether the upper left neighboring block is intra-coded, inter-coded, or intra-inter-coded. Example values ​​of N include 8, 16, 32, 64, 128, 256. If the neighboring block meets the condition, a weight of 1 can be applied at S22, and if the neighboring block does not meet the condition, a weight of 2 can be applied at S23.

[0220] According to an embodiment, at S21, PDPC is directly applied on top of inter prediction as follows:

[0221] pred(x, y) = (w_intraL*wL×R -1，y +w_intraT*wT×R x，-1 +(N-w_intraL*wL-w_intraT*wT)×InterPred(x,y)+N / 2)>>log2(N)

[0222] Where InterPred is the input inter prediction, and w_intraL is selected from a predefined set of values, and the selection depends on whether the left neighboring block is intra-coded, inter-coded, or intra-inter-coded, and w_intraT is selected from a predefined set of values, and the selection depends on whether the upper neighboring block is intra-coded, inter-coded, or intra-inter-coded. Example values ​​of N include 8, 16, 32, 64, 128, 256. If the neighboring block meets the condition, a weight of 1 can be applied at S22, and if the neighboring block does not meet the condition, a weight of 2 can be applied at S23.

[0223] Therefore, through the exemplary embodiments described herein, the above-mentioned technical problems can be advantageously improved through these technical solutions.

[0224] The above techniques may be implemented as computer software using computer-readable instructions and physically stored in one or more computer-readable media, or may be implemented by one or more specially configured hardware processors. For example, Fig.12 A computer system 1400 suitable for implementing certain embodiments of the disclosed subject matter is shown.

[0225] Computer software may be encoded using any suitable machine code or computer language, which may be subjected to mechanisms such as assembly, compilation, and linking to create code comprising instructions that may be executed directly by a computer central processing unit (CPU), graphics processing unit (GPU), etc., or through interpretation, microcode execution, etc.

[0226] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smart phones, gaming devices, Internet of Things devices, etc.

[0227] Fig.12The components for computer system 1400 shown in the example are exemplary in nature and are not intended to suggest any limitation on the functionality or scope of use of computer software implementing embodiments of the present disclosure. Nor should the configuration of components be interpreted as having any dependency or requirement related to any one or combination of components shown in the exemplary embodiment of computer system 1400.

[0228] Computer system 1400 may include certain human-machine interface input devices. Such human-machine interface input devices may respond to input by one or more human users through, for example, tactile input (such as: keystrokes, swipes, data glove movements), audio input (such as: voice, tapping), visual input (such as: gestures), olfactory input (not shown). Human-machine interface devices may also be used to capture certain media that are not necessarily directly related to human conscious input, for example, audio (such as: voice, music, ambient sound), images (such as: scanned images, photographic images obtained from a still image camera), and videos (such as two-dimensional video, three-dimensional video including stereoscopic video).

[0229] Input human interface devices may include one or more of the following (only one of each is depicted): keyboard 1402 , mouse 1404 , trackpad 1403 , touch screen 1410 , joystick 1405 , microphone 1406 , scanner 1408 , camera 1407 .

[0230] The computer system 1400 may also include certain human-machine interface output devices. Such human-machine interface output devices may stimulate the senses of one or more human users through, for example, tactile output, sound, light, and smell / taste. Such human-machine interface output devices may include tactile output devices (e.g., tactile feedback through touch screen 1410, or joystick 1405, but there may also be tactile feedback devices that are not used as input devices), audio output devices (e.g., speakers 1409, headphones (not depicted)), visual output devices (e.g., screen 1410, including CRT screens, LCD screens, plasma screens, OLED screens, each with or without touch screen input capabilities, each with or without tactile feedback capabilities - some of which may be able to output two-dimensional visual outputs or more than three-dimensional outputs by means such as stereoscopic image output; virtual reality glasses (not depicted), holographic displays, and cigarette cans (not depicted)) and printers (not depicted). A graphics adapter 1429 may be provided for interfacing with various input devices.

[0231] The computer system 1400 may also include human-accessible storage devices and their associated media, such as optical media including CD / DVD ROM / RW 1412 with CD / DVD etc. media 1411, thumb drives 1413, removable hard drives or solid-state drives 1414, legacy magnetic media such as tapes and floppy disks (not depicted), dedicated ROM / ASIC / PLD based devices such as security dongles (not depicted), and the like.

[0232] Those skilled in the art will also appreciate that the term "computer-readable media" used in connection with the presently disclosed subject matter does not include transmission media, carrier waves, or other transient signals.

[0233] The computer system 1400 may also include an interface to one or more communication networks 1415. The network 1415 may be, for example, wireless, wired, optical. The network 1415 may also be local, wide, metropolitan, automotive and industrial, real-time, delay-tolerant, etc. Examples of networks 1415 include: local area networks such as Ethernet, wireless LANs, cellular networks including GSM, 3G, 4G, 5G, LTE, etc., television wired or wireless wide-area digital networks including cable, satellite, and terrestrial broadcast television, automotive and industrial networks including CANBus, etc. Some networks 1415 typically require an external network interface adapter 1427 attached to some general data port or peripheral bus 1425 (e.g., a USB port of the computer system 1400); others are typically integrated into the core of the computer system 1400 by attaching to the system bus as described below (e.g., an Ethernet interface integrated into a PC computer system or a cellular network interface integrated into a smartphone computer system). Using any of these networks, the computer system 1400 can communicate with other entities. Such communications can be one-way receive-only (e.g., broadcast television), one-way send-only (e.g., CANbus to certain CANbus devices), or two-way, for example, to other computer systems using local or wide area digital networks. Certain protocols and protocol stacks can be used on each of these networks 1415 and network interfaces 1427 as described above.

[0234] The above-mentioned human-machine interface devices, human-accessible storage devices, and network interface 1427 may be attached to the core 1416 of the computer system 1400 .

[0235] The core 1416 may include one or more central processing units (CPUs) 1421, graphics processing units (GPUs) 1422, dedicated programmable processing units in the form of field programmable gate arrays (FPGAs) 1424, hardware accelerators 1426 for certain tasks, etc. These devices, along with read-only memory (ROM) 1419, random access memory 1418, and internal mass storage devices such as internal non-user accessible hard drives, SSDs, etc., may be connected through a system bus 1428. In some computer systems, the system bus 1428 may be accessed in the form of one or more physical plugs to enable expansion by additional CPUs, GPUs, etc. Peripheral devices may be attached to the system bus 1428 of the core directly or through a peripheral bus 1401. The architecture of the peripheral bus includes PCI, USB, etc.

[0236] The CPU 1421, GPU 1422, FPGA 1424, and accelerator 1426 may execute certain instructions, which may be combined to form the computer code mentioned above. The computer code may be stored in ROM 1419 or RAM 1418. Transitional data may also be stored in RAM 1418, while permanent data may be stored in, for example, an internal mass storage device 1420. Fast storage and retrieval to any of the storage devices may be achieved by using a cache memory, which may be closely associated with one or more CPUs 1421, GPUs 1422, mass storage devices 1420, ROM 1419, RAM 1418, etc.

[0237] The computer readable medium may have computer code thereon for performing various computer-implemented operations. The medium and computer code may be those specially designed and constructed for the purposes of the present disclosure, or they may be of a type well known and available to those skilled in the art of computer software.

[0238] As an example and not limitation, a computer system 1400 having an architecture, in particular a core 1416, can provide functionality due to a processor (including a CPU, GPU, FPGA, accelerator, etc.) executing software implemented in one or more tangible computer-readable media. Such a computer-readable medium can be a medium associated with a user-accessible mass storage device as described above, and certain storage devices of the core 1416 having non-transitory properties, for example, a core internal mass storage device 1420 or ROM 1419. Software implementing various embodiments of the present disclosure can be stored in such a device and executed by the core 1416. Depending on specific needs, the computer-readable medium may include one or more storage devices or chips. The software can enable the core 1416, in particular the processor therein (including a CPU, GPU, FPGA, etc.), to perform a specific process or a specific part of a specific process described herein, including defining a data structure stored in the RAM 1418 and modifying such a data structure according to a process defined by the software. Additionally or alternatively, the computer system may provide functionality due to logic hardwired or otherwise implemented in a circuit (e.g., accelerator 1426) that may operate in place of or in conjunction with software to perform a particular process or a particular portion of a particular process described herein. Where appropriate, references to software may include logic, and vice versa. Where appropriate, references to computer-readable media may include circuits (e.g., integrated circuits (ICs)) storing software for execution, circuits implementing logic for execution, or both. The present disclosure includes any suitable combination of hardware and software.

[0239] Although the present disclosure has described several exemplary embodiments, there are changes, permutations, and various replacement equivalents that fall within the scope of the present disclosure. Therefore, it should be recognized that although not explicitly shown or described herein, those skilled in the art will be able to conceive of many systems and methods that implement the principles of the present disclosure and are therefore within the spirit and scope of the present disclosure.

Claims

1. A method for video decoding, characterized in that: The method comprises: In intra-inter mode, selecting a weighted set of inter-position dependent prediction combined PDPC filtering processes for the intra-inter mode according to whether the neighboring blocks are intra-coded, inter-coded, or intra-inter-coded; and Applying the inter-frame PDPC filtering process in the intra-frame-inter-frame mode to modify at least one of the inter-frame prediction samples and the reconstructed samples of the inter-frame coded coding unit CU; wherein applying the inter-frame PDPC filtering in the intra-frame-inter-frame mode includes: Modifying only the luminance component of the inter-coded CU; Assigning a default intra prediction mode to the current coding block CB, and The default intra prediction mode is applied to CBs following the current CB for at least intra mode decoding and most probable mode (MPM) derivation.

2. The method according to claim 1, characterized in that The adjacent blocks include an upper adjacent block and a left adjacent block.

3. The method according to claim 1, characterized in that The method further comprises: The inter-position dependent intra prediction combination flag interPDPCFlag is signaled in response to determining that the coding block CB is encoded by the merge mode.

4. The method according to claim 1, characterized in that: The method further comprises: Before applying any one of the in-loop deblocking filters including deblocking, sample adaptive offset SAO, and adaptive loop filter ALF, the inter PDPC in the intra-inter mode is applied to the plurality of reconstructed samples.

5. The method according to claim 1, characterized in that The default intra prediction mode is the planar intra mode or the DC intra mode.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Applying the PDPC filter set to at least one of the inter-prediction samples other than the inter-prediction samples and at least one of the reconstructed samples of the inter-coded CUs other than the reconstructed samples of the inter-coded CUs; Each PDPC filter in the PDPC filter set is defined for a corresponding block size among a plurality of block sizes.

7. The method according to claim 6, characterized in that The method further comprises: Signaling at least one inter-position dependent intra prediction combination flag interPDPCFlag indicating whether PDPC is applied; Determining whether the interPDPCFlag is set equal to true; responsive to determining that the interPDPCFlag is set equal to a first value, signaling a second flag indicating that at least one PDPC filter is to be applied to at least one of the inter-predicted samples and the reconstructed samples of the inter-coded CU after inter-prediction, wherein the first value indicates that the interPDPCFlag is set equal to true; and In response to determining that the interPDPCFlag is set equal to a second value, determining to exclude PDPC from at least one of the inter-prediction samples and the reconstructed samples of the inter-coded CU, wherein the second value indicates that the interPDPCFlag is set not equal to true.

8. The method according to claim 1, characterized in that The method further comprises: signaling whether weights in the set of weights are equal to each other; and In response to determining that the weights are unequal indicated by the signal, inter prediction is weighted in PDPC filtering according to whether the neighboring blocks are any one of intra-coded, inter-coded, and intra-inter-coded.

9. The method according to claim 1, characterized in that: The Intra-Inter mode involves applying the PDPC filtering process directly on top of the Inter prediction.

10. The method according to claim 9, characterized in that The process of directly applying PDPC filtering on top of inter-frame prediction includes: pred(x,y)=(w_intra*wL×R -1,y +intra*inT×R x,-1 –w_intra*wTL×R -1,-1 + (N–w_intra*wL–w_intra*wT+w_intra*wTL)×InterPred(x,y)+N / 2)>>log2(N), Wherein, InterPred indicates the input inter-frame prediction, w_intra is selected from a predefined value set according to whether the adjacent block is intra-coded, inter-coded, and intra-inter-coded, wT represents the weighting factor of the reference samples with the same horizontal coordinate located in the upper reference line, wL represents the weighting factor of the reference samples with the same vertical coordinate located in the left reference line, wTL represents the weighting factor of the upper left reference sample of the current block, the upper reference line refers to the line with the same horizontal coordinate in the upper reference block and the left reference block, and the left reference line refers to the line with the same vertical coordinate in the upper reference block and the left reference block; wherein N indicates one of 8, 16, 32, 64, 128 and 256.

11. The method according to claim 1, characterized in that: If PDPC is applied to DC intra mode, planar intra mode, horizontal intra mode and vertical intra mode, no additional boundary filter is required.

12. A method for video encoding, characterized in that The method comprises: In intra-inter mode, setting a weighted set for an inter-position dependent prediction combined PDPC filtering process in the intra-inter mode according to whether the neighboring block is intra-coded, inter-coded, or intra-inter-coded; and Applying the inter-frame PDPC filtering process in the intra-frame-inter-frame mode to modify at least one of the inter-frame prediction samples and the reconstructed samples of the inter-frame coded coding unit CU; wherein applying the inter-frame PDPC filtering in the intra-frame-inter-frame mode includes: Modifying only the luminance component of the inter-coded CU; Assigning a default intra prediction mode to the current coding block CB, and The default intra prediction mode is applied to CBs following the current CB for at least intra mode encoding and most probable mode (MPM) derivation.

13. A device for video encoding / decoding, characterized in that: The device comprises a selection unit, a modification unit and an encoding / decoding unit: The selection unit is used to select a weighted set of inter-frame position-dependent prediction combined PDPC filtering processes for the intra-frame-inter mode according to whether the adjacent block is intra-frame coded, inter-frame coded or intra-frame-inter-frame coded in the intra-frame-inter mode; as well as The modification unit is configured to apply the inter-frame PDPC filtering process in the intra-frame-inter-frame mode to modify at least one of the inter-frame prediction samples and the reconstructed samples of the inter-frame coded coding unit CU; wherein applying the inter-frame PDPC filtering in the intra-frame-inter-frame mode includes modifying only the luminance component of the inter-frame coded CU; The encoding / decoding unit is used to assign a default intra prediction mode to the current coding block CB, and The default intra prediction mode is applied to CBs following the current CB for at least intra mode encoding and most probable mode (MPM) derivation.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method according to any one of claims 1 to 12.

15. A computer device, characterized in that: The device comprises a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is configured to execute the method according to any one of claims 1 to 12 according to the instructions in the program code.

16. A method for generating a video bitstream, characterized in that: The encoding method according to claim 12 generates a video bit stream.

17. A method for processing a video bitstream, characterized in that: The video bit stream is decoded based on the decoding method according to any one of claims 1 to 11.

Images