Method and apparatus for improving intra-inter prediction modes

Abstract

A method for video decoding performed by at least one processor of an encoding device is provided.SOLUTION: An improved scheme of intra-inter prediction mode and adaptive intra-inter prediction mode for video coding / decoding, including selecting a set of weights in the intra-inter mode, and applying position dependent intra prediction combination (PDPC) to modify at least one of inter prediction samples and reconstructed samples of an inter-coded coding unit.SELECTED DRAWING: Figure 10

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to U.S. Provisional Application No. 62 / 767,228, filed November 14, 2018, U.S. Provisional Application No. 62 / 788,056, filed January 3, 2019, and U.S. Application No. 16 / 673,097, filed November 4, 2019, each of which is incorporated herein by reference in its entirety. [Technical Field]

[0002] This disclosure relates to a set of advanced video coding techniques, including improvement schemes used for intra-inter prediction and adaptive intra-inter prediction modes. [Background technology]

[0003] ITU-T VCEG (Q6 / 16) and ISO / IEC MPEG (JTC 1 / SC 29 / WG 11) published the H.265 / HEVC (High Efficiency Video Coding) standard in 2013 (Version 1), 2014 (Version 2), 2015 (Version 3), and 2016 (Version 4). In 2015, these two standardization organizations jointly formed the Joint Video Exploration Team (JVET) to explore the possibility of developing a next-generation video coding standard beyond HEVC. In October 2017, these two standardization organizations announced a joint Call for Proposal (CfP) for video compression with capabilities beyond HEVC. By February 15, 2018, a total of 22 CfP responses for standard dynamic range (SDR), 12 for high dynamic range (HDR), and 12 for 360 video categories had been submitted. In April 2018, all received CfP responses were evaluated at the 122 MPEG / 10th JVET meeting. As a result of this meeting, JVET officially launched the standardization process for next-generation video coding beyond HEVC. The new standard was named Versatile Video Coding (VVC), and JVET was renamed the Joint Video Experts Team.

[0004] The following sentence from VVC Draft 2 describes PDPC, where nScale is used to specify the rate of decrease of the weighting coefficients, and the "32" in equations 8-77, 8-78, 8-82 and 8-83 specifies the initial weighting coefficients.

[0005] The inputs to the position-dependent intra prediction combining process may be as follows: - intra prediction mode predModelntra, - variable nTbW, which specifies the width of the transformation block, - variable nTbH, which specifies the height of the transformation block, - variable refW, which specifies the width of the reference sample, - variable refH, which specifies the height of the reference sample, - predicted samples predSamples[x][y], where x=0 nTbW-1 and y=0 nTbH-1, - neighboring samples p[x][y], where x=-1, y=-1···refH-1 and x=0···refW-1, y=-1, - Variable cIdx that specifies the color component of the current block.

[0006] The output of the process is the modified predicted samples predSamples[x][y], where x = 0···nTbW-1 and y = 0···nTbH-1.

[0007] Depending on the value of cIdx, the function clip1Cmp is set as follows: If -cIdx is equal to 0, then clip1Cmp is set equal to Clip1Y. Otherwise, clip1Cmp is set equal to Clip1C.

[0008] The variable nScale is set to ((Log2(nTbW)+Log2(nTbH)-2)≫2).

[0009] The reference sample arrays mainRef[x] and sideRef[y] are derived as follows, where x=0···refW and y=0···refH.

number

[0010] The variables refL[x][y], refT[x][y], wT[y], wL[y] and wTL[x][y] are derived as follows, where x = 0···nTbW−1 and y = 0···nTbH−1. When -predModeIntra is equal to INTRA_PLANAR, INTRA_DC, INTRA_ANGULAR18 or INTRA_ANGULAR50, the following applies:

number

[0011] Otherwise, if predModeIntra is equal to INTRA_ANGULAR2 or INTRA_ANGULAR66, the following applies:

number

[0012] Otherwise, if predModeIntra is less than or equal to INTRA_ANGULAR10, the following ordered procedure applies: -Depending on intraPredMode, derive the variables dXPos[y], dXFrac[y], dXInt[y] and dX[y] as follows, e.g. using invAngle as specified in clause 8.2.4.2.7:

number

[0013] The variables refL[x][y], refT[x][y], wT[y], wL[y], and wTL[x][y] are derived as follows:

number

[0014] Otherwise, if predModeIntra is INTRA_ANGULAR58 or greater, the following ordered procedure applies: Depending on intraPredMode, derive the variables dYPos[x], dYFrac[x], dYInt[x] and dY[x], for example using invAngle as specified in clause 8.2.4.2.7, as follows:

number

number

[0015] The values ​​of the filtered samples filtSamples[x][y] are derived as follows, where x = 0···nTbW-1 and y = 0···nTbH-1:

[0016] For the chroma components of an intra-coded block, the encoder selects the optimal chroma prediction mode from five modes, including planar (mode index 0), DC (mode index 1), horizontal (mode index 18), vertical (mode index 50), and diagonal (mode index 66), and selects a direct copy of the intra prediction mode to use for the associated luma component (i.e., DM mode). The mapping between chroma intra prediction mode numbers and intra prediction directions is shown in Table 1. Table 1 - Mapping between chroma intra prediction modes and intra prediction directions [Table 1]

[0017] To avoid overlapping modes, the four modes other than DM are assigned according to the intra prediction mode of the associated luma component. If the intra prediction mode number of the chroma component is 4, the intra prediction direction of the luma component is used to generate intra prediction samples for the chroma component. If the intra prediction mode number of the chroma component is not 4 and is the same as the intra prediction mode number of the luma component, intra prediction direction 66 is used to generate intra prediction samples for the chroma component.

[0018] Merge mode for inter-picture prediction is used to indicate for a block that motion data is estimated rather than explicitly signaled. First, a merge candidate list of candidate motion parameters is constructed, and then an index identifying the candidate to be used is signaled.

[0019] The merge candidate list includes a non-sub-CU merge candidate list and a sub-CU merge candidate list. The non-sub-CU merge candidates are constructed based on spatially adjacent motion vectors, co-located temporal motion vectors, and history-based motion vectors. The sub-CU merge candidate list includes affine merge candidates and ATMVP merge candidates. The sub-CU merge candidates are used to derive multiple motion vectors for the current CU, and different portions of samples in the current CU can have different motion vectors.

[0020] Skip mode is used to indicate for a block that motion data is estimated rather than explicitly signaled and that the prediction residual is zero, i.e., no transform coefficients are sent. At the start of each CU of an inter-picture predicted slice, a skip_flag is signaled, which means: Use merge mode to derive motion data No residual data in the bitstream

[0021] Multi-hypothesis intra-inter prediction is a combination of one intra prediction and one merge index prediction, i.e., an intra-inter prediction mode. In a merge CU, for the merge mode, one flag is signaled to select an intra mode from the intra candidate list when the flag is true. For the luma component, the intra candidate list is derived from four prediction modes, including DC mode, planar mode, horizontal mode, and vertical mode. The size of the intra candidate list may be three or four depending on the shape of the block. If the width of the CU is greater than twice the height of the CU, the horizontal mode is removed from the intra mode list, and if the height of the CU is greater than twice the width of the CU, the vertical mode is removed from the intra mode list. A weighted average is used to combine one intra prediction mode selected by the intra mode index and one merge index prediction selected by the merge index. For the chroma component, DM is always applied without additional signaling.

[0022] The weighting for combining predictions is described below. When DC mode or planar mode is selected, or when the width or height of a coding block (CB) is less than 4, equal weighting is applied. For a CB whose width and height are 4 or greater, when horizontal / vertical mode is selected, a CB is first divided vertically / horizontally into four equal-area regions. Each weighting set, denoted as (w_intrai, w_interi), is applied to a corresponding region, where i is 1 to 4, and (w_intra1, w_inter1) = (6, 2), (w_intra2, w_inter2) = (5, 3), (w_intra3, w_inter3) = (3, 5), and (w_intra4, w_inter4) = (2, 6). (w_intra1, w_inter1) are for the region closest to the reference sample, and (w_intra4, w_inter4) are for the region farthest from the reference sample. Then, the combined prediction can be calculated by summing the two weighted predictions and right-shifting 3 bits. Also, if the next neighboring CBs are intra-coded, we store the intra prediction mode used in the predictor's intra hypotheses for their intra-mode coding.

[0023] Currently, for the luma component, in the case of intra-inter prediction mode, up to four intra prediction modes are adopted, which increases the complexity.

[0024] Currently, for intra-inter mode, one buffer is used to store the inter predicted value and one additional buffer is added to store the intra predicted value, which increases the decoder memory cost.

[0025] For the current intra-inter mode, both intra-prediction and inter-prediction must be performed, and the intra-prediction process involves multiplications, which adds complexity to the decoder.

[0026] For the current intra-inter mode, for different block sizes, the weighting is a fixed value, which may not be optimal and may be improved.

[0027] The weights (w_intra, w_inter) applied in multiple hypotheses are selected depending on the relative position of the sample within the block, but this selection may not be reliable or flexible.

[0028] The weights (w_intra, w_inter) applied in multiple hypotheses may be correlated with whether the neighboring blocks are intra- or inter-coded. By considering whether the neighboring blocks above or to the left are intra- or inter-coded, the weights can be selected more efficiently. Summary of the Invention [Problem to be solved by the invention]

[0029] Therefore, a technical solution to such problems is desirable. [Means for solving the problem]

[0030] The present invention includes a method and apparatus, the apparatus including a memory configured to store computer program code and one or more hardware processors configured to access the computer program code and operate according to the instructions of the computer program code, the computer program including selection code configured to cause the processor to select a set of weights in an intra-inter mode, and application code configured to cause the processor to apply position-dependent intra prediction combining (PDPC) to modify at least one of inter-predicted samples and reconstructed samples of inter-coded coding units (CUs).

[0031] According to an exemplary embodiment, the application code is further configured to cause the processor to apply PDPC to modify only the luma component of the inter-coded CU.

[0032] According to an exemplary embodiment, the signaling code is further configured to cause the processor to signal an inter position-dependent intra prediction combining flag interPDPCFlag in response to determining that the coding block (CB) is coded by merge mode.

[0033] According to an exemplary embodiment, the application code is further configured to cause the processor to apply PDPC to the plurality of reconstructed samples before applying any of deblocking, sample adaptive offset (SAO), and in-loop deblocking filters, including adaptive loop filters (ALF).

[0034] According to an exemplary embodiment, the program code further includes assignment code configured to cause the processor to assign a default intra-prediction mode to a current coding block (CB), and the application code is further configured to cause the processor to apply the default intra-prediction mode to a CB next to the current CB for at least intra-mode coding and derivation of a most probable mode (MPM).

[0035] According to an example embodiment, the application code is further configured to cause the processor to apply a set of PDPC filters to at least one of inter-predicted samples other than the inter-predicted samples and reconstructed samples of an inter-coded CU other than reconstructed samples of the inter-coded CU, wherein each PDPC filter in the set of PDPC filters is restricted for a corresponding block size of a plurality of block sizes.

[0036] According to an example embodiment, the signaling code is further configured to cause the processor to signal an interPDPCFlag indicating whether to apply at least one of the PDPC filters; determine whether the interPDPCFlag is set equal to true; in response to determining that the interPDPCFlag is set equal to the first value, signal a second flag, the second flag indicating applying at least one PDPC filter to at least one of the inter-predicted samples and the reconstructed samples of the inter-coded CU after inter prediction; and in response to determining that the interPDPCFlag is set equal to the second value, exclude the PDPC filter from at least one of the inter-predicted samples and the reconstructed samples of the inter-coded CU.

[0037] According to an exemplary embodiment, the program code further includes signaling code configured to cause a processor to select whether weights in the set of weights are equal to each other, and weighting code configured to cause the processor, in response to determining that the weights are not equal, to weight inter prediction in PDPC filtering depending on whether the neighboring blocks are coded by intra coding, inter coding, or intra-inter coding.

[0038] According to an exemplary embodiment, in the intra-inter mode, there is no intra prediction, and the intra-inter mode includes directly applying PDPC on inter prediction, where directly applying PDPC on inter prediction includes:

number

[0039] Other features, nature and various advantages of the disclosed subject matter will become more apparent from the following detailed description and the accompanying drawings, in which: [Figure 1] FIG. 1 is a schematic diagram of a diagram according to an embodiment. [Figure 2] FIG. 1 is a schematic diagram of a diagram according to an embodiment. [Figure 3] FIG. 1 is a schematic diagram of a diagram according to an embodiment. [Figure 4] FIG. 1 is a schematic diagram of a diagram according to an embodiment. [Figure 5] FIG. 1 is a schematic diagram of a diagram according to an embodiment. [Figure 6] FIG. 1 is a schematic diagram of a diagram according to an embodiment. [Figure 7] FIG. 1 is a schematic diagram of a diagram according to an embodiment. [Figure 8] FIG. 1 is a schematic diagram of a diagram according to an embodiment. [Figure 9A] FIG. 1 is a schematic diagram of a diagram according to an embodiment. [Figure 9B] FIG. 1 is a schematic diagram of a diagram according to an embodiment. [Figure 10] 1 is a simplified flowchart according to an embodiment. [Figure 11] 1 is a simplified flowchart according to an embodiment. [Figure 12] 1 is a simplified flowchart according to an embodiment. [Figure 13] 1 is a simplified flowchart according to an embodiment. [Figure 14] FIG. 1 is a schematic diagram of a diagram according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0040] The proposed features described below can be used individually or in any order and combination. Also, embodiments may be implemented by processing circuitry (e.g., one or more processors or one or more integrated circuits). In one example, the one or more processors execute a program stored on a non-transitory computer-readable medium. In this disclosure, most probable mode (MPM) can refer to the primary MPM, the secondary MPM, or both the primary and secondary MPMs.

[0041] 1 shows a simplified block diagram of a communication system 100 according to an embodiment of the present disclosure. The communication system 100 may include at least two terminals 102, 103 connected to each other via a network 105. For one-way data transmission, a first terminal 103 may encode video data locally and transmit it to another terminal 102 via the network 105. A second terminal 102 may receive the encoded video data of the other terminal from the network 105, decode the encoded data, and display the decoded video data. One-way data transmission may be common in media service applications, for example.

[0042] 1 shows a second pair of terminals 101, 104 provided to support bidirectional transmission of encoded video, such as may occur during a video conference. For bidirectional transmission of data, each terminal 101, 104 may encode video data captured at a local location and transmit it to the other terminal over network 105. Each terminal 101, 104 may also receive encoded video data transmitted by the other terminal, decode the encoded data, and display the decoded video data on a local display device.

[0043] In FIG. 1 , terminals 101, 102, 103, and 104 may be illustrated as servers, personal computers, or smartphones, although the principles of the present disclosure are not limited thereto. Embodiments of the present disclosure may also include laptop computers, tablet computers, media players, and / or dedicated videoconferencing equipment. Network 105 represents any number of networks, including, for example, wired and / or wireless communication networks, for transmitting encoded video data among terminals 101, 102, 103, and 104. Communication network 105 may exchange data over circuit-switched and / or packet-switched channels. Such networks may include telecommunications networks, local area networks, wide area networks, and / or the Internet. For purposes of this discussion, the architecture and topology of network 105 may not be important to the operation of the present disclosure, unless otherwise described herein.

[0044] As an example of an application of the subject matter of this disclosure, Figure 2 illustrates a configuration of video encoders and decoders in a streaming environment. The disclosed subject matter may equally be applied to other applications that support video, including, for example, video conferencing, digital TV, and storage of compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0045] The streaming system can include a capture subsystem 203, which may include a video source 201, such as a digital camera, that creates an uncompressed video sample stream 213. The sample stream 213, depicted as a thick line to emphasize its larger amount of data compared to an encoded video bitstream, can be processed by an encoder 202 coupled to the camera 201. The encoder 202 can include hardware, software, or a combination thereof to implement or perform aspects of the disclosed subject matter, as described in more detail below. The encoded video bitstream 204, depicted as a thin line to emphasize its smaller amount of data compared to the sample stream, can be stored on a stream server 205 for future use. One or more streaming clients 212, 207 can access the streaming server 205 to retrieve copies 208, 206 of the encoded video bitstream 204. The client 212 may include a video decoder 210 that decodes an incoming copy 208 of the encoded video bitstream to create an outgoing video sample stream 210 that can be rendered on a display 209 or other rendering device (not shown). In some streaming systems, the video bitstreams 204, 206, 208 may be encoded according to a particular video encoding / compression standard. Examples of these standards are mentioned above and described further herein.

[0046] FIG. 3 may be a functional block diagram of a video decoder 300 according to an embodiment of the present disclosure.

[0047] Receiver 302 can receive one or more codec video sequences to be decoded by decoder 300. In the same or another embodiment, it receives one coded video sequence at a time, with the decoding of each coded video sequence being independent of the other coded video sequences. The coded video sequences can be received from channel 301, which may be a hardware / software link to a storage device that stores the coded video data. Receiver 302 may receive coded video data and other data, such as coded audio data and / or auxiliary data streams, that can be forwarded to respective using entities (not shown). Receiver 302 can separate coded video sequences from other data. To prevent network jitter, buffer memory 303 can be coupled between receiver 302 and entropy decoder / parser 304 (hereinafter referred to as "parser"). If receiver 302 receives data from a storage / forwarding device or an isochronous real-time network with sufficient bandwidth and controllability, buffer memory 303 may not be required, or may be small. To make optimal use of packet networks such as the Internet, buffer memory 303 may be required, which can be relatively large and advantageously have an adaptive size.

[0048] The video decoder 300 may include a parser 304 to reconstruct symbols 313 based on the entropy-coded video sequence. These symbol categories include information for managing the operation of the decoder 300 and potentially information for controlling a rendering device that is not part of the decoder but may be coupled to the decoder, such as a display 312. The control information for the rendering device may be in the form of a Supplemental Enhancement Information (SEI) message or a Video Usability Information (VUI) parameter set fragment (not shown). The parser 304 may parse / entropy decode the received coded video sequence. The coding of the coded video sequence may conform to a video coding technique or standard and may follow principles well known to those skilled in the art, including variable-length coding, Huffman coding, context-dependent or non-context-dependent arithmetic coding, etc. The parser 304 may extract a subgroup parameter set for at least one of the subgroups of pixels in the video decoder from the coded video sequence based on at least one parameter corresponding to the group. The subgroups may include Groups of Pictures (GOPs), pictures, tiles, slices, macroblocks, coding units (CUs), blocks, transform units (TUs), prediction units (PUs), etc. The entropy decoder / parser may also extract, for example, transform coefficients, quantizer parameter QP values, motion vectors, etc. from the coded video sequence information.

[0049] The parser 304 may perform an entropy decoding / parsing operation on the video sequence received from the buffer memory 303 to create symbols 313. The parser 304 may receive the encoded data and selectively decode particular symbols 313. The parser 304 may also determine whether the particular symbol 313 is provided to the motion compensated prediction unit 306, the scaler / inverse transform unit 305, the intra prediction unit 307, or the loop filter 311.

[0050] The reconstruction of symbols 313 may involve several different units, depending on the type of coded or partially coded video picture (e.g., inter-picture vs. intra-picture, inter-block vs. intra-block) and other factors. The units involved, and the manner in which they are involved, may be controlled by subgroup control information parsed from the coded video sequence by parser 304. For simplicity, the flow of such subgroup control information between parser 304 and the following units is not described.

[0051] In addition to the functional blocks already mentioned, the decoder 200 can be conceptually subdivided into a number of functional units, which are described below. In an actual implementation that operates under commercial constraints, many of these units will interact closely with each other and may be at least partially integrated with each other. However, for purposes of illustrating the disclosed subject matter, a conceptual subdivision into the following functional units is appropriate:

[0052] The first unit is a scalar / inverse transform unit 305. The scalar / inverse transform unit 305 may receive quantized transform coefficients as symbols 313 and control information from the parser 304, including the transform method to use, block size, quantization coefficients, quantization scaling matrix, etc. The scalar / inverse transform unit 305 may output blocks containing sample values ​​that may be input to an aggregator 310.

[0053] In some cases, the output samples of the scalar / inverse transform unit 305 may belong to intra-coded blocks, i.e., blocks that do not use prediction information from a previously reconstructed picture, but do use prediction information from a previously reconstructed part of the current picture. Such prediction information can be provided by the intra prediction unit 307. In some cases, the intra prediction unit 307 generates blocks of the same size and shape as the block it is reconstructing using surrounding already reconstructed information extracted from the current (partially reconstructed) picture 309. In some cases, the aggregator 310 adds, based on each sample, the prediction information generated by the intra prediction unit 307 to the output sample information provided by the scalar / inverse transform unit 305.

[0054] In other cases, the output samples of the scalar / inverse transform unit 305 may belong to an inter-coded, potentially motion-compensated block. In such cases, the motion compensation prediction unit 306 may access the reference picture memory 308 to extract samples for prediction. After performing motion compensation on the extracted samples based on symbols 313 belonging to the block, these samples can be added by the aggregator 310 to the output of the scalar / inverse transform unit (in this case, referred to as residual samples or residual signals) to generate output sample information. The addresses in the reference picture memory from which the motion compensation unit extracts prediction samples may be controlled by motion vectors, which can be used by the motion compensation unit in the form of symbols 313, which may have, for example, X, Y, and reference picture components. Motion compensation may include interpolation of sample values ​​extracted from the reference picture memory when sub-sample accurate motion vectors are used, motion vector prediction mechanisms, etc.

[0055] The output samples of aggregator 310 may be processed by various loop filtering techniques in loop filter unit 311. The video compression techniques may include in-loop filtering techniques controlled by parameters contained in the coded video bitstream and available to loop filter unit 311 as symbols 313 from parser 304; however, the video compression techniques may also respond to meta-information obtained during decoding of a coded picture or a previous part of a coded video sequence (in decoding order), or may respond to previously reconstructed loop-filtered sample values.

[0056] The output of the loop filter unit 311 may be a sample stream, which may be output to the rendering device 312 and stored in the reference picture memory 557 for use in future inter-picture prediction.

[0057] Once fully reconstructed, some coded pictures can be used as reference pictures for future prediction. Once a coded picture is fully reconstructed and the coded picture is identified as a reference picture (e.g., by parser 304), the current reference picture 309 can become part of the reference picture buffer 308, and a new current picture memory can be reallocated before starting reconstruction of the next coded picture.

[0058] The video decoder 300 may perform decoding operations in accordance with a predetermined video compression technology, such as that documented in the ITU-T H.265 Recommendation. A coded video sequence may conform to the syntax specified by the video compression technology or standard being used, in the sense that the coded video sequence conforms to the syntax of the video compression technology or standard as specified in the video compression technology document or standard and specifically in the document files therein. Compliance also requires that the complexity of the coded video sequence be within the range limited by the level of the video compression technology or standard. In some cases, the level limits the maximum picture size, maximum frame rate, maximum reconstructed sample rate (e.g., measured in megasamples per second), maximum reference picture size, etc. In some cases, the limits set by the level may be further constrained by a Hypothetical Reference Decoder (HRD) specification and HRD buffer management metadata signaled in the coded video sequence.

[0059] In embodiments, the receiver 302 may receive additional (redundant) data along with the encoded video. The additional data may be included as part of one or more encoded video sequences. The additional data may be used by the video decoder 300 to accurately decode the data and / or more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0060] FIG. 4 may be a functional block diagram of a video encoder 400 according to an embodiment of the present disclosure.

[0061] The encoder 400 may receive video samples from a video source 401 (not part of the encoder), which may capture the video images to be encoded by the encoder 400 .

[0062] The video source 401 may provide a source video sequence in the form of a digital video sample stream to be encoded by the encoder 400. The digital video sample stream may have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, etc.), any color space (e.g., BT.601 Y CrCB, RGB, etc.), and any suitable sampling configuration (e.g., Y CrCb 4:2:0, Y CrCb 4:4:4). In a media serving system, the video source 401 may be a storage device that stores previously prepared video. In a video conferencing system, the video source 401 may be a camera that captures local image information as a video sequence. The video data may be provided as multiple individual pictures that, when viewed sequentially, impart motion. These pictures may themselves be organized as spatial pixel arrays, where each pixel may contain one or more samples, depending on the sampling configuration, color space, etc. used. Those skilled in the art will readily understand the relationship between pixels and samples. The following discussion will focus on samples.

[0063] According to an embodiment, the encoder 400 may encode pictures of a source video sequence and compress them into an encoded video sequence 410 in real time or under any other time constraint required by the application. Performing the appropriate encoding rate is one of the functions of the controller 402. The controller controls and is functionally coupled to other functional units described below. For simplicity, the coupling is not depicted. Parameters set by the controller may include rate control-related parameters (picture skip, quantizer, λ value for rate-distortion optimization techniques, etc.), picture size, picture group GOP layout, maximum motion vector search range, etc. Those skilled in the art will readily recognize other functions of the controller 402, as these may belong to a video encoder 400 optimized for a particular system design.

[0064] Some video encoders operate in what those skilled in the art will readily recognize as a "coding loop." As a highly simplified description, the encoding loop may include an encoder 402 (hereafter referred to as the "source encoder") encoding portion (responsible for constructing symbols based on an input picture to be coded and one or more reference pictures) and a (local) decoder 406 embedded in the encoder 400, which reconstructs the symbols to produce sample data that is also produced by a (remote) decoder (because the compression between the symbols and the coded video bitstream is lossless in the video compression techniques considered in the disclosed subject matter). The reconstructed sample stream is input into a reference picture memory 405. Because decoding the symbol stream produces bit-accurate results independent of the location of the decoder (local or remote), the contents of the reference picture buffer are also bit-accurate between the local and remote encoders. That is, the reference picture samples that the predictive portion of the encoder "sees" are identical to the sample values ​​that the decoder "sees" when using prediction during decoding. The basic principle of reference picture synchronization (and the drift that occurs when synchronization cannot be maintained, for example due to channel errors) is well known to those skilled in the art.

[0065] The operation of the "local" decoder 406 may be the same as the operation of the "remote" decoder 300 described in detail above with reference to Figure 3. However, and again with brief reference to Figure 4, the entropy decoding portion of the decoder 300, including the channel 301, receiver 302, buffer memory 303 and parser 304, may not be fully implemented in the local decoder 406, provided that symbols are available and the entropy coder 408 and parser 304 are able to losslessly encode / decode the symbols into an encoded video sequence.

[0066] In this case, in addition to the analysis / entropy decoding present in the decoder, any decoder techniques will necessarily be found to exist in the corresponding encoder in essentially the same functional form. The description of the encoder technique can be simplified because the encoder technique and the fully described decoder technique are the inverse of each other. Further details are only necessary in certain areas and are provided below.

[0067] As part of the operation of the source encoder 402, the source encoder 403 may perform motion-compensated predictive coding, which predictively codes an input frame by referencing one or more previously coded frames from the video sequence designated as "reference frames." In this manner, the coding engine 407 may code differences between pixel blocks of the input frame and pixel blocks of reference frames that may be selected as predictive references for the input frame.

[0068] The local video decoder 406 may decode the encoded video data of frames that may be designated as reference frames based on symbols created by the source encoder 403. The operation of the encoding engine 407 may advantageously be a lossy process. When the encoded video data can be decoded by a video decoder (FIG. 4, not shown), the reconstructed video sequence will typically be a copy of the source video sequence, with some errors. The local video decoder 406 may replicate the decoding process that may be performed by the video decoder on the reference frames and store the reconstructed reference frames in the reference picture cache 405. In this way, the encoder 400 can locally store copies of the reconstructed reference frames that have common content (free of transmission errors) with the reconstructed reference frames obtained by the remote video decoder.

[0069] The predictor 404 may perform a prediction search for the coding engine 407. That is, for a new frame to be coded, the predictor 404 may search the reference picture memory 405 for sample data (as candidate reference pixel blocks) that can be used as suitable prediction references for the new picture, or specific metadata, such as the reference picture's motion vectors, block shape, etc. The predictor 404 may operate pixel block by pixel block based on the sample blocks to find suitable prediction references. In some cases, for example, as determined by the search results obtained by the predictor 404, the input picture may have prediction references extracted from multiple reference pictures stored in the reference picture memory 405.

[0070] The controller 402 may manage the encoding operations of the source encoder 403, including, for example, setting parameters and subgroup parameters for encoding the video data.

[0071] The output of all the functional units mentioned above may be entropy coded in entropy coder 408. The entropy coder converts the symbols produced by each functional unit into an encoded video sequence by performing lossless compression on the symbols according to techniques well known to those skilled in the art (e.g., Huffman coding, variable length coding, arithmetic coding, etc.).

[0072] The transmitter 409 may buffer one or more encoded video sequences created by the entropy encoder 408 in preparation for transmission over a communication channel 411, which may be a hardware / software link to a storage device that stores the encoded video data. The transmitter 409 may merge the encoded video data from the source encoder 403 with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (sources not shown).

[0073] The controller 402 may manage the operation of the encoder 400. During encoding, the controller 405 may assign to each coded picture a particular coded picture type, which may affect the coding technique that may be applied to the corresponding picture. For example, a picture may typically be assigned as one of the following picture types:

[0074] An intra picture (I-picture) may be a picture that is coded and decoded without using any other frame in the sequence as a source of prediction. Some video codecs allow different types of intra pictures, including, for example, Independent Decoder Refresh pictures. Those skilled in the art are aware of these variations of I-pictures and their corresponding uses and characteristics.

[0075] A predicted picture (P picture) may be a picture that is coded and decoded using intra- or inter-prediction, whereby sample values ​​for each block can be predicted using at most one motion vector and reference index.

[0076] A bidirectionally predictive picture (B-picture) may be a picture that is encoded and decoded using intra- or inter-prediction to predict the sample values ​​of each block using at most two motion vectors and reference indices. Similarly, a multiple-prediction picture may use more than two reference images and associated metadata to reconstruct a single block.

[0077] A source picture is typically spatially subdivided into multiple sample blocks (e.g., blocks of 4x4, 8x8, 4x8, or 16x16 samples each) and may be coded block by block. A block may be predictively coded with reference to other (already coded) blocks determined by the coding assignment applied to the block's corresponding picture. For example, a block of an I-picture may be non-predictively coded or predictively coded with reference to coded blocks of the same picture (spatial prediction or intra prediction). A pixel block of a P-picture may be non-predictively coded via spatial or temporal prediction with reference to one previously coded reference picture. A block of a B-picture may be non-predictively coded via spatial or temporal prediction with reference to one or two previously coded reference pictures.

[0078] The video encoder 400 may perform encoding operations based on a predetermined video encoding technique or standard, such as the ITU-T H.265 recommendation. During operation, the video encoder 400 may perform various compression operations, including predictive coding operations that exploit temporal and spatial redundancies in the input video sequence. Thus, the encoded video data may conform to a syntax specified by the video encoding technique or standard being used.

[0079] In an embodiment, the transmitter 409 may transmit additional data along with the encoded video. The source encoder 403 may include such data as part of the encoded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant pictures, slices, Supplementary Enhancement Information (SEI) messages, Visual Usability Information (VUI) parameter set fragments, etc.

[0080] Figure 5 shows the intra-prediction modes used in HEVC and JEM. To capture any edge direction present in natural video, the number of directional intra-modes is expanded from 33 used in HEVC to 65. The additional directional modes in JEM over HEVC are shown as dotted arrows in Figure 5, while the planar and DC modes remain the same. These denser directional intra-prediction modes apply to all block sizes and both luma and chroma intra-prediction. As shown in Figure 5, directional intra-prediction modes associated with odd-numbered intra-prediction mode indexes and marked with dotted arrows are referred to as odd intra-prediction modes. Directional intra-prediction modes associated with even-numbered intra-prediction mode indexes and marked with solid arrows are referred to as even intra-prediction modes. In this application, the directional intra-prediction modes indicated by solid or dotted arrows in Figure 5 are also referred to as angular modes.

[0081] In JEM, a total of 67 intra prediction modes are used for luma intra prediction. To encode in intra mode, an MPM list of size 6 is constructed based on the intra modes of neighboring blocks. If an intra mode is not in the MPM list, a flag is signaled to indicate whether the intra mode belongs to the selected mode. In JEM-3.0, there are 16 selection modes, which are uniformly selected as four angular modes. In JVET-D0114 and JVET-G0060, 16 secondary MPMs are derived to replace the uniformly selected modes.

[0082] 6 shows N reference layers employed for the intra-directional mode. There are block unit 611, segment A 601, segment B 602, segment C 603, segment D 604, segment E 605, segment F 606, first reference layer 610, second reference layer 209, third reference layer 608 and fourth reference layer 607.

[0083] In both HEVC and JEM, as well as other standards such as H.264 / AVC, the reference samples used to predict the current block are limited to the nearest reference line (row or column). In the multiple reference line intra prediction method, for the intra-directive mode, the number of candidate reference lines (rows or columns) increases from 1 (i.e., the nearest) to N, where N is an integer equal to or greater than 1. Figure 2 illustrates the concept of the multiple line intra-directional prediction method using a 4x4 prediction unit (PU) as an example. The intra-directional mode can arbitrarily select one of N reference layers to generate a predictor. In other words, the predictor p(x, y) is generated from one of the reference samples S1, S2, ..., S N. A flag is signaled to indicate the reference layer selected for the intra-directive mode. If N is set to 1, the intra-directional mode method is the same as the conventional method of JEM2.0. 6, reference lines 610, 609, 608, and 607 include six segments 601, 602, 603, 604, 605, and 606, along with an upper-left reference sample. In this application, a reference layer is also referred to as a reference line. The coordinates of the upper-left pixel in the current block unit are (0, 0), and the coordinates of the upper-left pixel of the first reference line are (-1, -1).

[0084] In JEM, for the luma component, neighboring samples used to generate intra-predicted samples are filtered before generation. The filtering is controlled by the specified intra-prediction mode and transform block size. If the intra-prediction mode is DC or the transform block size is equal to 4x4, neighboring samples are not filtered. If the distance between the specified intra-prediction mode and the vertical mode (or horizontal mode) is greater than a predefined threshold, the filtering process is started. A [1, 2, 1] filter and a bilinear filter are used to filter neighboring samples.

[0085] The position dependent intra prediction combination (PDPC) method is an intra prediction method that combines filtered boundary reference samples, HEVC-style intra prediction, with unfiltered boundary reference samples. Each prediction sample pred[x][y] at (x, y) is calculated as follows:

number

[0086] Figure 7 shows a diagram 700 in which DC mode PDPC assigns weights (wL, wT, wTL) to the (0,0) and (1,0) positions within a 4x4 block. When applying PDPC to DC intra mode, planar intra mode, horizontal intra mode, and vertical intra mode, no additional boundary filters are required, such as the HEVC DC mode boundary filter or horizontal / vertical mode edge filter. Figure 7 shows a reference sample R of PDPC applied to the top right diagonal mode. x、-1、 R -1、y and R -1、-1 The definition of is shown below. The predicted sample pred(x', y') is located at (x', y') in the predicted block. The reference sample R x、-1 The coordinate x of is given by x=x'+y'+1, and the reference sample R -1、yThe coordinate y of is similarly given by y=x'+y'+1.

[0087] 8 shows a Local Illumination Compensation (LIC) diagram 800, which is based on a linear model of illumination changes with a scaling factor a and an offset b, and is adaptively enabled or disabled per coding unit (CU) for inter mode coding.

[0088] When LIC is applied to a CU, the parameters a and b are derived using the neighboring samples of the current CU and its corresponding reference samples using the least squares error method. More specifically, as shown in Figure 8, the subsampled (2:1 subsampled) neighboring samples of the CU and the corresponding samples in the reference picture (identified by the motion information of the current CU or sub-CU) are used. IC parameters are derived and applied to each prediction direction separately.

[0089] If the CU is coded in merge mode, copy the LIC flag from the neighboring block, similar to copying motion information in merge mode; otherwise, signal the LIC flag to the CU to indicate whether LIC is applied.

[0090] 9A shows intra-prediction modes 900 used in HEVC. In HEVC, there are a total of 35 intra-prediction modes, of which mode 10 is the horizontal mode, mode 26 is the vertical mode, and modes 2, 18, and 34 are the diagonal modes. The intra-prediction modes are signaled by three most probable modes (MPMs) and the remaining 32 modes.

[0091] 9B shows an embodiment of a VVC implementation in which there are a total of 87 intra-prediction modes, where mode 18 is the horizontal mode, mode 50 is the vertical mode, and modes 2, 34, and 66 are diagonal modes. Modes -1 to -10 and modes 67 to 76 are referred to as Wide-Angle Intra Prediction (WAIP) modes.

[0092] According to the PDPC formula, a linear combination of intra prediction modes (DC, planar, angular) and reference samples is used to predict a prediction sample pred(x, y) at position (x, y). pred(x, y) = (wL × R -1、y +wT×R x、-1 -wTL×R -1、-1 +(64-wL-wT+wTL)×pred(x,y)+32)≫6 where R x、-1 , R -1、y denote the reference samples above and to the left of the current sample (x, y), respectively, and R -1、-1 denotes the reference sample in the top left corner of the current block.

[0093] In DC mode, for a block with dimensions width and height, the weights are calculated as follows: wT=32≫((y≪1)≫nScale),wL=32≫((x≪1)≫ nScale),wTL=(wL≫4)+(wT≫4) where nScale = (log²(width) - 2 + log²(height) - 2 + 2) >> 2, wT indicates the weighting factor of the reference sample on the reference line above with the same horizontal coordinate, wL indicates the weighting factor of the reference sample on the reference line to the left with the same vertical coordinate, and wTL indicates the weighting factor of the top-left reference sample of the current block. nScale specifies the rate at which the weighting factor decreases along the axis (wL decreases from left to right, or wT decreases from top to bottom), i.e., the rate at which the weighting factor decreases. In the current design, the rate at which the weighting factor decreases is the same along the x (left to right) and y (top to bottom) axes. Also, 32 indicates the initial weighting factor of the neighboring sample. The initial weighting factor is also the upper (left or top-left) weighting assigned to the top-left sample of the current CB. The weighting factor of the neighboring sample in the PDPC processing must be less than or equal to this initial weighting factor.

[0094] For planar mode, wTL = 0, for horizontal mode, wTL = wT, and for vertical mode, wTL = wL. The PDPC weights can be calculated using only additions and shifts. We can calculate the value of pred(x, y) in one step using equation (1).

[0095] 10 shows a flowchart 1000 according to an example embodiment, in which in S1, PDPC may be applied to reconstructed or inter-predicted samples of an inter-coded CU. When PDPC is applied to the reconstructed or inter-predicted samples of the inter-coded CU, the PDPC process further modifies the reconstructed or predicted samples of the inter-coded CU.

[0096] In S2, a flag, for example, interPDPCFlag, is signaled to indicate whether PDPC is applied. According to an embodiment, interPDPCFlag can be signaled at the CU level to indicate whether PDPC is applied to reconstructed or inter-predicted samples of an inter-coded CU, and according to another embodiment, interPDPCFlag can be signaled at the TU level to indicate whether PDPC is applied to reconstructed or inter-predicted samples of an inter-coded CU. Also, interPDPCFlag is signaled only if it is determined that the current CU is neither an intra-coded CU nor a skip CU.

[0097] According to an embodiment, in S2, interPDPCFlag may be signaled only for coding blocks coded by merge mode, and the flag may be signaled before or after the merge index, thereby, according to an embodiment, preventing PDPC from being applied to skip CUs and / or preventing PDPCl from being applied to either skip CUs or sub-block merge CUs.

[0098] In S3, after reconstruction or inter prediction of the inter-coded CU, consider whether interPDPCFlag is set equal to true, and if so, apply PDPC to the reconstructed samples or inter-predicted samples of the inter-coded CU in S4 to generate reconstructed samples or final predicted values ​​of the inter-coded CU.

[0099] According to an embodiment, in S4, when PDPC is applied to the reconstructed samples, the PDPC is applied to the reconstructed samples before applying any in-loop deblocking filters (e.g., deblocking, Sample Adaptive Offset (SAO), and Adaptive Loop Filter (ALF)).

[0100] According to an embodiment, if interPDPCflag is set equal to true in S3, then deblocking and SAO are not applied to the top and left boundaries of the current CU in S4, and according to another embodiment, if interPDPCflag is set equal to true in S3, then any other filter other than PDPC is not applied to the top and left boundaries of the current CU.

[0101] Otherwise, in S5, after reconstruction or inter-prediction of the inter-coded CU, PDPC may not be applied to the reconstructed or inter-predicted samples of the inter-coded CU, and instead, normal inter-prediction processing may be used.

[0102] According to an embodiment, PDPC may be determined and applied only to the luma component of an inter-coded CU.

[0103] According to an embodiment, when PDPC is applied to the reconstructed samples or inter-predicted samples of the inter-coded CU of the current CB in S6, it can be determined in S7 whether to assign a default intra-prediction mode to the current CB, so that the next CB can use the default intra-prediction mode for intra-mode coding and Most Probable Mode (MPM) derivation. For example, the default intra-prediction mode can be either planar mode or DC mode.

[0104] In S8, it can be determined whether to use different initial weighting factors for PDPC for normal intra in S5 and PDPC in S4, and if so, in S9, the weighting factor is set to 16, 8, 4, 2, or 1 for PDPC in S4. According to an embodiment, when PDPC is applied to reconstructed samples or inter-predicted samples of an inter-coded CU of the current CB, the intra-prediction mode is marked as unavailable, and for the next neighboring block, the current CB is not considered an intra-coded block, and for the next neighboring block, the current CB cannot be used for intra-mode coding or MPM derivation.

[0105] According to an embodiment, when applying PDPC to the reconstructed samples or inter-predicted samples of the inter-coded CU in S5, the weighting factor reduction rate and / or the initial weighting factor are taken into account in S10, and in S11 the weighting factor reduction rate and / or the initial weighting factor are set to a different value than the value used in the original PDPC for intra prediction.

[0106] According to an embodiment, in S11, the weighting factor reduction rate may be set equal to a fixed value N, independent of the block size of the current coding unit, where N may be any integer greater than or equal to 0. For example, N may be set equal to 0, 1, 2, 3, or 4.

[0107] According to an embodiment, in S11, if the block size (width*height) is less than or equal to Thres_1, the weighting factor reduction rate may be set equal to a fixed value N1. Otherwise, the weighting factor reduction rate may be set equal to another fixed value N2. N1 and N2 may be any integers greater than or equal to 0. For example, N1 is set equal to 1, and N2 is set equal to 2. Thres_1 may be any positive integer, for example, 32, 64, 128, 256, 512, or 1024.

[0108] According to an embodiment, in S11, the weighting factor reduction rate may be set equal to ((Log2(nTbW)+Log2(nTbH)+S)≫2), where S may be any integer, for example, −3, −2, −1, 0, 1, 2, or 3.

[0109] According to an embodiment, when applying PDPC to inter-predicted samples in S11, the weighting factor reduction rate may have different values ​​for the x-axis (from left to right) and the y-axis (from top to bottom). For example, the weighting factor reduction rate along the x-axis may be related to the width of the current CB, such as Log2(nTbH)+S1)≫1, and the weighting factor reduction rate along the y-axis may be related to the height of the current CB, such as Log2(nTbW)+S2)≫1. S1 and S2 may be any integers, -3, -2, -1, 0, 1, 2, or 3.

[0110] Also, in embodiments according to FIG. 10, high level syntax elements, including but not limited to SPS (Sequence Parameter Set), PPS (Picture Parameter Set), slice header, CTU header may signal whether PDPC is allowed to be applied on inter-predicted or inter-reconstructed blocks, and one flag may be signaled to indicate whether PDPC is allowed to be applied on luma inter-predicted or inter-reconstructed blocks, and another flag may be signaled to indicate whether PDPC is allowed to be applied on chroma inter-predicted or inter-reconstructed blocks.

[0111] 11 shows a flowchart 1100 according to an example embodiment, which is according to or different from the embodiment of FIG. 10, except that S1, S2, S3, and S4 are similar, and if it is determined in S3 that the flag is set equal to true, a second flag or index can be signaled in S12. The flowchart 1100 relates to applying a set of multiple PDPC filters to inter-predicted samples (or reconstructed samples of an inter-coded CU). The set of multiple PDPC filters can be limited for each block size, and when applying PDPC to inter-predicted samples or reconstructed samples of an inter-coded CU, the predicted samples of the PDPC process can be modified to the inter-predicted samples or reconstructed samples of an inter-coded CU. The selection of a PDPC filter can be signaled for a coding block to which PDPC can be applied.

[0112] At S12, according to an embodiment, one flag, namely, interPDPCFlag, may be signaled to indicate whether PDPC is applied at S2. At S3, if interPDPCFlag is set equal to true, a second flag or index may be signaled to indicate the use of a PDPC filter after inter prediction. The specified filter type may be applied to the inter-predicted samples or reconstructed samples of the inter-coded CU to generate a final inter-predicted value or reconstructed samples of the inter-coded CU.

[0113] At S13, according to an embodiment, there may be N different PDPC filter types, and these N different PDPC filter types may be signaled by unary truncation coding or fixed length coding, where N may be any positive integer, for example 2, 3 or 4.

[0114] In S13, according to an embodiment, the number of PDPC filter types may depend on coding information including, but not limited to, the width of the block, the height of the block, the width-to-height ratio of the block, the area size of the block, whether the current block is uni-predicted or uni-predicted, and whether the current block is coded using MERGE mode or AMVP mode.

[0115] In S13, according to an embodiment, different PDPC filter types may be specified by weighting factor reduction rates or scaling factors, and / or initial weighting factors for the upper and / or left sides, and / or an upper-left weighting assigned to the upper-left sample of the current block. For example, the weighting factor reduction rate may be set to ((Log2(nTbW)+Log2(nTbH)-2+S)>>2), where S is any integer and has different settings for different PDPC filter types. In an embodiment, for each PDPC filter type, the weighting factor reduction rate may be set to N, where N may be any integer and may have different settings for different PDPC filter types. In an embodiment, the derivation of the weighting factor reduction rate may use the method described when there are N different PDPC filter types, and the initial value of the weighting factor may be 32, 16, 8, 4, 2, 1, or 0. In other embodiments, only the initial values ​​of the weighting coefficients may be different for different PDPC filter types, and example values ​​for the initial values ​​of the weighting coefficients may include, but are not limited to, 32, 16, 8, 4, 2, 1, or 0.

[0116] Also, like S6 and S7, S13 assigns a default intra prediction mode to the current CB when applying PDPC to an inter predicted sample or a reconstructed sample of an inter coded CU, so that the next CB can use the default intra prediction mode to derive the MPM, and the default intra prediction mode may be a planar mode or a DC mode.

[0117] 12 shows a flowchart 1200 related to multi-hypothesis intra-inter prediction, also referred to as intra-inter mode or inter-intra mode, or a combination of merge and intra prediction modes. Weighting that applies the same weighting to intra prediction and inter prediction is called equal weighting, for example, (w_intra, w_inter)=(1, 1) or (w_intra, w_inter)=(4, 4) is equal weighting, where w_intra is the weighting applied to intra prediction and w_inter is the weighting applied to inter prediction.

[0118] In S14, a set of weights is determined for the weights applied to the intra-inter mode, i.e., w_intra and w_inter, and a process of signaling a weight selection for a block coded by the intra-inter mode is initiated. In S14, parameters of neighboring blocks can be checked, and in S15, a first flag can be signaled to indicate whether equal weights are used. According to an embodiment, the context for entropy coding the first flag depends on whether the neighboring block is intra-coded, inter-coded, or intra-inter-coded. According to an embodiment, the context for entropy coding the first flag depends on whether one of the neighboring blocks above and to the left is intra-coded. According to an embodiment, the context for entropy coding the first flag depends on whether one of the neighboring blocks above and to the left is intra-coded or intra-inter-coded. According to an embodiment, the context for entropy coding the first flag depends on whether one of the upper and left neighboring blocks is inter-coded.

[0119] For example, in S16, it may be determined whether a certain condition is met: if a first flag is signaled with a value indicating that equal weighting is not applied, then weighting is selected depending on whether the neighboring blocks are intra-coded, inter-coded, or intra-inter-coded.

[0120] According to an exemplary embodiment, if any of the exemplary conditions described below are met, then in S17, one weighting is applied, for example, exemplary candidate weightings (w_intra, w_inter)=(3, 1), and if any of the exemplary conditions described below are not met, then in S18, one weighting is applied, for example, exemplary candidate weightings (w_intra, w_inter)=(1, 3).

[0121] The condition in S16 may be considered depending on whether the context for entropy coding the first flag depends on whether both the upper adjacent block and the left adjacent block are intra-coded, whether the context for entropy coding the first flag depends on whether both the upper adjacent block and the left adjacent block are intra-coded or intra-inter-coded, or whether the context for entropy coding the first flag depends on whether both the upper adjacent block and the left adjacent block are inter-coded. However, according to an exemplary embodiment, if the context for entropy coding the first flag depends on whether both the upper adjacent block and the left adjacent block are inter-coded, the weights may be set to (w_intra, w_inter)=(1, 3), in contrast to the other embodiments described above.

[0122] In S19, a second flag can be signaled to indicate which of the set of candidate weights to use for weighting. According to an embodiment, the context for entropy coding of the second flag depends on whether the neighboring blocks are intra-coded, inter-coded, or intra-inter-coded. For example, if both the upper neighboring block and the left neighboring block are intra-coded, the context value can be set to a first value; otherwise, if either the upper neighboring block and the left neighboring block are intra-coded, the context value can be set to a second value; otherwise, if neither the upper neighboring block nor the left neighboring block is intra-coded, the context value can be set to a third value. According to another embodiment, if both the upper neighboring block and the left neighboring block are intra-coded or intra-inter-coded, the context value can be set to a first value; otherwise, if one of the upper neighboring block and the left neighboring block is intra-coded or intra-inter-coded, the context value can be set to a second value; otherwise, if neither the upper neighboring block nor the left neighboring block is intra-coded, the context value can be set to a third value. According to an embodiment, the set of candidate weights may include (w_intra, w_inter)=(1, 3) and (w_intra, w_inter)=(3, 1).

[0123] FIG. 13 shows a flowchart 1300, in which in S20, intra prediction is not used in intra-inter mode, and PDPC is applied directly on inter prediction; however, the weighting applied to inter prediction in PDPC filtering depends on whether the neighboring blocks are intra-coded, inter-coded, or intra-inter-coded.

[0124] According to an embodiment, in S21, PDPC is applied directly on top of inter prediction as follows:

number

[0125] where InterPred is the input inter prediction, and w_intra is selected from a set of predefined values, the selection depending on whether the neighboring block is intra-coded, inter-coded, or intra-inter-coded, and example values ​​of N include 8, 16, 32, 64, 128, and 256. If the neighboring block satisfies the condition, a weighting of 1 is applied in S22, and if the neighboring block does not satisfy the condition, a weighting of 2 is applied in S23.

[0126] According to an embodiment, in S21, PDPC is applied directly on top of inter prediction as follows:

number

[0127] According to an embodiment, in S21, PDPC is applied directly on top of inter prediction as follows:

number

number

[0128] According to an embodiment, in S21, the weighting applied to the above reference sample (wT) in PDPC depends on whether the upper neighboring block is intra-coded, inter-coded, or intra-inter-coded, and the weighting applied to the left reference sample (wL) in PDPC depends on whether the left neighboring block is intra-coded, inter-coded, or intra-inter-coded.

[0129] According to an embodiment, in S21, PDPC is applied directly on top of inter prediction as follows:

number

[0130] According to an embodiment, in S21, PDPC is applied directly on top of inter prediction as follows:

number

[0131] Therefore, according to the exemplary embodiments described herein, the technical solutions can advantageously improve the above technical problems.

[0132] The techniques described above may be implemented as computer software with computer-readable instructions and physically stored on one or more computer-readable media. For example, Figure 14 illustrates a computer system 1400 suitable for implementing some embodiments of the disclosed subject matter.

[0133] Computer software can be encoded in any suitable machine code or computer language, which can be subjected to mechanisms such as assembling, compiling, linking, etc. to create code containing instructions that can be executed directly by a computer central processing unit (CPU), graphics processing unit (GPU), etc., or interpreted, executed by microcode, etc.

[0134] The instructions may be executed by various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, Internet of Things devices, etc.

[0135] 14 is exemplary in nature and is not intended to imply any limitation on the scope or functionality of the computer software used to implement embodiments of the present disclosure. Neither the arrangement of components should be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in the exemplary embodiment of computer system 1400.

[0136] Computer system 1400 may include several human-machine interface input devices. Such human-machine interface input devices may respond to input by one or more human users, such as tactile input (e.g., keystrokes, slides, data glove movements), audio input (e.g., voice, hand claps), visual input (e.g., posture), and olfactory input (not shown). Human-machine interface devices may also be used to capture certain media not necessarily directly related to conscious human input, such as audio (e.g., voice, music, ambient sounds), images (e.g., scanned images, photographic images obtained from static image capture devices), and video (e.g., two-dimensional video, three-dimensional video including stereo video).

[0137] The human-machine interface input devices may include one or more of a keyboard 1402, a mouse 1404, a touchpad 1403, a touch panel 1410, a joystick 1405, a microphone 1406, a scanner 1408, and an imaging device 1407 (only one of each is shown).

[0138] The computer system 1400 may also include human-machine interface output devices. Such human-machine interface output devices may stimulate one or more of the human user's senses, for example, through tactile output, sound, light, and smell / taste. Such human-machine interface output devices may include haptic output devices (e.g., touchscreen 1410 or haptic feedback devices via joystick 1405, although haptic feedback devices that are not used as input devices also exist), audio output devices (e.g., speakers 1409, headphones (not shown)), visual output devices (e.g., screen 1410, virtual reality glasses (not shown), holographic displays, and smoke tanks (not shown), including CRT screens, LCD screens, plasma screens, and OLED screens, each of which may or may not have touchscreen input and haptic feedback capabilities, some of which may be capable of outputting two-dimensional visual output or three-dimensional or higher-dimensional output, such as by means of stereoscopic output), and printers (not shown). A graphics adapter 1429 may be provided for interfacing to various input devices.

[0139] Computer system 1400 may further include human-accessible storage devices and associated media, including, for example, optical media including CD / DVD ROM / RW 1412 with media 1411 such as CD / DVD, thumb drive 1413, removable hard drive or solid state drive 1414, traditional magnetic media (e.g., magnetic tape and floppy disks (not shown)), dedicated ROM / ASIC / PLD based devices (e.g., dongles (not shown)), etc.

[0140] Those skilled in the art should understand that the term "computer-readable medium" as used in conjunction with the presently disclosed subject matter does not include transmission media, carrier waves, or other transient signals.

[0141] The computer system 1400 may further include an interface to one or more communications networks 1415. The network 1415 may be, for example, wireless, wired, or optical. The network 1415 may also be a local, wide-area, metropolitan, vehicular, industrial, real-time, delay-tolerant network, etc. Examples of the network 1415 include local area networks such as Ethernet; cellular networks including WLAN, GSM, 3G, 4G, 5G, LTE, etc.; television wired or wireless wide-area digital networks including cable television, satellite television, and terrestrial television; and vehicular and industrial networks including CANbus. Some networks 1415 typically require an external network interface adapter 1427 connected to a general-purpose data port or peripheral bus (1425) (e.g., a USB port on the computer system 1400), while other networks are typically integrated into the core of the computer system 1400 by connecting to a system bus described below (e.g., an Ethernet interface to a PC computer system or a cellular network interface to a smartphone computer system). Through any of these networks, the computer system 1400 can communicate with other entities. Such communications may be one-way receive only (e.g., broadcast television), one-way transmit only (e.g., a CANbus to a CANbus device), or two-way (e.g., over a local or wide area digital network to reach another computer system). Specific protocols and protocol stacks may be utilized for each of these networks and network interfaces 1415 and 1427, as described above.

[0142] The above-mentioned human-machine interface devices, human-accessible storage devices and network interface 1427 may be connected to core 1416 of computer system 1400 .

[0143] The core 1412 includes one or more central processing units (CPUs) 1421, graphics processing units (GPUs) 1422, specialized programmable processing units in the form of field programmable gate arrays (FPGAs) 1424, hardware accelerators for certain tasks 1426, etc. These devices, along with read-only memory (ROM) 1419, random access memory 1418, and internal mass storage devices 1420 such as internal non-user-accessible hard disk drives, SSDs, etc., are connected via a system bus 1428. In some computer systems, access to the system bus 1426 in the form of one or more physical plugs may allow expansion with additional CPUs, GPUs, etc. Peripheral devices connect to the core's system bus 1428 directly or via a peripheral bus 1401. Peripheral bus architectures include PCI, USB, etc.

[0144] The CPU 1421, GPU 1422, FPGA 1424, and accelerator 1426 may execute instructions that, when combined, constitute the computer code referred to above. The computer code may be stored in the ROM 1419 or RAM 1418. Temporary data may be stored in the RAM 1418, while permanent data may be stored, for example, in the internal mass storage device 1420. A cache memory may provide fast storage and retrieval from any of the memory devices, and the cache memory may be closely associated with one or more of the CPU 1421, GPU 1422, mass storage device 1420, ROM 1419, RAM 1418, etc.

[0145] The computer-readable medium may have computer code thereon for performing various computer-implemented operations, and the medium and computer code may be specially designed and constructed for the purposes of this disclosure, or may be of the type known and available to those skilled in the art of computer software.

[0146] By way of example and not limitation, the computer system 1400 having the architecture, and in particular the core 1416, can provide functionality by having a processor (including a CPU, GPU, FPGA, accelerator, etc.) execute software embodied in one or more tangible computer-readable media. Such computer-readable media may include the user-accessible mass storage media discussed above, as well as some storage of the core 1416 that is non-transitory, such as the core's internal mass storage 1420 or ROM 1419. Software for implementing embodiments of the present disclosure may be stored on such devices and executed by the core 1416. Depending on particular needs, the computer-readable media may include one or more storage devices or chips. The software may cause the core 1416, and in particular its processor (including a CPU, GPU, FPGA, etc.), to execute specific processes or portions of specific processes described herein, define data structures stored in RAM 1418, and modify such data structures based on the software-defined processes. Additionally or alternatively, the computer system may provide functionality implemented in hardwired or otherwise circuitry (e.g., accelerator 1426) that can operate in place of or in conjunction with software to perform particular processes or portions of particular processes described herein. Where appropriate, references to software may include logic, and conversely, references to logic may include software. Where appropriate, references to computer-readable media may include circuitry (e.g., integrated circuits (ICs)) on which software is stored for execution, circuitry that embodies logic for execution, or both. The present disclosure includes any appropriate combination of hardware and software.

[0147] While this disclosure has described certain exemplary embodiments, there are alterations, substitutions, and various substitute equivalents that fall within the scope of this disclosure. It should thus be understood that many systems and methods that, although not explicitly shown or described herein, embody the principles of this disclosure and therefore are within the spirit and scope of the present disclosure, may be contemplated by those skilled in the art.

[0148] (Appendix 1) 1. A method for video decoding executed by at least one processor, comprising: selecting a set of weights in an intra-inter mode; applying position-dependent intra prediction combining (PDPC) to modify at least one of inter predicted samples and reconstructed samples of inter coded coding units (CUs); A method comprising: (Appendix 2) 2. The method of claim 1, wherein applying PDPC includes modifying only the luma component of the inter-coded CU. (Appendix 3) 3. The method of claim 1 or 2, further comprising signaling an interPDPC flag (interPDPCFlag) in response to determining that the coding block (CB) is coded in merge mode. (Appendix 4) 2. The method of claim 1, further comprising applying PDPC to the plurality of reconstructed samples before applying any of deblocking, sample adaptive offset (SAO), and in-loop deblocking filters, including adaptive loop filters (ALF). (Appendix 5) assigning a default intra-prediction mode to the current coding block (CB); applying the default intra-prediction mode to a CB next to the current CB for at least intra-mode coding and derivation of a most probable mode (MPM); 5. The method according to any one of claims 1 to 4, further comprising: (Appendix 6) applying a set of PDPC filters to at least one of inter-predicted samples other than the inter-predicted samples and reconstructed samples of inter-coded CUs other than reconstructed samples of the inter-coded CUs; 2. The method of claim 1, wherein each PDPC filter in the set of PDPC filters is defined for a corresponding block size of a plurality of block sizes. (Appendix 7) signaling at least one of the interPDPC flags (interPDPCFlag) indicating whether PDPC applies; determining whether the interPDPCFlag is set equal to true; signaling a second flag in response to determining that the interPDPCFlag is set equal to a first value, the second flag indicating applying at least one PDPC filter to at least one of the inter-predicted samples and the reconstructed samples of the inter-coded CU after inter prediction; excluding a PDPC from at least one of inter predicted samples and reconstructed samples of an inter coded CU in response to determining that the interPDPCFlag is set equal to a second value; 7. The method of claim 6, further comprising: (Appendix 8) signaling whether the weights are equal to one another; in response to determining that the signaling indicates that the weightings are unequal, weighting inter prediction in PDPC filtering depending on whether the neighboring blocks are intra-coded, inter-coded, or intra-inter-coded; 8. The method according to any one of appendices 1 to 7, further comprising: (Appendix 9) In the intra-inter mode, there is no intra prediction, 2. The method of claim 1, wherein the intra-inter mode includes applying PDPC directly on top of inter prediction. (Appendix 10) Applying PDPC directly on top of inter prediction involves:

number

number

[0149] [Patent Document 1] U.S. Patent No. 9,609,343

Claims

[Claim 1] 1. A method for video decoding, executed by at least one processor of an encoding device, comprising: For position-dependent intra prediction combining (PDPC), selecting a set of weightings to be used to generate intra-predicted samples for output in an intra-inter mode that combines intra prediction and inter prediction, based on whether the intra mode uses different initial weighting factors of PDPC corresponding to each of the different intra prediction modes for intra prediction, the set of weightings including weightings for each of the X and Y coordinates in the current block; applying the PDPC based on the set of weights to modify at least one of inter predicted samples and reconstructed samples of an inter coded coding unit (CU); A method comprising:

Images