Method and apparatus for improvement for intra-inter prediction mode, and computer program

Abstract

To provide an improved scheme to use for an intra-inter prediction mode and an adaptive intra-inter prediction mode.SOLUTION: The improved scheme of the intra-inter prediction mode and an adaptive intra-inter prediction mode for video coding / decoding includes: selecting a set of weightings in an intra-inter mode; and modifying at least one of an inter prediction sample and a reconstructed sample of an inter coded coding unit by applying position dependent intra prediction combination.SELECTED DRAWING: Figure 10

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to U.S. Provisional Application No. 62 / 767,228, filed November 14, 2018, U.S. Provisional Application No. 62 / 788,056, filed January 3, 2019, and U.S. Application No. 16 / 673,097, filed November 4, 2019, each of which is incorporated herein by reference in its entirety. [Technical field]

[0002] This disclosure relates to a set of advanced video coding techniques, including improvement schemes used for intra-inter prediction modes and adaptive intra-inter prediction modes. [Background technology]

[0003] ITU-T VCEG (Q6 / 16) and ISO / IEC MPEG (JTC l / SC 29 / WG 11) published the H.265 / HEVC (High Efficiency Video Coding) standard in 2013 (version 1), 2014 (version 2), 2015 (version 3), and 2016 (version 4). In 2015, the two standardization organizations jointly formed the Joint Video Exploration Team (JVET) to explore the possibility of developing the next generation video coding standard beyond HEVC. In October 2017, the two standardization organizations announced a joint Call for Proposal (CfP) for video compression with capabilities beyond HEVC. By February 15, 2018, a total of 22 CfP responses for standard dynamic range (SDR), 12 CfP responses for high dynamic range (HDR), and 12 CfP responses for 360 video categories had been proposed. In April 2018, all received CfP responses were evaluated at the 122 MPEG / 10th JVET meeting. As a result of this meeting, JVET formally started the standardization process for the next generation of video coding beyond HEVC. The new standard was named Versatile Video Coding (VVC) and JVET was renamed the Joint Video Experts Team.

[0004] The following sentence from VVC Draft 2 describes PDPC, where nScale is used to specify the rate of decrease of the weighting coefficients, and the "32" in equations 8-77, 8-78, 8-82, and 8-83 specifies the initial weighting coefficients.

[0005] The inputs to the position-dependent intra prediction combining process may be as follows: - intra prediction mode predModelntra, - the variable nTbW, which specifies the width of the transformation block; - variable nTbH, which specifies the height of the transformation block; - variable refW, which specifies the width of the reference sample, - variable refH, which specifies the height of the reference sample, - the predicted samples predSamples[x][y], where x=0···nTbW−1 and y=0···nTbH−1, - the adjacent samples p[x][y], where x=-1, y=-1·refH-1 and x=0·refW-1, y=-1, - Variable cIdx that specifies the color components of the current block.

[0006] The output of the process is the modified predicted samples predSamples[x][y], where x = 0 · · · nTbW-1, y = 0 · · · nTbH-1.

[0007] Depending on the value of cIdx, the function clip1Cmp is set as follows: -If cIdx is equal to 0, clip1Cmp is set equal to Clip1Y. - Otherwise, clip1Cmp is set equal to Clip1C.

[0008] The variable nScale is set to ((Log2(nTbW)+Log2(nTbH)-2)≫2).

[0009] The reference sample arrays mainRef[x] and sideRef[y] are derived as follows, where x=0···refW and y=0···refH.

number

[0010] The variables refL[x][y], refT[x][y], wT[y], wL[y] and wTL[x][y] are derived as follows, where x = 0 · · · nTbW-1 and y = 0 · · · nTbH-1. When -predModeIntra is equal to INTRA_PLANAR, INTRA_DC, INTRA_ANGULAR18 or INTRA_ANGULAR50, the following applies:

number

[0011] Otherwise, if predModeIntra is equal to INTRA_ANGULAR2 or INTRA_ANGULAR66, the following applies:

number

[0012] Otherwise, if predModeIntra is less than or equal to INTRA_ANGULAR10, the following ordered procedure applies: Depending on the intraPredMode, derive the variables dXPos[y], dXFrac[y], dXInt[y] and dX[y] as follows, e.g. using invAngle as specified in clause 8.2.4.2.7:

number

[0013] The variables refL[x][y], refT[x][y], wT[y], wL[y], and wTL[x][y] are derived as follows.

number

[0014] Otherwise, if predModeIntra is greater than or equal to INTRA_ANGULAR58, the following ordered procedure applies: Depending on the intraPredMode, for example using invAngle as specified in clause 8.2.4.2.7, derive the variables dYPos[x], dYFrac[x], dYInt[x] and dY[x] as follows:

number

number

[0015] Derive the values ​​of the filtered samples filtSamples[x][y] as follows, where x = 0 · · · nTbW-1 and y = 0 · · · nTbH-1.

[0016] For chroma components of an intra-coded block, the encoder selects an optimal chroma prediction mode from five modes, including planar (mode index 0), DC (mode index 1), horizontal (mode index 18), vertical (mode index 50), and diagonal (mode index 66), and selects a direct copy of the intra prediction mode to use for the associated luma component (i.e., DM mode). The mapping between chroma intra prediction mode numbers and intra prediction directions is shown in Table 1. Table 1 - Mapping between chroma intra prediction modes and intra prediction directions [Table 1]

[0017] To avoid overlapping modes, the four modes other than DM are assigned according to the intra prediction mode of the associated luma component. If the intra prediction mode number of the chroma component is 4, the intra prediction direction of the luma component is used to generate the intra prediction sample of the chroma component. If the intra prediction mode number of the chroma component is not 4 and is the same as the intra prediction mode number of the luma component, the intra prediction direction 66 is used to generate the intra prediction sample of the chroma component.

[0018] Merge mode for inter-picture prediction is used to indicate to a block that motion data is estimated rather than explicitly signaled: first, a merge candidate list of candidate motion parameters is constructed, and then an index identifying the candidate to be used is signaled.

[0019] The merge candidate list includes a non-sub-CU merge candidate list and a sub-CU merge candidate list. The non-sub-CU merge candidates are constructed based on spatially adjacent motion vectors, co-located temporal motion vectors, and history-based motion vectors. The sub-CU merge candidate list includes affine merge candidates and ATMVP merge candidates. The sub-CU merge candidates are used to derive multiple MVs for the current CU, and different portions of samples in the current CU can have different motion vectors.

[0020] Skip mode is used to indicate for a block that motion data is estimated rather than explicitly signaled and that the prediction residual is zero, i.e., no transform coefficients are transmitted. At the start of each CU of an inter-picture predicted slice, a skip_flag is signaled, which means: Derive motion data using merge mode No residual data in the bitstream

[0021] Multi-hypothesis intra-inter prediction is a combination of one intra prediction and one merge index prediction, i.e., intra-inter prediction mode. In merge CU, for merge mode, one flag is signaled to select intra mode from intra candidate list when the flag is true. For luma component, intra candidate list is derived from four prediction modes including DC mode, planar mode, horizontal mode, and vertical mode, and the size of intra candidate list may be 3 or 4 depending on the shape of the block. If the width of the CU is larger than twice the height of the CU, remove horizontal mode from intra mode list, and if the height of the CU is larger than twice the width of the CU, remove vertical mode from intra mode list. Use weighted average to combine one intra prediction mode selected by intra mode index and one merge indexed prediction selected by merge index. For chroma component, DM is always applied without additional signaling.

[0022] The weighting for combining predictions is described below. If DC mode or planar mode is selected or the width or height of the coding block (CB) is less than 4, equal weighting is applied. For a CB whose width and height are 4 or more, if horizontal / vertical mode is selected, first divide one CB vertically / horizontally into four equal area regions. Each weighting set, denoted as (w_intrai, w_interi), is applied to a corresponding region, where i is 1 to 4 and (w_intra1, w_inter1)=(6, 2), (w_intra2, w_inter2)=(5, 3), (w_intra3, w_inter3)=(3, 5) and (w_intra4, w_inter4)=(2, 6). (w_intra1, w_inter1) is for the region closest to the reference sample, and (w_intra4, w_inter4) is for the region farthest from the reference sample. Then, we can compute the combined prediction by summing the two weighted predictions and right shifting 3 bits. Also, if the next neighboring CBs are intra coded, we save the intra prediction mode used for the predictor's intra hypothesis for their intra mode coding.

[0023] Currently, for the luma component, in the case of intra-inter prediction mode, up to four intra prediction modes are adopted, which increases the complexity.

[0024] Currently, for intra-inter mode, one buffer is used to store the inter prediction value and one additional buffer is added to store the intra prediction value, which increases the decoder memory cost.

[0025] For current intra-inter modes, both intra and inter predictions need to be performed, and the intra prediction process involves multiplications, which adds complexity to the decoder.

[0026] For the current intra-inter mode, for different block sizes, the weightings are fixed values, may not be optimal, and may be improved.

[0027] The weights (w_intra, w_inter) applied in the multi-hypothesis case are selected depending on the relative position of the sample within the block, however this selection may be unreliable and inflexible.

[0028] The weights (w_intra, w_inter) applied in the multi-hypothesis case may be correlated with whether the neighboring blocks are intra- or inter-coded. By considering whether the neighboring blocks above or to the left are intra- or inter-coded, the weights can be selected more efficiently. Summary of the Invention [Problem to be solved by the invention]

[0029] Therefore, technical solutions to such problems are desirable. [Means for solving the problem]

[0030] The present invention includes a method and apparatus, the apparatus including a memory configured to store computer program code and one or more hardware processors configured to access the computer program code and operate according to instructions of the computer program code, the computer program including selection code configured to cause the processor to select a set of weights in an intra-inter mode, and application code configured to cause the processor to apply position-dependent intra prediction combining (PDPC) to modify at least one of inter predicted samples and reconstructed samples of an inter coded coding unit (CU).

[0031] According to an example embodiment, the application code is further configured to cause the processor to apply the PDPC to modify only the luma component of the inter-coded CU.

[0032] According to an exemplary embodiment, the signaling code is further configured to cause the processor to signal an inter position-dependent intra prediction combining flag interPDPCFlag in response to determining that the coding block (CB) is coded by merge mode.

[0033] According to an exemplary embodiment, the application code is further configured to cause the processor to apply the PDPC to the plurality of reconstructed samples before applying any of the deblocking, sample adaptive offset (SAO), and in-loop deblocking filters, including an adaptive loop filter (ALF).

[0034] According to an exemplary embodiment, the program code further includes an assignment code configured to cause the processor to assign a default intra-prediction mode to a current coding block (CB), and the application code is further configured to cause the processor to apply the default intra-prediction mode to a CB next to the current CB, at least for intra-mode coding and derivation of a most probable mode (MPM).

[0035] According to an example embodiment, the application code is further configured to cause the processor to apply a set of PDPC filters to at least one of inter predicted samples other than the inter predicted samples and reconstructed samples of the inter coded CU other than the reconstructed samples of the inter coded CU, each PDPC filter in the set of PDPC filters being restricted for a corresponding block size of a plurality of block sizes.

[0036] According to an example embodiment, the signaling code is further configured to cause the processor to signal interPDPCFlag indicating whether to apply at least one of the PDPC filters; determine whether interPDPCFlag is set equal to true; in response to determining that interPDPCFlag is set equal to the first value, signal a second flag, the second flag indicating applying at least one PDPC filter to at least one of the inter predicted samples and the reconstructed samples of the inter coded CU after inter prediction; and in response to determining that interPDPCFlag is set equal to the second value, exclude the PDPC filter from at least one of the inter predicted samples and the reconstructed samples of the inter coded CU.

[0037] According to an exemplary embodiment, the program code further includes signaling code configured to cause the processor to select whether the weightings in the set of weightings are equal to one another, and weighting code configured to cause the processor, in response to determining that the weightings are not equal, to weight inter prediction in the PDPC filtering depending on whether the neighboring blocks are coded by intra coding, inter coding, or intra-inter coding.

[0038] According to an example embodiment, in the intra-inter mode, there is no intra prediction, and the intra-inter mode includes directly applying PDPC on the inter prediction, where the directly applying PDPC on the inter prediction includes:

number

[0039] Other features, nature and various advantages of the disclosed subject matter will become more apparent from the following detailed description and the accompanying drawings, in which: [Figure 1] FIG. 2 is a schematic diagram of a diagram according to an embodiment. [Diagram 2] FIG. 2 is a schematic diagram of a diagram according to an embodiment. [Diagram 3] FIG. 2 is a schematic diagram of a diagram according to an embodiment. [Figure 4] FIG. 2 is a schematic diagram of a diagram according to an embodiment. [Diagram 5] FIG. 2 is a schematic diagram of a diagram according to an embodiment. [Figure 6] FIG. 2 is a schematic diagram of a diagram according to an embodiment. [Figure 7] FIG. 2 is a schematic diagram of a diagram according to an embodiment. [Figure 8] FIG. 2 is a schematic diagram of a diagram according to an embodiment. [Figure 9A] FIG. 2 is a schematic diagram of a diagram according to an embodiment. [Figure 9B] FIG. 2 is a schematic diagram of a diagram according to an embodiment. [Figure 10] 1 is a simplified flow chart according to an embodiment. [Figure 11] 1 is a simplified flow chart according to an embodiment. [Figure 12] 1 is a simplified flow chart according to an embodiment. [Figure 13] 1 is a simplified flow chart according to an embodiment. [Figure 14] FIG. 2 is a schematic diagram of a diagram according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0040] The proposed features described below can be used individually or in any combination in any order. Also, the embodiments can be implemented by processing circuitry (e.g., one or more processors or one or more integrated circuits). In one example, the one or more processors execute a program stored on a non-transitory computer-readable medium. In this disclosure, most probable mode (MPM) can refer to the primary MPM, the secondary MPM, or both the primary and secondary MPM.

[0041] 1 shows a simplified block diagram of a communication system 100 according to an embodiment of the present disclosure. The communication system 100 may include at least two terminals 102, 103 connected to each other via a network 105. For one-way data transmission, a first terminal 103 may transmit video data to another terminal 102 via the network 105 by encoding the video data at a local location. A second terminal 102 may receive the encoded video data of another terminal from the network 105, decode the encoded data, and display the restored video data. One-way data transmission may be common in media service applications, etc.

[0042] 1 illustrates a second pair of terminals 101, 104 provided to support bidirectional transmission of encoded video, such as may occur during a video conference. For bidirectional transmission of data, each terminal 101, 104 may transmit video data captured at a local location to the other terminal over network 105 by encoding the video data. Each terminal 101, 104 may also receive encoded video data transmitted by the other terminal, decode the encoded data, and display the reconstructed video data on a local display device.

[0043] In FIG. 1, terminals 101, 102, 103, and 104 may be illustrated as servers, personal computers, and smartphones, although the principles of the present disclosure are not so limited. Embodiments of the present disclosure may be laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. Network 105 represents any number of networks, including, for example, wired and / or wireless communication networks, for transmitting encoded video data between terminals 101, 102, 103, and 104. Communication network 105 may exchange data in circuit-switched and / or packet-switched channels. Such networks may include telecommunications networks, local area networks, wide area networks, and / or the Internet. For purposes of this discussion, the architecture and topology of network 105 may not be important to the operation of the present disclosure unless otherwise described herein below.

[0044] As an example of an application of the subject matter of this disclosure, Figure 2 shows a configuration of a video encoder and decoder in a streaming environment. The disclosed subject matter may equally be applied to other applications that support video, including, for example, video conferencing, digital TV, and storage of compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0045] The streaming system can include a capture subsystem 203, which may include a video source 201, such as a digital camera, that creates an uncompressed video sample stream 213. The sample stream 213, depicted as a thick line to emphasize the amount of data that is larger compared to an encoded video bitstream, can be processed by an encoder 202 coupled to the camera 201. The encoder 202 can include hardware, software, or a combination thereof to realize or implement aspects of the disclosed subject matter, which are described in more detail below. The encoded video bitstream 204, depicted as a thin line to emphasize the amount of data that is smaller compared to the samplestream, can be stored in a stream server 205 for future use. One or more streaming clients 212, 207 can access the streaming server 205 to retrieve copies 208, 206 of the encoded video bitstream 204. The client 212 may include a video decoder 210 that decodes an incoming copy 208 of the encoded video bitstream to create an outgoing video sample stream 210 that can be rendered on a display 209 or other rendering device (not shown). In some streaming systems, the video bitstreams 204, 206, 208 may be encoded according to a particular video encoding / compression standard. Examples of these standards are mentioned above and described further herein.

[0046] FIG. 3 may be a functional block diagram of a video decoder 300 according to an embodiment of the present disclosure.

[0047] The receiver 302 may receive one or more codec video sequences to be decoded by the decoder 300. In the same or another embodiment, it receives one coded video sequence at a time, with the decoding of each coded video sequence being independent of the other coded video sequences. It may receive coded video sequences from a channel 301, which may be a hardware / software link to a storage device that stores the coded video data. The receiver 302 may receive coded video data and other data, such as coded audio data and / or auxiliary data streams, that may be forwarded to a respective using entity (not shown). The receiver 302 may separate the coded video sequences from the other data. To prevent network jitter, a buffer memory 303 may be coupled between the receiver 302 and the entropy decoder / parser 304 (hereafter referred to as the "parser"). The buffer memory 303 may not be needed or may be small if the receiver 302 receives data from a storage / forwarding device with sufficient bandwidth and controllability, or from an isochronous real-time network. To make optimal use of packet networks such as the Internet, a buffer memory 303 may be required, which may be relatively large and may advantageously have an adaptive size.

[0048] The video decoder 300 may include a parser 304 to reconstruct symbols 313 based on the entropy coded video sequence. These categories of symbols include information for managing the operation of the decoder 300 and potentially information for controlling a rendering device that is not part of the decoder but may be coupled to the decoder, such as a display 312. The control information for the rendering device may be in the form of a Supplemental Enhancement Information (SEI) message or a Video Usability Information (VUI) parameter set fragment (not shown). The parser 304 may parse / entropy decode the received coded video sequence. The coding of the coded video sequence may conform to a video coding technique or standard and may follow principles well known to those skilled in the art, including variable length coding, Huffman coding, arithmetic coding with or without context-dependent, etc. The parser 304 may extract a subgroup parameter set for at least one of the subgroups of pixels in the video decoder from the coded video sequence based on at least one parameter corresponding to the group. Subgroups may include Groups of Pictures (GOPs), pictures, tiles, slices, macroblocks, Coding Units (CUs), blocks, Transform Units (TUs), Prediction Units (PUs), etc. The entropy decoder / parser may also extract, for example, transform coefficients, quantizer parameter QP values, motion vectors, etc. from the coded video sequence information.

[0049] The parser 304 may perform an entropy decoding / parsing operation on the video sequence received from the buffer memory 303 to create symbols 313. The parser 304 may receive the encoded data and selectively decode a particular symbol 313. The parser 304 may also determine whether the particular symbol 313 has been provided to the motion compensated prediction unit 306, the scaler / inverse transform unit 305, the intra prediction unit 307, or the loop filter 311.

[0050] The reconstruction of symbols 313 may involve several different units, depending on the type of coded or partially coded video picture (e.g., inter-picture vs. intra-picture, inter-block vs. intra-block) and other factors. The units involved, and the form of involvement, may be controlled by subgroup control information parsed by parser 304 from the coded video sequence. For simplicity, the flow of such subgroup control information between parser 304 and the following units is not described.

[0051] In addition to the functional blocks already mentioned, the decoder 200 can be conceptually subdivided into a number of functional units, which are described below. In practical implementations that operate under commercial constraints, many of these units will interact closely with each other and may be at least partially integrated with each other. However, for purposes of illustrating the disclosed subject matter, a conceptual subdivision into the following functional units is appropriate:

[0052] The first unit is a scalar / inverse transform unit 305. The scalar / inverse transform unit 305 may receive quantized transform coefficients as symbols 313 and control information from the parser 304, including the transform method to use, block size, quantization coefficients, quantization scaling matrices, etc. The scalar / inverse transform unit 305 may output blocks containing sample values ​​that may be input to an aggregator 310.

[0053] In some cases, the output samples of the scalar / inverse transform unit 305 may belong to intra-coded blocks, i.e. blocks that do not use prediction information from a previously reconstructed picture, but use prediction information from a previously reconstructed part of the current picture. Such prediction information can be provided by the intra prediction unit 307. In some cases, the intra prediction unit 307 uses surrounding already reconstructed information extracted from the current (partially reconstructed) picture 309 to generate blocks of the same size and shape as the block it is reconstructing. In some cases, the aggregator 310 adds the prediction information generated by the intra prediction unit 307 based on each sample to the output sample information provided by the scalar / inverse transform unit 305.

[0054] In other cases, the output samples of the scalar / inverse transform unit 305 may belong to an inter-coded, potentially motion-compensated block. In such a case, the motion compensation prediction unit 306 may access the reference picture memory 308 to extract samples for prediction. After performing motion compensation on the extracted samples based on the symbols 313 belonging to the block, these samples can be added by the aggregator 310 to the output of the scalar / inverse transform unit (called residual samples or residual signals in this case) to generate output sample information. The addresses in the reference picture memory from which the motion compensation unit extracts the prediction samples may be controlled by a motion vector, which can be used by the motion compensation unit in the form of a symbol 313, which may have, for example, X, Y, and reference picture components. The motion compensation may include an interpolation of sample values ​​extracted from the reference picture memory when a sub-sample accurate motion vector is used, a motion vector prediction mechanism, etc.

[0055] The output samples of the aggregator 310 may be processed by various loop filtering techniques in a loop filter unit 311. The video compression techniques may include in-loop filter techniques, which are controlled by parameters contained in the encoded video bitstream and available to the loop filter unit 311 as symbols 313 from the parser 304; however, the video compression techniques may also be responsive to meta-information obtained during decoding of a previous part (in decoding order) of the encoded picture or encoded video sequence, or to previously reconstructed loop filtered sample values.

[0056] The output of the loop filter unit 311 may be a sample stream, which may be output to the rendering device 312 and stored in the reference picture memory 557 for use in future inter-picture prediction.

[0057] Once fully reconstructed, some coded pictures can be used as reference pictures for future prediction. Once a coded picture is fully reconstructed and the coded picture is identified as a reference picture (e.g., by the parser 304), the current reference picture 309 can become part of the reference picture buffer 308, and a new current picture memory can be reallocated before starting the reconstruction of the next coded picture.

[0058] The video decoder 300 may perform decoding operations according to a given video compression technique, e.g., as documented in the ITU-T H.265 Recommendation standard. The encoded video sequence may comply with the syntax specified by the video compression technique or standard used, in the sense that the encoded video sequence complies with the syntax of the video compression technique or standard as specified in the video compression technique document or standard and specifically specified in the document files therein. Compliance also requires that the complexity of the encoded video sequence is within the range limited by the level of the video compression technique or standard. In some cases, the level limits the maximum picture size, maximum frame rate, maximum reconstructed sample rate (e.g., measured in mega samples per second), maximum reference picture size, etc. In some cases, the limits set by the level may be further limited by the specification of a Hypothetical Reference Decoder (HRD) and HRD buffer management metadata signaled in the encoded video sequence.

[0059] In an embodiment, the receiver 302 may receive additional (redundant) data along with the encoded video. The additional data may be included as part of one or more encoded video sequences. The additional data may be used by the video decoder 300 to accurately decode the data and / or to more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0060] FIG. 4 may be a functional block diagram of a video encoder 400 according to an embodiment of the present disclosure.

[0061] The encoder 400 may receive video samples from a video source 401 (not part of the encoder), which may capture the video images to be encoded by the encoder 400 .

[0062] The video source 401 may provide a source video sequence in the form of a digital video sample stream to be encoded by the encoder 400, the digital video sample stream may have any suitable bit depth (e.g. 8 bit, 10 bit, 12 bit, etc.), any color space (e.g. BT.601 Y CrCB, RGB, etc.) and any suitable sampling configuration (e.g. Y CrCb 4:2:0, Y CrCb 4:4:4). In a media serving system, the video source 401 may be a storage device that stores previously prepared video. In a video conferencing system, the video source 401 may be a camera that captures local image information as a video sequence. The video data may be provided as a number of separate pictures that are given motion when viewed in sequence. The pictures themselves may be organized as a spatial pixel array, where each pixel may contain one or more samples depending on the sampling configuration, color space, etc. used. Those skilled in the art can easily understand the relationship between pixels and samples. In the following description, the focus is on samples.

[0063] According to an embodiment, the encoder 400 may encode pictures of a source video sequence and compress it into an encoded video sequence 410 in real-time or under any other time constraint required by an application. Performing an appropriate encoding rate is one of the functions of the controller 402. The controller controls and is functionally coupled to other functional units described below. For simplicity, the coupling is not depicted. The parameters set by the controller may include rate control related parameters (picture skip, quantizer, lambda value for rate distortion optimization techniques, etc.), picture size, picture group GOP layout, maximum motion vector search range, etc. A person skilled in the art can easily recognize other functions of the controller 402, as these functions may belong to a video encoder 400 optimized for a particular system design.

[0064] Some video encoders operate in what those skilled in the art will easily recognize as a "coding loop". As a very simplified description, the encoding loop may include an encoding part (responsible for constructing symbols based on the input picture to be encoded and one or more reference pictures) of an encoder 402 (hereafter referred to as the "source encoder"), and a (local) decoder 406 embedded in the encoder 400, which reconstructs the symbols to create sample data that is also created by the (remote) decoder (because the compression between the symbols and the encoded video bitstream is lossless in the video compression techniques considered in the disclosed subject matter). The reconstructed sample stream is input to a reference picture memory 405. The decoding of the symbol stream produces results that are bit-exact independent of the location of the decoder (local or remote), so the contents of the reference picture buffer are also bit-exact between the local and remote encoders. That is, the reference picture samples that the predictive part of the encoder "sees" are exactly the same as the sample values ​​that the decoder "sees" when using the prediction during decoding. The basic principle of reference picture synchrony (and the drift that occurs when synchrony cannot be maintained, for example due to channel errors) is well known to those skilled in the art.

[0065] The operation of the "local" decoder 406 may be the same as that of the "remote" decoder 300, described in detail above with reference to Fig. 3. However, and referring briefly to Fig. 4, the entropy decoding part of the decoder 300, including the channel 301, the receiver 302, the buffer memory 303 and the parser 304, may not be fully implemented in the local decoder 406, provided that symbols are available and the entropy coder 408 and the parser 304 are able to losslessly code / decode the symbols into an encoded video sequence.

[0066] In this case, in addition to the analysis / entropy decoding present in the decoder, it is understood that any decoder techniques will necessarily be present in the corresponding encoder in essentially the same functional form. The description of the encoder technique can be simplified since the encoder technique and the fully described decoder technique are mutually inverse. More detailed descriptions are only necessary in certain areas and are provided below.

[0067] As part of the operation of the source encoder 402, the source encoder 403 may perform motion compensated predictive encoding, which predictively encodes an input frame by referencing one or more previously encoded frames from the video sequence designated as "reference frames." In this manner, the encoding engine 407 may encode differences between pixel blocks of the input frame and pixel blocks of reference frames that may be selected as predictive references for the input frame.

[0068] The local video decoder 406 may decode the encoded video data of the frames that may be designated as reference frames based on the symbols created by the source encoder 403. The operation of the encoding engine 407 may advantageously be a lossy process. If the encoded video data may be decoded in a video decoder (FIG. 4, not shown), the reconstructed video sequence may be a copy of the source video sequence, usually with some errors. The local video decoder 406 may replicate the decoding process that may be performed by the video decoder on the reference frames and store the reconstructed reference frames in the reference picture cache 405. In this way, the encoder 400 may store locally a copy of the reconstructed reference frames that has a common content (no transmission errors) with the reconstructed reference frames obtained by the remote video decoder.

[0069] The predictor 404 may perform a prediction search on the coding engine 407. That is, for a new frame to be coded, the predictor 404 may search in the reference picture memory 405 for sample data (as candidate reference pixel blocks) that may be used as suitable prediction references for the new picture, or specific metadata, such as, for example, motion vectors, block shapes, etc., of the reference pictures. The predictor 404 may operate pixel block by pixel block based on the sample blocks to find suitable prediction references. In some cases, the input picture may have prediction references extracted from multiple reference pictures stored in the reference picture memory 405, for example, as determined by the search results obtained by the predictor 404.

[0070] The controller 402 may manage the encoding operations of the source encoder 403, including, for example, setting parameters for encoding the video data and subgroup parameters.

[0071] The output of all the functional units mentioned above may be entropy coded in an entropy coder 408. The entropy coder converts the symbols produced by each functional unit into an encoded video sequence by performing lossless compression on the symbols according to techniques well known to those skilled in the art (e.g. Huffman coding, variable length coding, arithmetic coding, etc.).

[0072] The transmitter 409 may buffer one or more encoded video sequences created by the entropy encoder 408 in preparation for transmission over a communication channel 411, which may be a hardware / software link to a storage device that stores the encoded video data. The transmitter 409 may merge the encoded video data from the source encoder 403 with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (sources not shown).

[0073] The controller 402 may manage the operation of the encoder 400. During encoding, the controller 405 may assign to each encoded picture a particular encoded picture type, which may affect the encoding technique that may be applied to the corresponding picture. For example, a picture may typically be assigned as one of the following picture types:

[0074] An Intra picture (I-picture) may be a picture that is coded and decoded without using any other frame in a sequence as a source of prediction. Some video codecs allow different types of Intra pictures, including, for example, Independent Decoder Refresh pictures. Those skilled in the art are aware of these variations of I-pictures and their corresponding uses and characteristics.

[0075] A predictive picture (P picture) may be a picture that is encoded and decoded using intra- or inter-prediction to predict the sample values ​​of each block using at most one motion vector and reference index.

[0076] A bidirectionally predictive picture (B-picture) may be a picture that is encoded and decoded using intra- or inter-prediction to predict the sample values ​​of each block using at most two motion vectors and reference indexes. Similarly, a multiple predicted picture may use more than two reference images and associated metadata to reconstruct a single block.

[0077] A source picture may generally be spatially subdivided into a number of sample blocks (e.g., blocks of 4x4, 8x8, 4x8 or 16x16 samples, respectively) and coded block by block. A block may be predictively coded with reference to other (already coded) blocks determined by the coding assignment applied to the block's corresponding picture. For example, a block of an I-picture may be non-predictively coded or predictively coded with reference to coded blocks of the same picture (spatial or intra prediction). A pixel block of a P-picture may be non-predictively coded via spatial or temporal prediction with reference to one previously coded reference picture. A block of a B-picture may be non-predictively coded via spatial or temporal prediction with reference to one or two previously coded reference pictures.

[0078] The video encoder 400 may perform encoding operations based on a given video encoding technique or standard, such as the ITU-T H.265 recommendation. During operation, the video encoder 400 may perform various compression operations, including predictive encoding operations due to temporal and spatial redundancies in the input video sequence. Thus, the encoded video data may conform to a syntax specified by the video encoding technique or standard being used.

[0079] In an embodiment, the transmitter 409 may transmit additional data along with the encoded video. The source encoder 403 may include such data as part of the encoded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data such as, for example, redundant pictures, slices, Supplementary Enhancement Information (SEI) messages, Visual Usability Information (VUI) parameter set fragments, etc.

[0080] FIG. 5 shows the intra prediction modes used in HEVC and JEM. In order to capture any edge direction shown in natural video, the number of directional intra modes is extended from 33 used in HEVC to 65. The additional directional modes in JEM on top of HEVC are shown as dotted arrows in FIG. 5, and the planar mode and DC mode remain the same. These denser directional intra prediction modes apply to all block sizes and both luma and chroma intra prediction. As shown in FIG. 5, the directional intra prediction modes marked with dotted arrows associated with odd intra prediction mode indexes are referred to as odd intra prediction modes. The directional intra prediction modes marked with solid arrows associated with even intra prediction mode indexes are referred to as even intra prediction modes. In this application, the directional intra prediction modes shown with solid or dotted arrows in FIG. 5 are also referred to as angular modes.

[0081] In JEM, a total of 67 intra prediction modes are used for luma intra prediction. To code in intra mode, we build an MPM list of size 6 based on the intra modes of neighboring blocks. If an intra mode is not in the MPM list, we signal a flag to indicate whether the intra mode belongs to the selected mode. In JEM-3.0, there are 16 selection modes, which are uniformly selected as four angle modes. In JVET-D0114 and JVET-G0060, 16 secondary MPMs are derived to replace the uniformly selected modes.

[0082] 6 shows N reference layers employed for the intra-directional mode. There are block unit 611, segment A 601, segment B 602, segment C 603, segment D 604, segment E 605, segment F 606, first reference layer 610, second reference layer 209, third reference layer 608 and fourth reference layer 607.

[0083] In both HEVC and JEM, and other standards such as H.264 / AVC, the reference samples used to predict the current block are limited to the nearest reference line (row or column). In the method of multiple reference lines intra prediction, for intra-directive mode, the number of candidate reference lines (rows or columns) increases from 1 (i.e., the nearest) to N, where N is an integer equal to or greater than 1. Figure 2 shows the concept of the multiple line intra-directional prediction method by taking a 4x4 prediction unit (PU) as an example. The intra-directive mode can arbitrarily select one of N reference layers to generate a predictor. In other words, the predictor p(x, y) is generated from one of the reference samples S1, S2, ..., SN. A flag is signaled to indicate the selected reference layer for the intra-directive mode. If N is set to 1, the intra-directive mode method is the same as the conventional method of JEM2.0. In Fig. 6, reference lines 610, 609, 608, and 607 include six segments 601, 602, 603, 604, 605, and 606, with an upper-left reference sample. In this application, the reference layer is also called a reference line. The coordinates of the upper-left pixel in the current block unit are (0, 0), and the upper-left pixel of the first reference line is (-1, -1).

[0084] In JEM, for the luma component, the neighboring samples used to generate intra prediction samples are filtered before the generation process. The filtering is controlled by the specified intra prediction mode and transform block size. If the intra prediction mode is DC or the transform block size is equal to 4x4, the neighboring samples are not filtered. If the distance between the specified intra prediction mode and the vertical mode (or horizontal mode) is greater than a predefined threshold, the filtering process is started. A [1, 2, 1] filter and a bilinear filter are used to filter the neighboring samples.

[0085] The position dependent intra prediction combination (PDPC) method is an intra prediction method that calls for a combination of filtered boundary reference sample HEVC-style intra prediction and non-filtered boundary reference sample. Each prediction sample pred[x][y] at (x, y) is calculated as follows:

number

[0086] Figure 7 shows a diagram 700 in which DC mode PDPC assigns weights (wL, wT, wTL) to the (0,0) and (1,0) positions in one 4x4 block. When applying PDPC to DC intra, planar, horizontal and vertical intra modes, no additional boundary filters are required, such as the HEVC DC mode boundary filter or the horizontal / vertical mode edge filter. Figure 7 shows a reference sample R of PDPC applied to the top right diagonal mode. x、-1、 R -1、y and R -1、-1 Here is the definition of the predicted sample pred(x', y') at (x', y') in the predicted block. x、-1 The coordinate x of is given by x=x'+y'+1, and the reference sample R -1、ySimilarly, the y coordinate of is given by y=x'+y'+1.

[0087] 8 shows a Local Illumination Compensation (LIC) diagram 800, which is based on a linear model of illumination changes with a scaling factor a and an offset b, and is adaptively enabled or disabled per coding unit (CU) for inter mode coding.

[0088] When LIC is applied to a CU, the parameters a and b are derived using the neighboring samples of the current CU and its corresponding reference samples using the least square error method. More specifically, the subsampled (2:1 subsampled) neighboring samples of the CU and the corresponding samples in the reference picture (identified by the motion information of the current CU or sub-CU) are used, as shown in Figure 8. IC parameters are derived and applied to each prediction direction respectively.

[0089] If the CU is coded in merge mode, copy the LIC flag from the neighboring block, similar to copying motion information in merge mode; otherwise, signal the LIC flag to the CU to indicate whether LIC is applied.

[0090] 9A shows intra prediction modes 900 used in HEVC. In HEVC, there are a total of 35 intra prediction modes, of which mode 10 is the horizontal mode, mode 26 is the vertical mode, and modes 2, 18, and 34 are the diagonal modes. The intra prediction modes are signaled by three Most Probable Modes (MPM) and the remaining 32 modes.

[0091] 9B shows an embodiment of a VVC embodiment in which there are a total of 87 intra-prediction modes, where mode 18 is the horizontal mode, mode 50 is the vertical mode, and modes 2, 34 and 66 are diagonal modes. Modes -1 to -10 and modes 67 to 76 are referred to as Wide-Angle Intra Prediction (WAIP) modes.

[0092] According to the PDPC formula, a linear combination of intra prediction modes (DC, planar, angular) and reference samples is used to predict a prediction sample pred(x, y) at position (x, y). pred(x, y) = (wL × R -1、y +wT×R x、-1 -wTL×R -1、-1 +(64-wL-wT+wTL)×pred(x, y)+32)≫6 Here, R x、-1 , R -1、y denote the reference samples above and to the left of the current sample (x, y), respectively, and R -1、-1 denotes the reference sample in the top-left corner of the current block.

[0093] In DC mode, for a block with dimensions width and height, the weights are calculated as follows: wT=32≫((y≪1)≫nScale),wL=32≫((x≪1)≫ nScale),wTL=(wL≫4)+(wT≫4) where nScale=(log2(width)-2+log2(height)-2+2)≫2, wT indicates the weighting factor of the reference sample in the above reference line with the same horizontal coordinate, wL indicates the weighting factor of the reference sample in the left reference line with the same vertical coordinate, wTL indicates the weighting factor of the top-left reference sample of the current block, nScale specifies the speed at which the weighting factor decreases along the axis (wL decreases from left to right or wT decreases from top to bottom), i.e., the decrease rate of the weighting factor, and in the current design, the decrease rate of the weighting factor is the same along the x (left to right) and y (top to bottom) axes. Also, 32 indicates the initial weighting factor of the adjacent sample, which is also the upper (left or top-left) weighting assigned to the top-left sample of the current CB, and the weighting factor of the adjacent sample in the PDPC process must be less than or equal to this initial weighting factor.

[0094] For planar mode, wTL = 0, for horizontal mode, wTL = wT, and for vertical mode, wTL = wL. The PDPC weights can be calculated using only additions and shifts. We can calculate the value of pred(x, y) in one step using equation (1).

[0095] 10 shows a flowchart 1000 according to an example embodiment, in which in S1, it is considered to apply PDPC to the reconstructed or inter-predicted samples of the inter-coded CU. When applying PDPC to the reconstructed or inter-predicted samples of the inter-coded CU, the PDPC process further modifies the reconstructed or predicted samples of the inter-coded CU.

[0096] In S2, a flag, for example interPDPCFlag, is signaled to indicate whether PDPC is applied. According to an embodiment, interPDPCFlag can be signaled at a CU level to indicate whether PDPC is applied to reconstructed or inter-predicted samples of an inter-coded CU, and according to another embodiment, interPDPCFlag can be signaled at a TU level to indicate whether PDPC is applied to reconstructed or inter-predicted samples of an inter-coded CU. Also, interPDPCFlag is signaled only if it is determined that the current CU is neither an intra-coded CU nor a skip CU.

[0097] According to an embodiment, in S2, interPDPCFlag may be signaled only for coding blocks coded by merge mode, and the flag may be signaled before or after the merge index, so that according to an embodiment, PDPC is not applied to skip CUs and / or PDPCl is not applied to either skip CUs or sub-block merge CUs.

[0098] In S3, after reconstruction or inter prediction of the inter-coded CU, consider whether interPDPCFlag is set equal to true, and if so, in S4, apply PDPC to the reconstructed samples or inter predicted samples of the inter-coded CU to generate reconstructed samples or final predicted values ​​of the inter-coded CU.

[0099] According to an embodiment, in S4, when the PDPC is applied to the reconstructed samples, the PDPC is applied to the reconstructed samples before applying any of the in-loop deblocking filters (e.g., deblocking, Sample Adaptive Offset (SAO) and Adaptive Loop Filter (ALF)).

[0100] According to an embodiment, if in S3, interPDPCflag is set equal to true, then in S4, deblocking and SAO are not applied to the top and left boundaries of the current CU, and according to another embodiment, if in S3, interPDPCflag is set equal to true, then any other filter other than PDPC is not applied to the top and left boundaries of the current CU.

[0101] Otherwise, in S5, after reconstruction or inter prediction of the inter-coded CU, the PDPC may not be applied to the reconstructed or inter-predicted samples of the inter-coded CU, and instead, a normal inter prediction process may be used.

[0102] According to an embodiment, the PDPC may be determined and applied only to the luma component of an inter-coded CU.

[0103] According to an embodiment, when applying PDPC to the reconstructed or inter-predicted samples of the inter-coded CU of the current CB in S6, it can be determined in S7 whether to assign a default intra-prediction mode to the current CB, so that the next CB can use the default intra-prediction mode for intra-mode coding and Most Probable Mode (MPM) derivation. For example, the default intra-prediction mode can be either planar mode or DC mode.

[0104] In S8, it can be decided whether to use different initial weighting factors of PDPC for normal intra in S5 and PDPC in S4, and if so, in S9, for PDPC in S4, the weighting factor is set to 16, 8, 4, 2 or 1. According to an embodiment, when applying PDPC to reconstructed samples or inter predicted samples of inter coded CU of current CB, the intra prediction mode is marked as unavailable, and for the next neighboring block, the current CB is not considered as an intra coded block, and for the next neighboring block, the current CB cannot be used for intra mode coding or MPM derivation.

[0105] According to an embodiment, when applying the PDPC to reconstructed samples or inter-predicted samples of an inter-coded CU in S5, in S10, a weighting factor reduction rate and / or an initial weighting factor are taken into account, and in S11, the weighting factor reduction rate and / or the initial weighting factor are set to a different value than the value used in the original PDPC for intra prediction.

[0106] According to an embodiment, in S11, the weighting factor reduction rate may be set equal to a fixed value N, independent of the block size of the current coding unit, where N may be any integer equal to or greater than 0. For example, N may be set equal to 0, 1, 2, 3 or 4.

[0107] According to an embodiment, in S11, if the block size (width*height) is less than or equal to Thres_1, the weighting factor reduction rate may be set equal to a fixed value N1. Otherwise, the weighting factor reduction rate may be set equal to another fixed value N2. N1 and N2 may be any integers equal to or greater than 0. For example, N1 is set equal to 1 and N2 is set equal to 2. Thres_1 may be any positive integer, for example 32, 64, 128, 256, 512 or 1024.

[0108] According to an embodiment, in S11, the weighting factor reduction rate may be set equal to ((Log2(nTbW)+Log2(nTbH)+S)>>2), where S may be any integer, for example, -3, -2, -1, 0, 1, 2 or 3.

[0109] According to an embodiment, when applying PDPC to inter-predicted samples in S11, the weighting factor reduction rate may have different values ​​for the x-axis (from left to right) and the y-axis (from top to bottom). For example, the weighting factor reduction rate along the x-axis may be related to the width of the current CB, such as Log2(nTbH)+S1)≫1, and the weighting factor reduction rate along the y-axis may be related to the height of the current CB, such as Log2(nTbW)+S2)≫1. S1 and S2 may be any integer, -3, -2, -1, 0, 1, 2, or 3.

[0110] Also, in an embodiment according to FIG. 10, high level syntax elements including, but not limited to, SPS (Sequence Parameter Set), PPS (Picture Parameter Set), slice header, CTU header may signal whether or not PDPC is allowed to be applied on inter-predicted or inter-reconstructed blocks, and one flag may be signaled to indicate whether PDPC is allowed to be applied on luma inter-predicted or inter-reconstructed blocks, and another flag may be signaled to indicate whether PDPC is allowed to be applied on chroma inter-predicted or inter-reconstructed blocks.

[0111] FIG. 11 shows a flowchart 1100 according to an exemplary embodiment, which is according to or different from the embodiment of FIG. 10, except that S1, S2, S3 and S4 are similar, and if it is determined that the flag is set equal to true in S3, a second flag or index can be signaled in S12. The flowchart 1100 relates to applying a set of multiple PDPC filters to inter-predicted samples (or reconstructed samples of inter-coded CUs). A set of multiple PDPC filters can be limited for each block size, and when applying PDPC to the inter-predicted samples or reconstructed samples of inter-coded CUs, the predicted samples of the PDPC process can be modified to the inter-predicted samples or reconstructed samples of inter-coded CUs. The selection of the PDPC filter may be signaled for the coding block to which the PDPC can be applied.

[0112] At S12, according to an embodiment, one flag, namely interPDPCFlag, may be signaled to indicate whether PDPC is applied at S2. At S3, if interPDPCFlag is set equal to true, a second flag or index may be signaled to indicate the use of a PDPC filter after inter prediction. The specified filter type may be applied to the inter predicted samples or reconstructed samples of the inter coded CU to generate a final inter predicted value or reconstructed samples of the inter coded CU.

[0113] At S13, according to an embodiment, there may be N different PDPC filter types, and these N different PDPC filter types may be signaled by unary truncation coding or fixed length coding, where N may be any positive integer, for example 2, 3 or 4.

[0114] At S13, according to an embodiment, the number of PDPC filter types may depend on coding information including, but not limited to, the width of the block, the height of the block, the width-to-height ratio of the block, the area size of the block, whether the current block is uni-predicted or uni-predicted, and whether the current block is coded using MERGE mode or AMVP mode.

[0115] In S13, according to an embodiment, different PDPC filter types may be specified by weighting factor reduction rates or scaling factors, and / or initial weighting factors for the upper and / or left sides, and / or top-left weighting assigned to the top-left sample of the current block. For example, the weighting factor reduction rate may be set to ((Log2(nTbW)+Log2(nTbH)-2+S)>>2), where S is any integer and S has different settings for different PDPC filter types. In an embodiment, for each PDPC filter type, the weighting factor reduction rate may be set to N, where N may be any integer and N may have different settings for different PDPC filter types. In an embodiment, the derivation of the weighting factor reduction rate may use the method described when there are N different PDPC filter types, and the initial value of the weighting factor may be 32, 16, 8, 4, 2, 1 or 0. In other embodiments, only the initial values ​​of the weighting coefficients may be different for different PDPC filter types, and example values ​​for the initial values ​​of the weighting coefficients may include, but are not limited to, 32, 16, 8, 4, 2, 1, or 0.

[0116] Also, like S6 and S7, S13 assigns a default intra prediction mode to the current CB when applying PDPC to an inter predicted sample or a reconstructed sample of an inter coded CU, so that the next CB can use the default intra prediction mode to derive the MPM, and the default intra prediction mode may be a planar mode or a DC mode.

[0117] 12 shows a flowchart 1200 relating to multi-hypothesis intra-inter prediction, also called intra-inter mode or inter-intra mode, or a combination of merge and intra prediction modes. Weighting that applies the same weighting to intra prediction and inter prediction is called equal weighting, for example, (w_intra, w_inter)=(1,1) or (w_intra, w_inter)=(4,4) is equal weighting, where w_intra is the weighting applied to intra prediction and w_inter is the weighting applied to inter prediction.

[0118] At S14, a set of weights is determined for the weights applied to the intra-inter mode, i.e. w_intra and w_inter, and a process of signaling a weight selection for the block coded by the intra-inter mode is started. At S14, the parameters of the neighboring blocks can be checked, and at S15, a first flag can be signaled to indicate whether equal weights are used. According to an embodiment, the context for entropy coding the first flag depends on whether the neighboring block is intra-coded, inter-coded or intra-inter-coded. According to an embodiment, the context for entropy coding the first flag depends on whether the neighboring block above and one of the neighboring blocks to the left are intra-coded. According to an embodiment, the context for entropy coding the first flag depends on whether the neighboring block above and one of the neighboring blocks to the left are intra-coded or intra-inter-coded. According to an embodiment, the context for entropy coding said first flag depends on whether one of the upper and left neighboring blocks is inter-coded.

[0119] For example, at S16, it may be determined whether a certain condition is met: If a first flag is signaled with a value indicating that equal weighting is not applied, then weighting is selected depending on whether the neighboring blocks are intra-coded, inter-coded, or intra-inter-coded.

[0120] According to an exemplary embodiment, if any of the exemplary conditions described below are met, in S17, one weighting is applied, for example, exemplary candidate weightings (w_intra, w_inter)=(3, 1), and if any of the exemplary conditions described below are not met, in S18, one weighting is applied, for example, exemplary candidate weightings (w_intra, w_inter)=(1, 3).

[0121] The condition in S16 may be considered according to whether the context for entropy coding the first flag depends on whether both the upper adjacent block and the left adjacent block are intra-coded, whether the context for entropy coding the first flag depends on whether both the upper adjacent block and the left adjacent block are intra-coded or intra-inter-coded, and whether the context for entropy coding the first flag depends on whether both the upper adjacent block and the left adjacent block are inter-coded. However, according to an exemplary embodiment, when the context for entropy coding the first flag depends on whether both the upper adjacent block and the left adjacent block are inter-coded, the weighting may be set to (w_intra, w_inter)=(1, 3), in contrast to the other embodiments described above.

[0122] In S19, a second flag may be signaled to indicate which of the set of candidate weights is used for weighting. According to an embodiment, the context for entropy coding the second flag depends on whether the neighboring block is intra-coded, inter-coded, or intra-inter-coded. For example, if both the upper neighboring block and the left neighboring block are intra-coded, the context value may be set to a first value, otherwise if either the upper neighboring block and the left neighboring block are intra-coded, the context value may be set to a second value, otherwise if neither the upper neighboring block and the left neighboring block are intra-coded, the context value may be set to a third value. According to another embodiment, if both the upper neighboring block and the left neighboring block are intra-coded or intra-inter-coded, the context value may be set to a first value, otherwise if either the upper neighboring block and the left neighboring block are intra-coded or intra-inter-coded, the context value may be set to a second value, otherwise if neither the upper neighboring block and the left neighboring block are intra-coded, the context value may be set to a third value. According to an embodiment, the set of candidate weightings may include (w_intra, w_inter)=(1, 3) and (w_intra, w_inter)=(3, 1).

[0123] FIG. 13 shows a flowchart 1300, in which in S20, intra prediction is not used in intra-inter mode, and PDPC is applied directly on inter prediction; however, the weighting applied to inter prediction in PDPC filtering depends on whether the neighboring blocks are intra-coded, inter-coded, or intra-inter-coded.

[0124] According to an embodiment, in S21, PDPC is applied directly on top of inter prediction as follows:

number

[0125] where InterPred is the input inter prediction and w_intra is selected from a set of predefined values, the selection depending on whether the neighboring block is intra-coded, inter-coded, or intra-inter-coded, and example values ​​of N include 8, 16, 32, 64, 128, and 256. If the neighboring block meets the condition, then apply weighting 1 in S22, and if the neighboring block does not meet the condition, apply weighting 2 in S23.

[0126] According to an embodiment, in S21, PDPC is applied directly on top of inter prediction as follows:

number

[0127] According to an embodiment, in S21, PDPC is applied directly on top of inter prediction as follows:

number

number

[0128] According to an embodiment, in S21, the weighting applied to the upper reference sample (wT) in PDPC depends on whether the upper neighboring block is intra-coded, inter-coded, or intra-inter-coded, and the weighting applied to the left reference sample (wL) in PDPC depends on whether the left neighboring block is intra-coded, inter-coded, or intra-inter-coded.

[0129] According to an embodiment, in S21, PDPC is applied directly on top of inter prediction as follows:

number

[0130] According to an embodiment, in S21, PDPC is applied directly on top of inter prediction as follows:

number

[0131] Therefore, according to the exemplary embodiments described herein, the technical solutions can advantageously improve the above technical problems.

[0132] The techniques described above may be implemented as computer software by computer-readable instructions and physically stored on one or more computer-readable media. For example, FIG. 14 illustrates a computer system 1400 suitable for implementing some embodiments of the disclosed subject matter.

[0133] Computer software can be encoded in any suitable machine code or computer language, which can be subjected to mechanisms such as assembling, compiling, linking, etc. to produce code containing instructions that can be executed directly by a computer central processing unit (CPU), graphics processing unit (GPU), etc., or interpreted, executed by microcode, etc.

[0134] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smart phones, gaming devices, Internet of Things devices, etc.

[0135] 14 are exemplary in nature and are not intended to imply limitations on the scope or functionality of the computer software used to implement embodiments of the present disclosure, nor should the arrangement of components be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in the exemplary embodiment of computer system 1400.

[0136] The computer system 1400 may include several human-machine interface input devices. Such human-machine interface input devices may be responsive to input by one or more human users, such as tactile input (e.g., keystrokes, slides, data glove movements), audio input (e.g., voice, hand clapping), visual input (e.g., posture), olfactory input (not shown), etc. Human-machine interface devices may also be used to capture certain media that are not necessarily directly related to the conscious input of a human being, such as, for example, audio (e.g., voice, music, ambient sounds), images (e.g., scanned images, photographic images obtained from a static image capture device), and video (e.g., two-dimensional video, three-dimensional video including stereo video).

[0137] The human machine interface input devices may include one or more of a keyboard 1402, a mouse 1404, a touchpad 1403, a touch panel 1410, a joystick 1405, a microphone 1406, a scanner 1408, and an imaging device 1407 (only one of each is shown).

[0138] The computer system 1400 may further include human machine interface output devices. Such human machine interface output devices may stimulate one or more of the human user's senses, for example, via haptic output, sound, light, and smell / taste. Such human machine interface output devices may include haptic output devices (e.g., touch panel 1410, or haptic feedback devices via joystick 1405, although there are also haptic feedback devices that are not used as input devices), audio output devices (e.g., speakers 1409, headphones (not shown)), visual output devices (e.g., screen 1410, virtual reality glasses (not shown), holographic displays, and smoke tanks (not shown), including screen 1410 including CRT screens, LCD screens, plasma screens, OLED screens, each of which may or may not have touch panel input capabilities, haptic feedback capabilities, some of which may be capable of outputting two-dimensional visual output or three or more dimensional output, such as by means of stereoscopic image output), and printers (not shown). A graphics adapter 1429 may be provided for interfacing to various input devices.

[0139] Computer system 1400 may further include human-accessible storage devices and associated media, including, for example, optical media including CD / DVD ROM / RW 1412 with media 1411 such as CDs / DVDs, thumb drives 1413, removable hard drives or solid state drives 1414, traditional magnetic media (e.g., magnetic tape and floppy disks (not shown)), dedicated ROM / ASIC / PLD based devices (e.g., dongles (not shown)), etc.

[0140] Those skilled in the art should understand that the term "computer-readable medium" as used in conjunction with the presently disclosed subject matter does not include transmission media, carrier waves, or other instantaneous signals.

[0141] The computer system 1400 may further include an interface to one or more communication networks 1415. The network 1415 may be, for example, wireless, wired, optical. The network 1415 may further be local, wide area, city, vehicular and industrial, real-time, delay tolerant, etc. Examples of the network 1415 include local area networks such as Ethernet, cellular networks including WLAN, GSM, 3G, 4G, 5G, LTE, etc., television wired or wireless wide area digital networks including cable television, satellite television and terrestrial television, vehicular and industrial networks including CANBus, etc. Some networks 1415 typically require an external network interface adapter 1427 connected to some general purpose data port or peripheral bus (1425) (e.g., USB port of the computer system 1400), while other networks are typically integrated into the core of the computer system 1400 by connecting to a system bus described below (e.g., an Ethernet interface to a PC computer system, or a cellular network interface to a smartphone computer system). Through any of these networks, the computer system 1400 can communicate with other entities. Such communications may be one-way receive only (e.g., broadcast television), one-way transmit only (e.g., a CANbus to a CANbus device), or two-way (e.g., over a local or wide area digital network to reach another computer system). Specific protocols and protocol stacks may be utilized for each of these networks and network interfaces of networks 1415 and network interfaces 1427 as described above.

[0142] The human machine interface devices, human accessible storage devices and network interface 1427 mentioned above may be connected to the core 1416 of the computer system 1400 .

[0143] The core 1412 includes one or more Central Processing Units (CPU) 1421, Graphics Processing Units (GPU) 1422, specialized programmable processing units in the form of Field Programmable Gate Arrays (FPGA) 1424, hardware accelerators for certain tasks 1426, etc. These devices are connected via a system bus 1428 along with a read only memory (ROM) 1419, a random access memory 1418, internal mass storage devices 1420 such as internal non-user accessible hard disk drives, SSDs, etc. In some computer systems, access to the system bus 1426 in the form of one or more physical plugs may allow expansion with additional CPUs, GPUs, etc. Peripheral devices are connected to the core's system bus 1428 directly or via a peripheral bus 1401. Peripheral bus architectures include PCI, USB, etc.

[0144] The CPU 1421, GPU 1422, FPGA 1424 and accelerator 1426 may execute a number of instructions, which, when combined, constitute the computer code referred to above. The computer code may be stored in ROM 1419 or RAM 1418. Temporary data may be stored in RAM 1418, and permanent data may be stored in, for example, internal mass storage device 1420. A cache memory may provide fast storage and retrieval from any of the memory devices, and the cache memory may be closely associated with one or more of the CPU 1421, GPU 1422, mass storage device 1420, ROM 1419, RAM 1418, etc.

[0145] The computer readable medium may have computer code thereon for performing various computer implemented operations, and the medium and computer code may be specially designed and constructed for the purposes of this disclosure, or may be of the type known and available to those skilled in the art of computer software.

[0146] By way of example and not limitation, the computer system 1400 having the architecture, and in particular the core 1416, may provide functionality by having a processor (including a CPU, GPU, FPGA, accelerator, etc.) execute software implemented in one or more tangible computer-readable media. Such computer-readable media may be the media related to mass storage accessible by a user as introduced above, as well as some storage of the core 1416 that is non-transitory in nature, such as the core internal mass storage 1420 or the ROM 1419. Software for implementing the embodiments of the present disclosure may be stored in such devices and executed by the core 1416. Depending on the particular needs, the computer-readable media may include one or more storage devices or chips. The software may cause the core 1416, and in particular the processor therein (including a CPU, GPU, FPGA, etc.), to execute a particular process or a particular part of a particular process described herein, to define data configurations stored in the RAM 1418, and to modify such data configurations based on the process defined by the software. Also, or alternatively, the computer system may provide functionality implemented in hardwired or otherwise circuitry (e.g., accelerator 1426) that may operate in place of or in conjunction with software to perform particular processes or portions of particular processes described herein. Where appropriate, references to software may include logic, and conversely, references to logic may include software. Where appropriate, references to computer-readable media may include circuitry (e.g., integrated circuits (ICs)) on which software is stored for execution, circuitry embodying logic for execution, or both. The present disclosure includes any appropriate combination of hardware and software.

[0147] While this disclosure has described certain exemplary embodiments, there are modifications, substitutions, and various substitute equivalents which are within the scope of this disclosure. It should thus be understood that many systems and methods which, although not explicitly shown or described herein, embody the principles of this disclosure and which are therefore within the spirit and scope of the subject matter of this disclosure may be contemplated by those skilled in the art.

[0148] (Appendix 1) 1. A method for video decoding, executed by at least one processor, comprising: selecting a set of weights in an intra-inter mode; applying position-dependent intra prediction combining (PDPC) to modify at least one of inter predicted samples and reconstructed samples of an inter coded coding unit (CU); The method includes: (Appendix 2) 2. The method of claim 1, wherein applying PDPC includes modifying only a luma component of the inter-coded CU. (Appendix 3) 3. The method of claim 1 or 2, further comprising the step of signaling an interPDPC flag (interPDPCFlag) in response to determining that the coding block (CB) is coded by merge mode. (Appendix 4) 2. The method of claim 1, further comprising applying a PDPC to the plurality of reconstructed samples before applying any of deblocking, sample adaptive offset (SAO), and an in-loop deblocking filter including an adaptive loop filter (ALF). (Appendix 5) assigning a default intra-prediction mode to a current coding block (CB); applying the default intra prediction mode to a CB next to the current CB, at least for intra mode coding and derivation of a most probable mode (MPM); 5. The method according to any one of claims 1 to 4, further comprising: (Appendix 6) applying a set of PDPC filters to at least one of inter predicted samples other than the inter predicted samples and reconstructed samples of inter coded CUs other than the reconstructed samples of the inter coded CUs; 2. The method of claim 1, wherein each PDPC filter in the set of PDPC filters is defined for a corresponding block size of a plurality of block sizes. (Appendix 7) signaling at least one of the interPDPC flags (interPDPCFlag) indicating whether a PDPC is applied; determining whether the interPDPCFlag is set equal to true; signaling a second flag in response to determining that the interPDPCFlag is set equal to a first value, the second flag indicating applying at least one PDPC filter to at least one of the inter predicted samples and the reconstructed samples of the inter coded CU after inter prediction; In response to determining that the interPDPCFlag is set equal to a second value, excluding a PDPC from at least one of an inter predicted sample and a reconstructed sample of an inter coded CU; 7. The method of claim 6, further comprising: (Appendix 8) signaling whether the weightings are equal to one another; in response to determining that the signaling indicates that the weightings are unequal, weighting inter prediction in PDPC filtering depending on whether the neighboring blocks are intra-coded, inter-coded, or intra-inter-coded; 8. The method according to any one of claims 1 to 7, further comprising: (Appendix 9) In the intra-inter mode, there is no intra prediction, 2. The method of claim 1, wherein the intra-inter mode includes directly applying PDPC on top of inter prediction. (Appendix 10) Applying PDPC directly on top of inter prediction includes:

number

number

[0149] [Patent Document 1] U.S. Pat. No. 9,609,343

Claims

1. A method executed by at least one processor of an encoding device, the method comprising: performing steps for video encoding to generate a bitstream and transmitting the bitstream to a decoding device, the steps for video encoding comprising: For position-dependent intra prediction combining (PDPC), based on whether the intra mode uses different initial weighting factors of PDPC corresponding to each of the different intra prediction modes for intra prediction, a set of weightings to be used for generating intra prediction samples for output in an intra-inter mode combining intra prediction and inter prediction, the set of weightings including weightings for each of the X and Y coordinates in the current block; applying the PDPC to modify at least one of inter predicted samples and reconstructed samples of an inter coded coding unit (CU) based on the set of weights; The method includes:

2. The method of claim 1 , wherein applying the PDPC includes modifying only a luma component of the inter-coded CU.

3. 2. The method of claim 1, wherein the flag for indicating whether to apply the PDPC is an inter-PDPC flag (interPDPCFlag) that is signaled in response to determining that a coding block (CB) included in the coding unit (CU) is coded by a merge mode.

4. 2. The method of claim 1, wherein the step for video encoding further comprises applying PDPC to the plurality of reconstructed samples before applying any of deblocking, sample adaptive offset (SAO), and an in-loop deblocking filter including an adaptive loop filter (ALF).

5. The steps for video encoding include: assigning a default intra-prediction mode to a current coding block (CB) included in the coding unit (CU); applying the default intra prediction mode to one or more CBs following the current CB, at least for intra mode coding and derivation of a most probable mode (MPM); The method of claim 1 further comprising:

6. The step for video encoding further comprises applying weighting coefficients of a PDPC filter used to apply the PDPC to at least one of the inter predicted samples and reconstructed samples of the inter coded CU; The method of claim 1 , wherein each PDPC filter in the PDPC filter weighting coefficients is defined for a corresponding block size of a plurality of block sizes.

7. The steps for video encoding include: signaling at least one of the interPDPC flags indicating whether the PDPC applies; determining whether the interPDPCFlag is set equal to true; signaling a second flag in response to determining that the interPDPCFlag is set equal to a first value, the second flag indicating applying at least one PDPC filter to at least one of the inter predicted samples and the reconstructed samples of the inter coded CU after inter prediction; in response to determining that the interPDPCFlag is set equal to a second value, excluding a PDPC from at least one of inter predicted samples and reconstructed samples of an inter coded CU; The method of claim 3 further comprising:

8. The steps for video encoding include: signaling whether the weightings are equal to one another; in response to determining that the signaling indicates that the weights are unequal, determining the set of weights depending on whether neighboring blocks adjacent to a current block to be decoded are intra-coded, inter-coded, or intra-inter-coded; The method of claim 1 further comprising:

9. In the intra-inter mode, there is no case where only intra prediction is performed, The method of claim 1 , wherein the intra-inter mode includes applying PDPC directly on top of inter prediction.

10. Applying PDPC directly on top of inter prediction includes: [0010] where InterPred indicates input inter prediction, and w_intra is selected from a set of predefined values ​​depending on whether the neighboring blocks adjacent to the current block to be decoded are at least one of intra-coded, inter-coded, or intra-inter-coded; 10. The method of claim 9, wherein N indicates any of 8, 16, 32, 64, 128 and 256.

11. A method executed by at least one processor of an encoding device, the method comprising: performing steps for video encoding to generate a bitstream and storing the bitstream in a storage device, the steps for video encoding comprising: For position-dependent intra prediction combining (PDPC), based on whether the intra mode uses different initial weighting factors of PDPC corresponding to each of the different intra prediction modes for intra prediction, a set of weightings to be used for generating intra prediction samples for output in an intra-inter mode combining intra prediction and inter prediction, the set of weightings including weightings for each of the X and Y coordinates in the current block; applying the PDPC to modify at least one of inter predicted samples and reconstructed samples of an inter coded coding unit (CU) based on the set of weights; The method includes:

12. 1. A computer program comprising instructions for causing at least one processor of an encoding device to perform steps for video encoding to generate a bitstream and to transmit said bitstream to a decoding device, said steps for video encoding comprising: For position-dependent intra prediction combining (PDPC), based on whether the intra mode uses different initial weighting factors of PDPC corresponding to each of the different intra prediction modes for intra prediction, a set of weightings to be used for generating intra prediction samples for output in an intra-inter mode combining intra prediction and inter prediction, the set of weightings including weightings for each of the X and Y coordinates in the current block; applying the PDPC to modify at least one of inter predicted samples and reconstructed samples of an inter coded coding unit (CU) based on the set of weights; A computer program comprising:

Images