Picture prediction method and device

Abstract

To provide a picture prediction method that improves efficiency of encoding and decoding.SOLUTION: A method includes the steps of receiving a bitstream, analyzing the bitstream to obtain a prediction mode of a current block, selecting the execution mode of a triangle prediction unit mode according to a preset policy when the prediction mode of the current block is the combined intra and inter prediction mode, determining motion information of the current block according to the execution mode of the triangle prediction unit mode, performing a motion compensation process on the basis of the motion information to obtain inter-prediction samples, obtaining intra-prediction samples according to the prediction mode of the current block, and combining the inter-predicted samples and the intra-predicted samples to obtain a predicted picture for the current block.SELECTED DRAWING: Figure 9

Description

[Technical field]

[0001] This application claims priority to Chinese Patent Application No. 201811262934.7, filed with the State Intellectual Property Office of the People's Republic of China on October 27, 2018, and entitled "Picture Prediction Method and Apparatus," the entirety of which is incorporated herein by reference.

[0002] This application relates to the field of video encoding and decoding, and more particularly to a picture prediction method and apparatus. [Background technology]

[0003] As Internet technology develops rapidly and people's material and spiritual culture is increasingly enriched, there is an increasing demand for the application of video in the Internet, especially the application of high-definition video. However, high-definition video has a very large amount of data. In order to transmit high-definition video in the Internet with limited bandwidth, it is necessary that the problem of video encoding and decoding be solved first. Video encoding and decoding is used in a wide range of digital video applications, such as broadcast digital television, video transmission over the Internet and mobile networks, real-time interactive applications such as video chat and video conferencing, DVD and Blu-ray discs, and video content collection and editing systems and security applications in camcorders.

[0004] Each picture in a video sequence is usually partitioned into a set of non-overlapping blocks, and coding is usually performed at the block level. For example, predictive blocks are generated through spatial (intra-picture) prediction and temporal (inter-picture) prediction. Correspondingly, prediction modes may include intra-prediction modes (spatial prediction) and inter-prediction modes (temporal prediction). The intra-prediction mode set may include 35 different intra-prediction modes, e.g., non-directional modes such as DC (or average) mode and planar mode, or directional modes such as those defined in H.265, or 67 different intra-prediction modes, e.g., non-directional modes such as DC (or average) mode and planar mode, or directional modes such as those defined in H.266, which is under development. The inter-prediction mode set depends on the available reference pictures and other inter-prediction parameters, e.g., whether the entire reference picture or only a part of the reference picture is used.

[0005] In the prior art, picture prediction is relatively inefficient. Summary of the Invention [Means for solving the problem]

[0006] An embodiment of this application (or this disclosure) provides a picture prediction apparatus and method.

[0007] According to a first aspect, an embodiment of the present invention provides a picture prediction method, which includes: receiving a bitstream; analyzing the bitstream to obtain a prediction mode of a current block; when the prediction mode of the current block is a combined intra and inter prediction mode, selecting an execution mode of a triangular prediction unit mode according to a preset policy; determining motion information of the current block according to the execution mode of the triangular prediction unit mode; performing a motion compensation process according to the motion information to obtain an inter prediction sample; obtaining an intra prediction sample according to the prediction mode of the current block; and combining the inter prediction sample and the intra prediction sample to obtain a predicted picture of the current block.

[0008] According to the method in the first aspect of the embodiment of the present invention, the efficiency of encoding and decoding can be improved.

[0009] According to a second aspect, an embodiment of the present invention provides a picture prediction method, the method includes: receiving a bitstream; analyzing the bitstream to obtain a triangular prediction unit mode of a current block; when the triangular prediction unit mode is used for the current block, selecting an execution mode of a combined intra and inter prediction mode according to a first preset policy; obtaining motion information of the current block according to the triangular prediction unit mode to obtain an inter prediction sample; and obtaining a predicted picture of the current block according to the execution mode and the inter prediction sample of the combined intra and inter prediction mode.

[0010] According to the method in the second aspect of the embodiment of the present invention, the efficiency of encoding and decoding can be improved.

[0011] According to a third aspect, an embodiment of the present invention provides an apparatus for decoding a video stream, comprising a processor and a memory, the memory storing instructions enabling the processor to perform the method of the first aspect.

[0012] According to a fourth aspect, an embodiment of the present invention provides an apparatus for decoding a video stream, comprising a processor and a memory, the memory storing instructions enabling the processor to perform the method of the second aspect.

[0013] According to a fifth aspect, there is provided a computer-readable storage medium. The computer-readable storage medium stores instructions that, when executed, enable one or more processors to encode or decode video data. The instructions enable the one or more processors to perform the method of any possible implementation of the first aspect.

[0014] According to a sixth aspect, an embodiment of the present invention provides a computer program comprising program code, which, when run on a computer, performs the method of any possible implementation of the first aspect.

[0015] According to a seventh aspect, there is provided a computer-readable storage medium. The computer-readable storage medium stores instructions that, when executed, enable one or more processors to encode or decode video data. The instructions enable the one or more processors to perform the method in any possible implementation of the second aspect.

[0016] According to an eighth aspect, an embodiment of the present invention provides a computer program comprising program code, which, when run on a computer, performs the method of any possible implementation of the second aspect.

[0017] The details of one or more embodiments are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description, drawings, and claims.

[0018] In order to more clearly describe the technical solutions in the embodiments or background of this application, the following describes the accompanying drawings required for the embodiments or background of this application. [Brief description of the drawings]

[0019] [Figure 1A] 1 is a block diagram illustrating an example video encoding system for implementing one embodiment of the present invention; [Figure 1B] 4 is a block diagram showing an example of a video coding system including any one or two of the encoder 20 in FIG. 2 and the decoder 30 in FIG. 3. [Diagram 2] FIG. 2 is a block diagram illustrating an exemplary structure of a video encoder for implementing one embodiment of the present invention; [Diagram 3] FIG. 2 is a block diagram illustrating an exemplary structure of a video decoder for implementing one embodiment of the present invention; [Figure 4] FIG. 2 is a block diagram showing an example of an encoding device or a decoding device. [Diagram 5] 11 is a block diagram showing an example of another encoding device or another decoding device. FIG. [Figure 6] 1 represents examples of spatial and temporal candidates for the current block. [Figure 7] FIG. 2 is a schematic diagram of an intra prediction mode. [Figure 8] FIG. 13 is a schematic diagram of a triangular prediction unit mode. [Figure 9] 1 is a schematic flow chart of a first embodiment of the present invention. [Figure 10] 1 is a schematic flow chart of a second embodiment of the present invention. [Figure 11] 1 is a schematic structural diagram of an apparatus 1100 according to an embodiment of the present invention. [Figure 12]31 is a block diagram showing an example structure of a content provision system 3100 for implementing a content distribution service. [Figure 13] FIG. 2 is a block diagram of an example structure of a terminal device.

[0020] In the following, identical reference signs denote identical or at least functionally equivalent features, unless specified otherwise. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] Video coding typically processes a sequence of pictures that form a video or a video sequence. In the field of video coding, the terms "picture", "frame", and "image" may be used synonymously. Video coding as used in this application (or this disclosure) refers to video encoding or video decoding. Video encoding is performed at the source side and typically involves processing the original video to reduce the amount of data required to represent the video (e.g., through compression) (for more efficient storage and / or transmission). Video decoding is performed at the destination side and typically involves the reverse process to the encoder to reconstruct the video picture. "Encoding" of a video picture in the embodiments should be understood as "encoding" or "decoding" in relation to a video sequence. A combination of encoding and decoding is also called coding (encoding and decoding).

[0022] Each picture in a video sequence is usually partitioned into a set of non-overlapping blocks, and coding is usually performed at the block level. Specifically, at the encoder side, video is usually processed, i.e., coded, at the block (also called picture block or video block) level. For example, a predictive block is generated through spatial (intra-picture) prediction and temporal (inter-picture) prediction, the predictive block is subtracted from a current block (a block being or to be processed) to obtain a residual block, and the residual block is transformed and quantized (compressed) in the transform domain to reduce the amount of data to be transmitted. At the decoder side, the inverse process to the encoder is applied to the coded or compressed block to reconstruct the current block for representation. In addition, the encoder replicates the processing loop of the decoder, so that the encoder and decoder generate the same prediction (e.g., intra-prediction and inter-prediction) and / or reconstruction to process, i.e., code, subsequent blocks.

[0023] The term "block" may be a portion of a picture or a frame. Important terms in this application are defined as follows:

[0024] Current block: The current block is the block being processed. For example, during encoding, the current block is the block being encoded, and during decoding, the current block is the block being decoded. If the block being processed is a chroma component block, the block is called a current chroma block. The luma block corresponding to the current chroma block may be called a current luma block.

[0025] Reference Block: A reference block is a block that provides a reference signal for a current block. In the search process, multiple reference blocks may be traversed to find the most suitable reference block.

[0026] Prediction Block: A block that provides a prediction for a current block is called a prediction block. For example, after multiple reference blocks are traversed, a best reference block is found. The best reference block provides a prediction for the current block, and the block is called a prediction block.

[0027] Picture block signal: A picture block signal is a pixel value, a sample value, or a sample signal within a picture block.

[0028] Prediction signal: The pixel values, sample values, or sample signals in a prediction block are called the prediction signal.

[0029] The following describes an embodiment of the encoder 20, the decoder 30 and the encoding system 10 based on FIGS. 1A and 1B to 3. FIG.

[0030] 1A is a conceptual or schematic block diagram illustrating an example encoding system 10, e.g., a video encoding system 10 that may use the techniques of this application (this disclosure). An encoder 20 (e.g., video encoder 20) and a decoder 30 (e.g., video decoder 30) in the video encoding system 10 represent example devices that may be configured to perform intra prediction according to various examples described in this application. As depicted in FIG. 1A, the encoding system 10 includes a source device 12, or the like, configured to provide encoded data 13, such as an encoded picture 13, to a destination device 14 for decoding the encoded data 13.

[0031] Source device 12 includes an encoder 20 , and may additionally or optionally include a picture source 16 , a preprocessing unit 18 , such as a picture preprocessing unit 18 , and a communications interface or unit 22 .

[0032] Picture source 16 may include or be any type of picture capture device configured to capture real-world pictures and the like, and / or any type of device for generating pictures or comments (for encoding screen content, some text on the screen is considered as part of the picture to be encoded), such as a computer graphics processing unit configured to generate computer animated pictures, or any type of device configured to obtain and / or provide real-world or computer animated pictures (e.g. screen content and virtual reality (VR) pictures) and / or any combination thereof (e.g. augmented reality (AR) pictures).

[0033] A picture may be considered as a two-dimensional array or matrix of samples with luma values. The samples in the array may be called pixels (short for picture element) or pels. The number of samples in the horizontal and vertical directions (or axes) of the array or picture defines the size and / or resolution of the picture. For color representation, three color components are usually used, i.e., a picture may be represented as or contain three sample arrays. In an RBG format or color space, a picture contains corresponding red, green, and blue sample arrays. However, in video coding, each pixel is usually represented in a luma / chrominance format or color space, e.g., a picture in YCbCr format contains a luma component denoted by Y (sometimes denoted by L) and two chrominance components denoted by Cb and Cr. The luma (or luma for short) component Y indicates the brightness or gray level intensity (e.g., the two are the same in a grayscale picture), and the two chroma (or chroma for short) components Cb and Cr represent the chroma or color information components. Thus, a picture in YCbCr format contains a luma sample array of luma sample values ​​(Y) and two chroma sample arrays of chroma values ​​(Cb and Cr). A picture in RGB format may be converted or transformed into a picture in YCbCr format, and vice versa. This process is also called color conversion or color transformation. If the picture is monochrome, the picture may contain only a luma sample array.

[0034] The picture source 16 (e.g., video source 16) may be, for example, a camera configured to capture a picture, a memory such as, for example, a picture memory that contains or stores previously captured or generated pictures, and / or any type of (internal or external) interface for acquiring or receiving pictures. The camera may be, for example, a local camera or an integrated camera integrated in the source device, and the memory may be a local memory or an integrated memory integrated in the source device. The interface may be, for example, an external interface for receiving pictures from an external video source. The external video source is, for example, an external picture capturing device such as a camera, an external memory, or an external picture generating device. The external picture generating device is, for example, an external computer graphics processing unit, a computer, or a server. The interface may be any type of interface according to any dedicated or standardized interface protocol, for example, a wired or wireless interface or an optical interface. The interface for acquiring picture data 17 may be the same interface as the communication interface 22 or may be part of the communication interface 22.

[0035] Different from pre-processing unit 18 and the processing performed by pre-processing unit 18, picture 17 or picture data 17 (eg, video data 16) may also be referred to as original picture 17 or original picture data 17.

[0036] Pre-processing unit 18 is configured to receive (original) picture data 17 and perform pre-processing on the picture data 17 to obtain pre-processed picture 19 or pre-processed picture data 19. For example, the pre-processing performed by pre-processing unit 18 may include cropping, color format conversion (e.g., from RGB to YCbCr), color correction, or noise reduction. It may be understood that pre-processing unit 18 may be an optional component.

[0037] The encoder 20 (e.g., video encoder 20) is configured to receive pre-processed picture data 19 and provide encoded picture data 21 (details are further described below, e.g., based on FIG. 2 or FIG. 4). In one example, the encoder 20 may be configured to perform the following embodiments 1 to 7.

[0038] The communications interface 22 of the source device 12 may be configured to receive the encoded picture data 21 and transmit the encoded picture data 21 to another device, e.g., the destination device 14 or any other device, for storage or direct reconstruction, or correspondingly, to process the encoded picture data 21 before storing the encoded data 13 and / or transmitting the encoded data 13 to another device, e.g., the destination device 14 or any other device used for decoding or storage.

[0039] Destination device 14 includes a decoder 30 (eg, a video decoder 30), and may additionally or optionally include a communications interface or unit 28, a post-processing unit 32, and a display device 34.

[0040] For example, communication interface 28 of destination device 14 is configured to receive encoded picture data 21 or encoded data 13 directly from source device 12 or any other source, such as a storage device, which may be, for example, a coded picture data storage device.

[0041] Communications interface 22 and communications interface 28 may be configured to transmit or receive encoded picture data 21 or encoded data 13 over a direct communications link between source device 12 and destination device 14 or over any type of network. The direct communications link may be, for example, a direct wired or wireless connection, and the any type of network may be, for example, a wired or wireless network or any combination thereof, or any type of private or public network or any combination thereof.

[0042] The communications interface 22 may be configured, for example, to encapsulate the encoded picture data 21 into a suitable format, such as packets, for transmission over a communications link or network.

[0043] Communications interface 28, as a corresponding part of communications interface 22, may be configured to decapsulate encoded data 13 to obtain encoded picture data 21 and the like.

[0044] Both communication interface 22 and communication interface 28 may be configured as a unidirectional communication interface, e.g., an arrow pointing from source device 12 to destination device 14, used for encoded picture data 13 in FIG. 1A, or may be configured as a bidirectional communication interface, e.g., configured to send and receive messages to establish a connection, confirm and exchange a communication link and / or any other information related to a data transmission, such as an encoded picture data transmission.

[0045] The decoder 30 is configured to receive the encoded picture data 21 and provide decoded picture data 31 or decoded pictures 31 (details are further described below, e.g., based on FIG. 3 or FIG. 5). In one example, the decoder 30 may be configured to perform the following embodiments 1 to 7.

[0046] Post-processing unit 32 of destination device 14 is configured to post-process decoded picture data 31 (also referred to as reconstructed picture data), such as decoded picture 131, to obtain post-processed picture data 33, such as post-processed picture 33. The post-processing performed by post-processing unit 32 may include, for example, color format conversion (e.g., from YCbCr to RGB), color correction, cropping, resampling, or any other processing to prepare decoded picture data 31 for display by display device 34.

[0047] A display device 34 of destination device 14 is configured to receive the post-processed picture data 33 for displaying the picture to a user, viewer, or the like. The display device 34 may be or include any type of display configured to present the reconstructed picture, such as an integrated or external display or monitor. For example, the display may include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro-LED display, a liquid crystal on silicon (LCoS) display, a digital light processor (DLP), or any other type of display.

[0048] 1A illustrates source device 12 and destination device 14 as separate devices, an embodiment of the devices may also include both source device 12 and destination device 14, or the functionality of source device 12 and the functionality of destination device 14, i.e., both source device 12 or corresponding functionality and destination device 14 or corresponding functionality. In such an embodiment, source device 12 or corresponding functionality and destination device 14 or corresponding functionality may be implemented using the same hardware and / or software, separate hardware and / or software, or any combination thereof.

[0049] Based on these descriptions, a person skilled in the art will readily understand that the functions of different units of source device 12 and / or destination device 14 depicted in FIG. 1A or the presence and (exact) division of one or more functions may vary in actual devices and applications.

[0050] Encoder 20 (e.g., video encoder 20) and decoder 30 (e.g., video decoder 30) may each be implemented as any one of a variety of suitable circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, or any combination thereof. If the techniques are implemented in part in software, a device may store software instructions in a suitable non-transitory computer-readable storage medium and execute the instructions in hardware to perform the techniques in this disclosure by using one or more processors. Any of the foregoing (including hardware, software, combinations of hardware and software, and the like) may be considered as one or more processors. Video encoder 20 and video decoder 30 may each be included in one or more encoders or decoders, any one of which may be integrated as part of an encoder / decoder combination (codec) in a corresponding device.

[0051] Source device 12 may also be referred to as a video encoding device or a video encoding apparatus. Destination device 14 may also be referred to as a video decoding device or a video decoding apparatus. Source device 12 and destination device 14 may each be an example of a video encoding device or a video decoding apparatus.

[0052] Source device 12 and destination device 14 may each include any one of a variety of devices, including any type of handheld or stationary device, such as a notebook or laptop computer, a mobile phone, a smartphone, a tablet or tablet computer, a video camera, a desktop computer, a set-top box, a television, a display device, a digital media player, a video game console, a video streaming transmission device (such as a content service server and a content delivery server), a broadcast receiver device, or a broadcast transmitter device, and may or may not use any type of operating system.

[0053] In some cases, source device 12 and destination device 14 may be equipped with wireless communication. Thus, source device 12 and destination device 14 may be wireless communication devices.

[0054] In some cases, the video encoding system 10 depicted in FIG. 1A is merely an example, and the techniques in this application may be applied to video coding settings (e.g., video encoding or video decoding) that do not need to include any data communication between an encoding device and a decoding device. In another example, data may be retrieved from a local memory, streamed over a network, or the like. A video encoding device may encode data and store the data in a memory, and / or a video decoding device may retrieve data from a memory and decode the data. In some examples, encoding and decoding are performed by devices that do not communicate with each other, but simply encode data in a memory and / or retrieve data from a memory and decode the data.

[0055] It should be understood that for each of the foregoing examples described with reference to video encoder 20, video decoder 30 may be configured to perform the inverse process. For signaling syntax elements, video decoder 30 may be configured to receive and parse the syntax elements and decode the associated video data accordingly. In some examples, video encoder 20 may entropy encode the syntax elements into an encoded video bitstream. In such examples, video decoder 30 may parse the syntax elements and decode the associated video data accordingly.

[0056] 1B is a diagram illustrating an example of a video coding system 40 including the encoder 20 in FIG. 2 and / or the decoder 30 in FIG. 3, according to an example embodiment. The system 40 may implement a combination of various techniques of this application. In the illustrated embodiment, the video coding system 40 may include an imaging device 41, a video encoder 20, a video decoder 30 (and / or a video decoder implemented by logic circuitry 47 of a processing unit 46), an antenna 42, one or more processors 43, one or more memories 44, and / or a display device 45.

[0057] As depicted in the figure, imaging device 41, antenna 42, processing unit 46, logic circuitry 47, video encoder 20, video decoder 30, processor 43, memory 44, and / or display device 45 may be in communication with one another. As illustrated, video coding system 40 is shown having both video encoder 20 and video decoder 30, but in different examples, video coding system 40 may include only video encoder 20 or only video decoder 30.

[0058] In some examples, as depicted in the figure, the video coding system 40 may include an antenna 42. For example, the antenna 42 may be configured to transmit or receive an encoded bitstream of video data. In addition, in some examples, the video coding system 40 may include a display device 45. The display device 45 may be configured to present the video data. In some examples, as depicted in the figure, the logic circuitry 47 may be realized by a processing unit 46. The processing unit 46 may include application-specific integrated circuit (ASIC) logic, a graphics processing unit, a general-purpose processor, or the like. The video coding system 40 may also include an optional processor 43. The optional processor 43 may also include application-specific integrated circuit (ASIC) logic, a graphics processing unit, a general-purpose processor, or the like. In some examples, the logic circuitry 47 may be realized by hardware, such as dedicated video coding hardware, and the processor 43 may be realized by universal software, an operating system, or the like. Additionally, memory 44 may be any type of memory, such as volatile memory (e.g., Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM)) or non-volatile memory (e.g., flash memory). In a non-limiting example, memory 44 may be implemented by a cache memory. In some examples, logic circuitry 47 may access memory 44 (e.g., to implement a picture buffer). In another example, logic circuitry 47 and / or processing unit 46 may include memory (e.g., a cache) to implement a picture buffer or the like.

[0059] In some examples, video encoder 20 implemented by logic circuitry may include a picture buffer (e.g., implemented by processing unit 46 or memory 44) and a graphics processing unit (e.g., implemented by processing unit 46). The graphics processing unit may be communicatively coupled to the picture buffer. The graphics processing unit may include video encoder 20 implemented by logic circuitry 47 to implement the various modules described with reference to FIG. 2 and / or any other encoder system or subsystem described herein. The logic circuitry may be configured to perform various operations described herein.

[0060] Video decoder 30 may likewise be implemented by logic circuitry 47 to implement the various modules described with reference to decoder 30 in FIG. 3 and / or any other decoder system or subsystem described herein. In some examples, video decoder 30 implemented by logic circuitry may include a picture buffer (implemented by processing unit 2820 or memory 44) and a graphics processing unit (e.g., implemented by processing unit 46). The graphics processing unit may be communicatively coupled to the picture buffer. The graphics processing unit may include video decoder 30 implemented by logic circuitry 47 to implement the various modules described with reference to FIG. 3 and / or any other decoder system or subsystem described herein.

[0061] In some examples, antenna 42 of video coding system 40 may be configured to receive an encoded bitstream of video data. As described, the encoded bitstream may include data, indicators, index values, mode selection data, or the like, related to the video frame coding described herein, such as data related to the coding partition (e.g., transform coefficients or quantized transform coefficients, optional indicators (as described), and / or data defining the coding partition). Video coding system 40 may further include a video decoder 30 coupled to antenna 42 and configured to decode the encoded bitstream. Display device 45 is configured to present the video frames.

[0062] Encoder and encoding method FIG. 2 is a schematic / conceptual block diagram of an example of a video encoder 20 configured to implement the techniques of this application (disclosure). In the example of FIG. 2, the video encoder 20 includes a residual calculation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, a reconstruction unit 214, a buffer 216, a loop filter unit 220, a decoded picture buffer (DPB) 230, a prediction processing unit 260, and an entropy coding unit 270. The prediction processing unit 260 may include an inter prediction unit 244, an intra prediction unit 254, and a mode selection unit 262. The inter prediction unit 244 may include a motion estimation unit and a motion compensation unit (not shown in the figure). The video encoder 20 depicted in FIG. 2 may also be referred to as a hybrid video encoder or a hybrid video codec-based video encoder.

[0063] For example, the residual calculation unit 204, the transform processing unit 206, the quantization unit 208, the prediction processing unit 260, and the entropy coding unit 270 form the forward signal path of the encoder 20, while the inverse quantization unit 210, the inverse transform processing unit 212, the reconstruction unit 214, the buffer 216, the loop filter unit 220, the decoded picture buffer (DPB) 230, the prediction processing unit 260, and the like form the backward signal path of the encoder. The backward signal path of the encoder corresponds to the signal path of the decoder (referred to as decoder 30 in FIG. 3).

[0064] The encoder 20 receives a picture 201 or a block 203 of a picture 201, e.g., a picture in a sequence of pictures forming a video or a video sequence, by using an input 202 or the like. The picture block 203 may also be referred to as a current picture block or a picture block to be coded, and the picture 201 may also be referred to as a current picture or a picture to be coded (particularly when the current picture is distinguished from another picture in video coding, e.g., when the other pictures in the same video sequence also include previously coded and / or decoded pictures in the video sequence of the current picture).

[0065] classification An embodiment of encoder 20 may include a partitioning unit (not shown in FIG. 2) configured to partition picture 201 into multiple non-overlapping blocks, such as block 203. The partitioning unit may be configured to use the same block size and corresponding raster that defines the block size for all pictures in a video sequence, or may be configured to vary block sizes between pictures, subsets, or groups of pictures and partition each picture into corresponding blocks.

[0066] In one example, prediction processing unit 260 of video encoder 20 may be configured to perform any combination of the aforementioned partitioning techniques.

[0067] For example, in picture 201, block 203 is or can be considered as a two-dimensional array or matrix of samples having luma values ​​(sample values), but the size of block 203 is smaller than the size of picture 201. In other words, block 203 may include, for example, one sample array (e.g., a luma array in case of a monochrome picture 201), three sample arrays (e.g., one luma array and two chroma arrays in case of a color picture), or any other number and / or type of arrays based on the color format used. The number of samples in the horizontal and vertical directions (or axes) of block 203 defines the size of block 203.

[0068] The encoder 20 depicted in FIG. 2 is arranged to code a picture 201 block-by-block, eg to perform encoding and prediction for each block 203 .

[0069] Residual calculation The residual calculation unit 204 is configured to calculate a residual block 205 based on the picture block 203 and a prediction block 265 (further details regarding the prediction block 265 are provided below), for example to obtain the residual block 205 in the sample domain by subtracting sample values ​​of the prediction block 265 from sample values ​​of the picture block 203 on a sample-by-sample (pixel-by-pixel) basis.

[0070] conversion The transform processing unit 206 is configured to apply a transform, such as a discrete cosine transform (DCT) or a discrete sine transform (DST), to the sample values ​​of the residual block 205 to obtain transform coefficients 207 in the transform domain. The transform coefficients 207 may also be referred to as residual transform coefficients, and refer to the residual block 205 in the transform domain.

[0071] The transform processing unit 206 may be configured to apply an integer approximation of the DCT / DST, e.g., a transform specified in HEVC / H.265. This integer approximation is usually proportionally scaled by a factor that is compared to an orthogonal DCT transform. To maintain the norm of the residual block obtained through the forward transform and the inverse transform, an additional scale factor is applied as part of the transform process. The scale factor is usually selected based on some constraints, e.g., a power of two, a bit depth of the transform coefficients, or a trade-off between the accuracy used for the shift operation and the implementation cost. For example, a particular scale factor may be specified for the inverse transform at the decoder 30 side by using the inverse transform processing unit 212 (and correspondingly, for the inverse transform at the encoder 20 side by using the inverse transform processing unit 212 or the like), and correspondingly, a corresponding scale factor may be specified for the forward transform at the encoder 20 side by using the transform processing unit 206.

[0072] Quantization The quantization unit 208 is configured to quantize the transform coefficients 207 by applying scale quantization, vector quantization, or the like to obtain quantized transform coefficients 209. The quantized transform coefficients 209 may also be referred to as quantized residual coefficients 209. The quantization process may reduce a bit depth associated with some or all of the transform coefficients 207. For example, an n-bit transform coefficient may be rounded down to an m-bit transform coefficient during quantization, where n is greater than m. The degree of quantization may be changed by adjusting a quantization parameter (QP). For example, in scale quantization, different scales may be applied to achieve finer or coarser quantization. A smaller quantization step corresponds to a finer quantization, and a larger quantization step corresponds to a coarser quantization. An appropriate quantization step may be indicated by using a quantization parameter (QP). For example, the quantization parameter may be an index of a set of predefined appropriate quantization steps. For example, a smaller quantization parameter may correspond to a finer quantization (smaller quantization step), and a larger quantization parameter may correspond to a coarser quantization (larger quantization step), and vice versa. Quantization may include division by a quantization step and corresponding quantization or inverse quantization performed by an inverse quantization unit 210 or the like, or may include multiplication by a quantization step. In embodiments according to some standards such as HEVC, a quantization parameter may be used to determine a quantization step. In general, the quantization step may be calculated based on the quantization parameter through a fixed-point approximation of a mathematical formula that includes a division. An additional scale factor may be introduced for quantization and inverse quantization to restore the norm of the residual block, which may be changed due to the scale used in the fixed-point approximation of the mathematical formula used for the quantization step and the quantization parameter. In an example implementation, the scale of the inverse transform may be combined with the scale of the inverse quantization.Instead, customized quantization tables may be used and signaled from the encoder to the decoder, e.g., in the bitstream. Quantization is a lossy operation, with larger quantization steps indicating larger losses.

[0073] The inverse quantization unit 210 is configured to apply the inverse quantization of the quantization unit 208 to the quantized coefficients to obtain inverse quantized coefficients 211, e.g., apply an inverse quantization scheme of the quantization scheme applied by the quantization unit 208 based on or by using the same quantization step as the quantization unit 208. The inverse quantized coefficients 211, also referred to as the inverse quantized residual coefficients 211, may correspond to the transform coefficients 207, although the loss caused by quantization is usually different from that of the transform coefficients.

[0074] The inverse transform processing unit 212 is configured to apply an inverse transform of the transform applied by the transform processing unit 206, for example, an inverse discrete cosine transform (DCT) or an inverse discrete sine transform (DST), to obtain an inverse transform block 213 in the sample domain. The inverse transform block 213 may also be referred to as an inverse transform inverse quantization block 213 or an inverse transform residual block 213.

[0075] The reconstruction unit 214 (e.g., adder 214) is configured to add the inverse transform block 213 (i.e., the reconstructed residual block 213) to the prediction block 265 to obtain a reconstructed block 215 in the sample domain, for example by adding sample values ​​of the reconstructed residual block 213 and sample values ​​of the prediction block 265.

[0076] Optionally, a buffer unit 216 (or "buffer" 216 for short), such as a line buffer 216, is configured to buffer or store the reconstructed blocks 215 and corresponding sample values ​​for intra prediction and the like. In other embodiments, the encoder may be configured to use the unfiltered reconstructed blocks and / or corresponding sample values ​​stored in the buffer unit 216 for any type of estimation and / or prediction, such as intra prediction.

[0077] For example, embodiments of the encoder 20 may be configured such that the buffer unit 216 is not only configured to store the reconstructed blocks 215 for intra prediction 254, but also to store the filtered blocks 221 (not shown in FIG. 2) of the loop filter unit 220, and / or the buffer unit 216 and the decoded picture buffer unit 230 form one buffer. Other embodiments may be used to use the filtered blocks 221 and / or blocks or samples from the decoded picture buffer 230 (not shown in FIG. 2) as input or basis for intra prediction 254.

[0078] Loop filter unit 220 (or “loop filter” 220 for short) is configured to perform filtering on reconstructed block 215 to smoothly perform pixel transformation or improve video quality to obtain filtered block 221. Loop filter unit 220 is intended to represent one or more loop filters, e.g., a deblocking filter, a sample-adaptive offset (SAO) filter, or another filter, such as a bilateral filter, an adaptive loop filter (ALF), a sharpening or smoothing filter, or a collaborative filter. Although loop filter unit 220 is depicted as an in-loop filter in FIG. 2, loop filter unit 220 may be realized as a post-loop filter in other configurations. Filtered block 221 may also be referred to as filtered reconstructed block 221. Decoded picture buffer 230 may store the reconstructed coded block after loop filter unit 220 performs a filtering operation on the reconstructed coded block.

[0079] An embodiment of encoder 20 (correspondingly, loop filter unit 220) may be used to output loop filter parameters (e.g., sample adaptive offset information) such that, for example, the loop filter parameters are output directly or the entropy encoding unit 270 or any other entropy encoding unit performs entropy encoding and then the decoder 30 can receive and apply the same loop filter parameters for decoding and the like.

[0080] The decoded picture buffer (DPB) 230 may be a reference picture memory that stores reference picture data for the video encoder 20 to encode the video data. The DPB 230 may be any one of a number of memories, such as a dynamic random access memory (DRAM) (including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), or resistive RAM (RRAM)), or another type of memory. The DPB 230 and the buffer 216 may be provided by the same memory or separate memories. In one example, the decoded picture buffer (DPB) 230 is configured to store the filtered block 221. The decoded picture buffer 230 may further be configured to store other previous filtered blocks, such as previous reconstructed and filtered block 221, of the same current picture or a different picture, such as a previous reconstructed picture, and may provide a complete previous reconstructed, i.e., decoded picture (and corresponding reference blocks and corresponding samples) and / or a partially reconstructed current picture (and corresponding reference blocks and corresponding samples) for inter prediction and the like. In one example, if the reconstructed block 215 is reconstructed without an in-loop filter, the decoded picture buffer (DPB) 230 is configured to store the reconstructed block 215.

[0081] The prediction processing unit 260, also referred to as the block prediction processing unit 260, is configured to receive or obtain the block 203 (the current block 203 of the current picture 201) and reconstructed picture data, e.g., reference samples from the same (current) picture in the buffer 216 and / or reference picture data 231 from one or more previous decoded pictures in the decoded picture buffer 230, and to process such data for prediction, i.e., to provide a prediction block 265, which may be an inter prediction block 245 or an intra prediction block 255.

[0082] The mode selection unit 262 may be configured to select a prediction mode (e.g., intra or inter prediction mode) and / or a corresponding prediction block 245 or 255 as a prediction block 265 to calculate the residual block 205 and reconstruct the reconstructed block 215.

[0083] An embodiment of the mode selection unit 262 may be used to select a prediction mode (e.g., from among prediction modes supported by the prediction processing unit 260). The prediction mode provides the best match or the smallest residual (smallest residual means better compression in transmission or storage), or provides the smallest signaling overhead (smallest signaling overhead means better compression in transmission or storage), or takes into account the two, or balances the two. The mode selection unit 262 may be configured to determine the prediction mode based on rate distortion optimization (RDO), i.e., select a prediction mode that provides the smallest rate distortion optimization, or select a prediction mode whose associated rate distortion at least satisfies a prediction mode selection criterion.

[0084] The prediction processing (eg, using prediction processing unit 260) and mode selection (eg, using mode selection unit 262) performed by an example encoder 20 are described in detail below.

[0085] As mentioned above, encoder 20 is configured to determine or select a best or optimal prediction mode from among a (predetermined) prediction mode set, which may include, for example, intra-prediction modes and / or inter-prediction modes.

[0086] The intra-prediction mode set may include 35 different intra-prediction modes, or may include 67 different intra-prediction modes, or may include the intra-prediction modes defined in the currently developing H.266.

[0087] The inter prediction mode set depends on the available reference pictures (e.g., at least a portion of the decoded pictures stored in DBP230) and other inter prediction parameters, such as whether the entire reference picture is used or only a portion of the reference picture is used, e.g., whether a search window region surrounding the area of ​​the current block is searched for the best matching reference block, and / or whether pixel interpolation such as half pixel and / or quarter pixel interpolation is applied.

[0088] In addition to the aforementioned prediction modes, skip mode and / or direct mode may also be applied.

[0089] Prediction processing unit 260 may be further configured to partition block 203 into smaller block partitions or sub-blocks, e.g., using a quad-tree (QT) partition, a binary-tree (BT) partition, a triple-tree (TT) partition, or any combination thereof iteratively, and perform prediction on each of the block partitions or sub-blocks. Mode selection includes selecting a tree structure for block 203 to be partitioned and selecting a prediction mode to be applied to each of the block partitions or sub-blocks.

[0090] The inter prediction unit 244 may include a motion estimation (ME) unit (not shown in FIG. 2) and a motion compensation (MC) unit (not shown in FIG. 2). The motion estimation unit is configured to receive or obtain a picture block 203 (current picture block 203 of current picture 201) and a decoded picture 231, or at least one or more reconstructed blocks, e.g., one or more other reconstructed blocks different from the previous decoded picture 231, to perform motion estimation. For example, a video sequence may include the current picture and the previous decoded picture 31. In other words, the current picture and the previous decoded picture 31 may be part of a picture sequence that forms a video sequence or forms a picture sequence.

[0091] For example, the encoder 20 may be configured to select a reference block from among multiple reference blocks of the same picture or different pictures among multiple other pictures, and provide the reference picture (or reference picture index) and / or an offset (spatial offset) between the position (X, Y coordinates) of the reference block and the position of the current block as an inter prediction parameter to a motion estimation unit (not shown in FIG. 2). This offset is also called a motion vector (MV).

[0092] The motion compensation unit is configured to obtain, for example, receive, inter prediction parameters and perform inter prediction by obtaining inter prediction block 245 based on or using the inter prediction parameters. The motion compensation performed by the motion compensation unit (not shown in FIG. 2) may include fetching or generating a prediction block based on a motion / block vector determined by motion estimation (possibly performing interpolation with sub-pixel accuracy). During the interpolation filtering, additional samples may be generated from known samples, thereby potentially increasing the number of candidate prediction blocks that may be used to encode the picture block. Once the motion vector used for the PU of the current picture block is received, the motion compensation unit 246 may locate the prediction block to which the motion vector points in the reference picture list. The motion compensation unit 246 may further generate syntax elements associated with the block and the video slice, so that the video decoder 30 uses the syntax elements when decoding the picture blocks of the video slice.

[0093] Inter prediction is a technique based on motion compensation, and is mainly treated as determining motion information of a current block, obtaining a reference picture block from a reference frame based on the motion information, and generating a predicted picture of the current block. The motion information includes an inter prediction direction, a reference frame index (ref_idx), a motion vector (MV), etc. During inter prediction, the inter prediction direction is used to indicate the prediction direction of forward prediction, backward prediction, or bidirectional prediction used for the current block, the reference frame index is used to indicate a reference frame, and the motion vector is used to indicate the position offset of a reference block of a current block in a reference frame relative to a current block in a current frame. The motion vector indicates a displacement vector of a reference picture block in a reference frame used to predict the current block relative to the current block. Thus, one motion vector corresponds to one reference picture block.

[0094] The intra prediction unit 254 is configured to obtain, e.g., receive, one or more previous reconstructed blocks, such as the picture block 203 (current picture block) of the same picture and reconstructed neighboring blocks, to perform intra estimation. For example, the encoder 20 may be configured to select an intra prediction mode from among multiple intra prediction modes.

[0095] An embodiment of the encoder 20 may be configured to select an intra prediction mode based on an optimization criterion, for example, based on the minimum residual (e.g., the intra prediction mode that provides the prediction block 255 that is most similar to the current picture block 203) or the minimum rate distortion.

[0096] The intra prediction unit 254 is further configured to determine an intra-predicted block 255 based on the intra prediction parameters of the selected intra prediction mode. In any case, after selecting the intra prediction mode to be used for the block, the intra prediction unit 254 is further configured to provide the intra prediction parameters to the entropy coding unit 270, i.e., to provide information indicating the selected intra prediction mode to be used for the block. In one example, the intra prediction unit 254 may be configured to perform any combination of the following intra prediction techniques:

[0097] The entropy encoding unit 270 is configured to apply an entropy encoding algorithm or scheme (e.g., a variable length coding (VLC) scheme, a context adaptive VLC (CAVLC) scheme, an arithmetic coding scheme, a context adaptive binary arithmetic coding (CABAC) scheme, a syntax-based context-adaptive binary arithmetic coding (SBAC) scheme, a probability interval partitioning entropy (PIPE) coding scheme, or another entropy encoding method or technique) to one or more (or none) of the quantized residual coefficients 209, the inter prediction parameters, the intra prediction parameters, and / or the loop filter parameters, to obtain encoded picture data 21, which can be output using an output 272, e.g., in the form of an encoded bitstream 21. The encoded bitstream may be transmitted to video decoder 30 or archived for later transmission or retrieval by video decoder 30. Entropy encoding unit 270 may be further configured to perform entropy encoding on another syntax element of the current video slice being encoded.

[0098] Other structural variations of the video encoder 20 may be configured to encode the video stream. For example, a non-transform-based encoder 20 may directly quantize the residual signal without the transform processing unit 206 for some blocks or frames. In another implementation, the encoder 20 may have the quantization unit 208 and the inverse quantization unit 210 combined into one unit.

[0099] 3 illustrates an example of a video decoder 30 configured to implement the techniques of the present application. The video decoder 30 is configured to receive encoded picture data (e.g., an encoded bitstream) 21, such as encoded by an encoder 20, to obtain a decoded picture 231. In the decoding process, the video decoder 30 receives video data from the video encoder 20, e.g., an encoded video bitstream indicating picture blocks and associated syntax elements of an encoded video slice.

[0100] 3, decoder 30 includes an entropy decoding unit 304, an inverse quantization unit 310, an inverse transform processing unit 312, a reconstruction unit 314 (e.g., summer 314), a buffer 316, a loop filter 320, a decoded picture buffer 330, and a prediction processing unit 360. Prediction processing unit 360 may include an inter prediction unit 344, an intra prediction unit 354, and a mode selection unit 362. In some examples, video decoder 30 may perform a decoding traversal that is generally inverse to the encoding traversal described with reference to video encoder 20 of FIG.

[0101] Entropy decoding unit 304 is configured to perform entropy decoding on encoded picture data 21 to obtain quantized coefficients 309, decoded coding parameters (not shown in FIG. 3), etc., such as, for example, any one or all of inter-prediction parameters, intra-prediction parameters, loop filter parameters, and / or other syntax elements (decoded). Entropy decoding unit 304 is further configured to forward the inter-prediction parameters, intra-prediction parameters, and / or other syntax elements to prediction processing unit 360. Video decoder 30 may receive video slice level syntax elements and / or video block level syntax elements.

[0102] The inverse quantization unit 310 may have the same function as the inverse quantization unit 110, the inverse transform processing unit 312 may have the same function as the inverse transform processing unit 212, the reconstruction unit 314 may have the same function as the reconstruction unit 214, the buffer 316 may have the same function as the buffer 216, the loop filter 320 may have the same function as the loop filter 220, and the decoded picture buffer 330 may have the same function as the decoded picture buffer 230.

[0103] The prediction processing unit 360 may include an inter prediction unit 344 and an intra prediction unit 354. The inter prediction unit 344 may have functionality similar to that of the inter prediction unit 244, and the intra prediction unit 354 may have functionality similar to that of the intra prediction unit 254. The prediction processing unit 360 is typically configured to perform block prediction and / or obtain a prediction block 365 from the encoded data 21, and to receive or obtain (explicitly or implicitly) prediction-related parameters and / or information regarding a selected prediction mode, e.g., from the entropy decoding unit 304.

[0104] When a video slice is coded as an intra-coded (I) slice, the intra prediction unit 354 of the prediction processing unit 360 is configured to generate a prediction block 365 used for a picture block of the current video slice based on the signaled intra prediction mode and data from a previous decoded block of the current frame or picture. When a video frame is coded as an inter-coded (i.e., B or P) slice, the inter prediction unit 344 (e.g., a motion compensation unit) of the prediction processing unit 360 is configured to generate a prediction block 365 used for a video block of the current video slice based on a motion vector and another syntax element received from the entropy decoding unit 304. In inter prediction, the prediction block may be generated from one of the reference pictures in one reference picture list. The video decoder 30 may construct the reference frame lists, List 0 and List 1, using a default construction technique based on the reference pictures stored in the DPB 330.

[0105] Prediction processing unit 360 is configured to determine prediction information to be used for video blocks of the current video slice by parsing the motion vectors and other syntax elements, and to use the prediction information to generate predictive blocks to be used for the current video blocks being decoded. For example, prediction processing unit 360 uses some of the received syntax elements to determine a prediction mode (e.g., intra prediction or inter prediction) to be used to encode the video blocks of the video slice, an inter-prediction slice type (e.g., a B slice, a P slice, or a GPB slice), construction information of one or more of the pictures in the reference picture list used for the slice, a motion vector for each inter-coded video block used for the slice, an inter-prediction state for each inter-coded video block used for the slice, and other information to decode the video blocks of the current video slice.

[0106] The inverse quantization unit 310 may be configured to perform inverse quantization (i.e., dequantization) on the quantized transform coefficients provided in the bitstream and decoded by the entropy decoding unit 304. The inverse quantization process may include using quantization parameters calculated by video encoder 20 for each video block in a video slice to determine a degree of quantization to be applied and determining a degree of inverse quantization to be applied.

[0107] The inverse transform processing unit 312 is configured to apply an inverse transform (eg, an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients in order to generate residual blocks in the sample domain.

[0108] The reconstruction unit 314 (e.g., adder 314) is configured to append the inverse transform block 313 (i.e., the reconstructed residual block 313) to the prediction block 365 to obtain a reconstructed block 315 in the sample domain, for example by adding the sample values ​​of the reconstructed residual block 313 to the sample values ​​of the prediction block 365.

[0109] The loop filter unit 320 (in the coding loop or after the coding loop) is configured to filter the reconstructed block 315 to obtain a filtered block 321 to perform pixel transformation smoothly or improve video quality. In an example, the loop filter unit 320 may be configured to perform any combination of the following filtering techniques: The loop filter unit 320 is intended to represent one or more loop filters, e.g., a deblocking filter, a sample-adaptive offset (SAO) filter, or another filter, such as a bilateral filter, an adaptive loop filter (ALF), a sharpening or smoothing filter, or a collaborative filter. Although the loop filter unit 320 is illustrated as an in-loop filter in FIG. 3, the loop filter unit 320 may be implemented as a post-loop filter in other configurations.

[0110] The decoded video blocks 321 in a given frame or picture are then stored in a decoded picture buffer 330, which stores reference pictures used for subsequent motion compensation.

[0111] The decoder 30 is configured to output the decoded picture 31 using, for example, an output 332 to present the decoded picture 31 to a user or provide the decoded picture 31 for viewing by a user.

[0112] Another variation of the video decoder 30 may be configured to decode the compressed bitstream. For example, the decoder 30 may generate an output video stream without the loop filter unit 320. For example, the non-transform-based decoder 30 may directly dequantize the residual signal without the inverse transform processing unit 312 for some blocks or frames. In another implementation, the video decoder 30 may have the inverse quantization unit 310 and the inverse transform processing unit 312 combined into one unit.

[0113] 4 is a schematic structural diagram of a video coding device 400 (e.g., video encoding device 400 or video decoding device 400) according to an embodiment of the present invention. The video coding device 400 is suitable for implementing embodiments described herein. In one embodiment, the video coding device 400 may be a video decoder (e.g., video decoder 30 of FIG. 1A) or a video encoder (e.g., video encoder 20 of FIG. 1A). In another embodiment, the video coding device 400 may be one or more components of the video decoder 30 of FIG. 1A or the video encoder 20 of FIG. 1A.

[0114] The video coding device 400 includes an ingress port 410 and a receiver (Rx) 420 configured to receive data, a processor, logic unit, or central processing unit (CPU) 430 configured to process the data, a transmitter (Tx) 440 and an egress port 450 configured to transmit the data, and a memory 460 configured to store the data. The video coding device 400 may further include optical / electrical and electrical / optical (EO) conversion components coupled to the ingress port 410, the receiver 420, the transmitter 440, and the egress port 450 to serve as an egress or ingress for optical or electrical signals.

[0115] The processor 430 is implemented using hardware or software. The processor 430 may be implemented as one or more CPU chips, cores (e.g., multi-core processors), FPGAs, ASICs, or DSPs. The processor 430 communicates with the ingress port 410, the receiving unit 420, the transmitting unit 440, the egress port 450, and the memory 460. The processor 430 includes a coding module 470 (e.g., an encoding module 470 and a decoding module 470). The coding / decoding module 470 implements the above-mentioned embodiments. For example, the coding / decoding module 470 performs, processes, or provides various coding operations. Thus, the coding / decoding module 470 substantially extends the functionality of the video coding device 400 and affects the transformation of the video coding device 400 into various states. Alternatively, the coding / decoding module 470 is implemented using instructions stored in the memory 460 and executed by the processor 430.

[0116] The memory 460 may include one or more disks, tape drives, and solid state drives, may be used as an overflow data storage device, may be configured to store programs when these programs are selectively executed, and to store instructions and data read during program execution. The memory 460 may be volatile and / or non-volatile, and may be read-only memory (ROM), random access memory (RAM), ternary content-addressable memory (TCAM), and / or static random access memory (SRAM).

[0117] 5 is a simplified block diagram of an apparatus 500 that can be used as any one or two of the source device 12 and destination device 14 of FIG. 1A according to an exemplary embodiment. The apparatus 500 can implement the technology of the present application. The apparatus 500 can take the form of a computing system including multiple computing devices, or can take the form of a single computing device such as a mobile phone, tablet computer, laptop computer, notebook computer, desktop computer, etc.

[0118] The processor 502 in the device 500 may be a central processing unit. Alternatively, the processor 502 may be any other type of existing or future device or devices capable of controlling or processing information. As shown in the figure, the disclosed embodiments may be implemented using a single processor, such as the processor 502, although benefits in speed and efficiency may be achieved by using at least one processor.

[0119] In one embodiment, the memory 504 in the device 500 may be a Read Only Memory (ROM) device or a Random Access Memory (RAM) device. Any other suitable type of storage device may be used as the memory 504. The memory 504 may include code and data 506 accessed by the processor 502 using a bus 512. The memory 504 may further include an operating system 508 and application programs 510. The application programs 510 include at least one program that enables the processor 502 to perform the methods described herein. For example, the application programs 510 may include application 1 through application N, which further includes a video encoding application for performing the methods described herein. The device 500 may further include additional memory in the form of a secondary memory 514. The secondary memory 514 may be, for example, a memory card used with a mobile computing device. Because a video communication session may contain a large amount of information, the information may be stored, completely or partially, in secondary memory 514 and loaded into memory 504 for processing as needed.

[0120] The apparatus 500 may further include one or more output devices, for example, a display 518. In one example, the display 518 may be a touch-sensitive display that combines the display with a touch-sensitive element that can operate to sense touch input. The display 518 may be coupled to the processor 502 using the bus 512. In addition to the display 518, another output device may be provided that allows a user to program or otherwise use the apparatus 500, or another output device may be provided as an alternative to the display 518. If the output device is or includes a display, the display may be realized in different ways, for example, using a liquid crystal display (LCD), a cathode-ray tube (CRT) display, a plasma display, or a light emitting diode (LED) display, such as an organic LED (OLED) display.

[0121] The apparatus 500 may further include or be connected to a picture sensing device 520. The picture sensing device 520 is, for example, a camera or any other existing or future picture sensing device 520 capable of sensing a picture. The picture is, for example, a picture of a user operating the apparatus 500. The picture sensing device 520 may be positioned directly facing the user operating the apparatus 500. In one example, the position and optical axis of the picture sensing device 520 may be configured such that the field of view of the picture sensing device 520 includes an area adjacent to the display 518 and the display 518 can be seen from that area.

[0122] Apparatus 500 may further include or be connected to a sound sensing device 522, which may be, for example, a microphone or any other existing or future sound sensing device capable of sensing sound in the vicinity of apparatus 500. Sound sensing device 522 may be positioned to directly face a user operating apparatus 500 and may be configured to receive sounds, such as voice or another sound made by a user when the user operates apparatus 500.

[0123] Although the processor 502 and memory 504 of the device 500 are integrated into one unit as shown in FIG. 5, other configurations can be used. The operations of the processor 502 can be distributed across multiple machines, which may be directly coupled (each machine having one or more processors), or distributed in a local area or another network. The memory 504 can be distributed across multiple machines, such as network-based memory and memory within multiple machines that operate the device 500. Although a single bus is depicted here, there may be multiple buses 512 of the device 500. Additionally, the secondary memory 514 may be directly coupled to other components of the device 500 or may be accessed over a network, and may include a single integrated unit, such as a memory card, or multiple units, such as multiple memory cards. Thus, the device 500 may be realized in multiple configurations.

[0124] In the bitstream, motion information occupies a large amount of data. To reduce the amount of data required, motion information is usually transmitted by prediction. In general, there are two types of modes: inter and merge.

[0125] Inter motion vector predictor (MVP) mode: The motion information transmitted includes inter prediction direction (forward, backward, or bidirectional), reference frame index, motion vector predictor index, and motion vector difference. For the motion vector information in the motion information, a method of transmitting the difference between the actual motion vector and the motion vector predictor (MVP) is usually used. The encoder transmits the motion vector difference (MVD) between the MVP and the actual motion vector to the decoder. There can be multiple motion vector predictors. In general, the motion vector predictor candidate list (mvp candidate list) is constructed in the same way in the encoder and the decoder, and the motion vector predictor index (MVP index) is transmitted to the decoder.

[0126] Merge mode: A merge motion information candidate list is constructed in the same way in the encoder and the decoder, and an index is sent to the decoder. A merge index is sent in the bitstream. The motion information in the motion information candidate list is usually obtained from spatially adjacent blocks or temporal blocks in a reference frame. The candidate motion information obtained from the motion information of picture blocks adjacent to the current block is called a spatial candidate, and the motion information of the picture block at the corresponding position of the current block in the reference picture is called a temporal candidate. The spatial candidates and temporal candidates of the current block are shown in Figure 6.

[0127] In intra prediction mode, a directional interpolation method is usually used, the interpolation direction is transmitted in the bitstream, and the predicted pixels of the current coded block are obtained at the decoder using the interpolation direction information and spatially neighboring coded pixels. Different prediction modes are shown in Figure 7.

[0128] Triangular prediction unit mode The triangular prediction unit mode divides a coding block into two triangular coding blocks along a diagonal line based on inter-coding in the original HEVC or VVC VTM. Each triangular coding block uses separate motion information. The diagonal division can be diagonal or inverse diagonal, as shown in FIG. 8. Diagonal division is from the upper left corner to the lower right corner of the current coding block, and inverse diagonal division is from the upper right corner to the lower left corner of the current coding block. Inverse diagonal division can also be called anti-diagonal division. The triangular prediction unit mode is only used in motion compensation prediction and is performed on the entire coding block obtained by combining two triangles in the transform and quantization process. The triangular prediction unit mode can be used for a coding block whose block size is 8×8 or larger and whose coding mode is merge or skip mode.

[0129] In a coded block that can use the triangular prediction unit mode, an identifier merge_triangle_flag is transmitted at the CU level to indicate whether the triangular prediction unit mode is used. If the triangular prediction unit mode is used for the current coded block / CU, a sequence number merge_triangle_idx is transmitted to indicate the direction in which the CU is divided into two triangular prediction units and the motion information of the two triangular prediction units. For example, the direction of the two triangular prediction units and the motion information index of the two triangular prediction units may be encapsulated using a lookup table, and the corresponding information is determined from the lookup table using merge_triangle_idx. Then, each triangular prediction unit obtains uni-prediction information (uni-prediction candidate) from a uni-prediction candidate list (uni-prediction candidate list) based on the motion information index.

[0130] In the weighted sample prediction process, an adaptive weighting process is performed according to the division direction and the weighting set (WS) obtained from the division direction.

[0131] Encoding and decoding in combined intra- and inter-prediction modes - Patents.com In a coded block / CU where a merge mode is used, an identifier mh_intra_flag is sent to indicate whether intra prediction is used. If intra prediction is used, an intra prediction block is generated based on an intra prediction mode selected from an intra candidate mode list based on the intra coding mode related syntax elements mh_intra_luma_mpm_flag and mh_intra_luma_mpm_idx. In a luma block, the intra candidate mode list may include four modes, DC mode, PLANAR mode, horizontal mode, and vertical mode. The size of the intra candidate mode list is selected based on the shape of the current coded block, and the intra candidate mode list may include three or four modes. If the width of the current coded block / CU is twice the height of the current coded block / CU, the intra candidate mode list does not include a horizontal mode. If the height of the current coded block / CU is twice the width of the current coded block / CU, the intra candidate mode list does not include a vertical mode. When a combined intra and inter prediction mode is used, the intra prediction obtained based on the intra prediction mode index and the inter prediction obtained based on the merge index are combined by weighted averaging. For the chroma components, only the DM mode is used.

[0132] In the combined intra and inter prediction mode weighting method, different weighting factors are used for different intra prediction modes. The same weighting factor is used for intra prediction when DC mode and planar mode are used, or the same weighting factor is used when the length or width of the current coded block is less than or equal to 4. A variable weighting factor is used for intra prediction when horizontal mode and vertical mode are used.

[0133] In a multi-hypothesis (MH) coding method for combined intra and inter prediction modes, for a coded block / CU where merge mode is used, an identifier is sent to indicate whether intra prediction is used or whether spatially adjacent pixels around the current coded block are used. If intra prediction is used, an intra prediction mode is used to generate an intra prediction block. For luma blocks, the intra candidate mode is set to PLANAR mode. The final prediction block is generated by performing a combination of intra prediction and inter prediction on the prediction block with adaptive weighting.

[0134] Both the merged triangular prediction unit mode and the multiple hypothesis coding method of the combined intra and inter prediction mode include a motion compensation process. The method and device provided in the embodiment of the present invention can combine two different prediction methods, namely the triangular prediction unit mode and the multiple hypothesis coding method of the combined intra and inter prediction mode, thereby reducing the complexity of coding and decoding, and improving the performance and efficiency of coding and decoding. According to the method provided in the embodiment of the present invention, the processing in the encoder is similar to the processing in the decoder. The following uses the embodiment 1 to the embodiment 5 as an example for detailed description. The embodiment 1 to the embodiment 5 can be performed by the device or system of FIG. 1A to FIG. 5.

[0135] EMBODIMENT 1 A decoding process is performed for at least one picture block in a picture to obtain a reconstructed picture of the picture block. As shown in Figure 9, the aforementioned decoding process includes steps 902 to 914. The block on which the decoding process is being performed is called a current block.

[0136] In this embodiment, the identifier for the combined intra and inter prediction mode is mh_intra_flag, and the identifier for the triangular prediction unit mode is merge_triangle_flag.

[0137] Step 902: Receive a bitstream.

[0138] Step 904: Parse the bitstream to obtain the prediction mode of the current block.

[0139] If the bitstream includes mh_intra_flag, and mh_intra_flag is true, then the prediction mode of the current block is a combined intra and inter prediction mode, and step 906 is performed.

[0140] Step 906: When the prediction mode of the current block is a combined intra and inter prediction mode, select an execution mode of the triangle prediction unit mode according to a preset policy. Specifically, the following implementations are included:

[0141] Implementation 1: When combined intra and inter prediction modes are used, triangular prediction unit modes are not used. intrainter_flag(); If(intrainter_flag == 0)Tri_flag(); if(intrainter_flag())intra_mode() does not exist or its position is not restricted If(Tri_flag()) triangle_idx() indicates that the position is not restricted.

[0142] Embodiment 2: When combined intra and inter prediction modes are used, if the intra prediction mode is DC or PLANAR mode, the triangular prediction unit mode is not used, otherwise the triangular prediction unit mode is used. intrainter_flag(); If(intrainter_flag())intra_mode() if((intrainter_flag == 0) ||((intrainter_flag == 1) && (intra_mode != DC) && (intra_mode != PLANAR))) Tri_flag(); If(Tri_flag())triangle_idx()

[0143] Embodiment 3: When combined intra and inter prediction modes are used, if the intra prediction mode is DC or PLANAR mode, only anti-diagonal partitioning is used in triangular prediction unit mode, otherwise, diagonal partitioning and anti-diagonal partitioning are used in triangular prediction unit mode.

[0144] Step 908: Determine the motion information of the current block according to the execution mode of the triangle prediction unit mode.

[0145] Step 910: Perform a motion compensation process based on the motion information to obtain inter-prediction samples.

[0146] Step 908 and step 910 can be combined to determine a motion compensation process according to the execution mode of the triangular prediction unit mode and obtain an inter-prediction sample. In this case, there are the following method 1 and method 2:

[0147] Method 1: If a triangular prediction unit mode is implemented, a motion compensation process is implemented according to the triangular prediction unit mode. For example, the bitstream may be parsed to obtain a triangular prediction unit mode index (merge_triangle_idx), and a split direction (TriangleDir) of the triangular prediction unit mode is obtained based on the merge_triangle_idx. Specifically, a uni-directional motion information candidate list (uni-prediction candidate list) may be generated, and the motion information of the current block is determined from the uni-directional motion information candidate list based on the triangular prediction unit mode index.

[0148] Method 2: If the triangular prediction unit mode is not implemented, the motion compensation process is implemented according to the merge mode. Specifically, a merge motion information candidate list can be generated, and then the motion information of the current block is determined based on the merge index carried in the bitstream.

[0149] Step 912: Obtain intra prediction samples according to the prediction mode of the current block.

[0150] Specifically, the bitstream may be parsed to obtain intra-prediction mode related syntax, and the intra-prediction mode is selected from an intra candidate mode list based on the syntax. For example, the syntax of the intra coding mode includes a most likely mode identifier mh_intra_luma_mpm_flag and a most likely mode index mh_intra_luma_mpm_idx.

[0151] Step 914: Combine the inter-prediction samples and the intra-prediction samples to obtain a predicted picture of the current block.

[0152] Steps 906 through 910 and step 912 do not follow any particular order.

[0153] For details, please refer to the descriptions of embodiments 3 to 5 below.

[0154] EMBODIMENT 2 A decoding process is performed for at least one picture block in a picture to obtain a reconstructed picture of the picture block. As shown in Figure 10, the aforementioned decoding process includes steps 1002 to 1010. The block on which the decoding process is being performed is called the current block.

[0155] In this embodiment, the identifier for the combined intra and inter prediction mode is mh_intra_flag, and the identifier for the triangular prediction unit mode is merge_triangle_flag.

[0156] Step 1002: Receive a bitstream.

[0157] Step 1004: Parse the bitstream to obtain the triangular prediction unit mode of the current block.

[0158] If the bitstream includes merge_triangle_idx and merge_triangle_idx is true, then triangle prediction unit mode is used for the current block and step 1006 is performed.

[0159] Step 1006: If the triangular prediction unit mode is used for the current block, select the execution mode of the combined intra and inter prediction modes according to a first preset policy. Specifically, the following implementations are included:

[0160] Implementation 1: When the triangular prediction unit mode is used, the combined intra and inter prediction mode is not used. Tri_flag() If(tri_flag == 0)intrainter_flag(); if(intrainter_flag())intra_mode() does not exist or its position is not restricted If(Tri_flag()) triangle_idx() indicates that the position is not restricted.

[0161] Embodiment 2: When a triangle prediction unit mode is used, if a triangle is split along a diagonal, the combined intra and inter prediction mode is not used. Tri_flag() If(Tri_flag())triangle_idx() TriangleDir = getTriangleDir(triangle_idx); If((tri_flag == 0) ||((tri_flag == 1) && (TriangleDir == inverse diagonal))) intrainter_flag(); If(intrainter_flag())intra_mode()

[0162] Implementation 3: When a triangle prediction unit mode is used, if a triangle is split along a diagonal, only a horizontal prediction mode is used when a combined intra and inter prediction mode is used.

[0163] Step 1008: Obtain the motion information of the current block according to the triangular prediction unit mode to obtain an inter-prediction sample.

[0164] For example, the bitstream may be parsed to obtain a triangle prediction unit mode index merge_triangle_idx, and the split direction TriangleDir of the triangle prediction unit mode is obtained from a lookup table based on merge_triangle_idx.

[0165] Then, a uni-prediction candidate list is generated, and the motion information of the current block is determined from the uni-prediction candidate list based on the triangular prediction unit mode index.

[0166] Step 1010: Obtain a predicted picture of the current block according to the running mode and the inter predicted sample of the combined intra and inter prediction mode.

[0167] Specifically, when the combined intra and inter prediction mode is performed according to a first pre-configured policy, intra prediction samples are obtained according to the combined intra and inter prediction mode, and the inter prediction samples and the intra prediction samples are combined to obtain a predicted picture of the current block.

[0168] Alternatively, if the combined intra and inter prediction mode is not performed according to the first preset policy, the predicted picture of the current block is obtained according to the inter prediction sample.

[0169] Steps 1006 and 1008 do not follow any particular order.

[0170] For details, please refer to the descriptions of embodiments 3 to 5 below.

[0171] EMBODIMENT 3 In embodiment 3, a decoding process is performed for at least one picture block in a picture to obtain a reconstructed picture of the picture block. The aforementioned decoding process includes steps 1101 to 1106. The block on which the decoding process is being performed is called a current block.

[0172] Step 1101: Parse the inter-prediction mode of the current block to obtain a triangular prediction unit mode identifier, a combined intra and inter prediction mode identifier, and mode-related syntax elements.

[0173] If the current coding block / CU is in merge mode, the joint analysis method of the triangle prediction unit mode identifier merge_triangle_flag and the combined intra and inter prediction mode identifier mh_intra_flag includes one of the following methods.

[0174] Method 1: If the current coded block / CU satisfies a separate constraint of the triangular prediction unit mode identifier merge_triangle_flag and the transmission of merge_triangle_flag is allowed, then merge_triangle_flag is analyzed. If merge_triangle_flag indicates that the triangular prediction unit mode is not performed and the current coded block / CU satisfies a separate constraint of the combined intra and inter prediction mode identifier mh_intra_flag and the transmission of mh_intra_flag is allowed, then mh_intra_flag is analyzed. If the constraint is not met, then mh_intra_flag is not analyzed and mh_intra_flag is 0. That is, if merge_triangle_flag indicates that the triangular prediction unit mode is performed, then mh_intra_flag is not transmitted and the multiple hypothesis method of the combined intra and inter prediction mode is not performed.

[0175] Method 2: If the current coded block / CU satisfies a separate constraint of the combined intra and inter prediction mode identifier mh_intra_flag and the transmission of mh_intra_flag is allowed, then mh_intra_flag is parsed. If mh_intra_flag indicates that the combined intra and inter prediction mode is not performed and the current coded block / CU satisfies a separate constraint of the triangular prediction unit mode identifier merge_triangle_flag and the transmission of merge_triangle_flag is allowed, then merge_triangle_flag is parsed. Thus, if the prediction mode of the current block is not the combined intra and inter prediction mode, then the triangular prediction unit mode is performed on the current block. If the constraint of the triangular prediction unit mode is met, then the identifier of the triangular prediction unit mode is 1. If the constraint is not met, then merge_triangle_flag is not parsed and merge_triangle_flag is 0. That is, if mh_intra_flag indicates that a combined intra and inter prediction mode is performed, merge_triangle_flag is not sent and the triangular prediction unit mode is not performed. In this case, the constraint of the triangular prediction unit mode can be understood as a preset policy. If the prediction mode of the current block is a combined intra and inter prediction mode, the triangular prediction unit mode is not performed on the current block.

[0176] If merge_triangle_flag indicates that a triangular prediction unit mode is performed, the bitstream is parsed to obtain a triangular prediction unit mode index merge_triangle_idx, and partition information and motion information of the triangular prediction unit mode may be obtained from a lookup table based on merge_triangle_idx. In addition, each element in the lookup table may include partition information of the triangular prediction unit mode and a unidirectional motion information candidate list index. In a specific implementation, if merge_triangle_flag indicates that a triangular prediction unit mode is performed, the bitstream may be parsed to obtain a division direction (TriangleDir) of the triangular prediction unit mode.

[0177] The identifier mh_intra_flag may be used to indicate whether the prediction mode of the current block is a combined intra and inter prediction mode. If mh_intra_flag indicates that a combined intra and inter prediction mode is performed, the bitstream is parsed to obtain intra coding mode related syntax. The intra coding mode related syntax includes a most likely mode identifier mh_intra_luma_mpm_flag and a most likely mode index mh_intra_luma_mpm_idx, in which case the intra coding mode is extracted based on mh_intra_luma_mpm_flag and mh_intra_luma_mpm_idx. Alternatively, a fixed mode such as PLANAR mode may be used directly without sending an index.

[0178] Note that if merge_triangle_idx is parsed after merge_triangle_flag, then the location of merge_triangle_idx in the bitstream is unspecified in this embodiment of the invention.

[0179] In this embodiment of the invention, the position of mh_intra_luma_mpm_flag and mh_intra_luma_mpm_idx in the bitstream is unspecified if mh_intra_luma_mpm_flag and mh_intra_luma_mpm_idx are parsed after mh_intra_flag.

[0180] Please note that in this embodiment of the present invention, only the method for using the triangular prediction unit mode identifier merge_triangle_flag and the combined intra and inter prediction mode identifier mh_intra_flag together is used. The parsing method for the triangular prediction unit mode identifier merge_triangle_flag or the combined intra and inter prediction mode identifier mh_intra_flag is not limited. For example, the constraint may be the following: The combined intra and inter prediction mode identifier mh_intra_flag is parsed only when the width and height of the coded block / CU are equal to or greater than 8 and less than 128, otherwise, mh_intra_flag is not parsed and mh_intra_flag is set to not perform combined intra and inter prediction mode. The triangle prediction unit mode identifier merge_triangle_flag is parsed only if the width + height of the coded block / CU is greater than 12, otherwise merge_triangle_flag is not parsed and merge_triangle_flag is set to not perform triangle prediction unit mode.

[0181] Step 1102: Obtain partition information and motion information of the triangular prediction unit mode according to the triangular prediction unit mode identifier merge_triangle_flag and the triangular prediction unit mode index merge_triangle_idx.

[0182] If the current block is in merge / skip mode and merge_triangle_flag is 0, a merge motion information candidate list is generated. Then, the motion information of the current block is determined based on the merge index carried in the bitstream. The HEVC or VTM method may be used, or another method for generating a motion vector predictor candidate list may be used. This is not limited in this embodiment of the present invention.

[0183] When the current block is in merge / skip mode and merge_triangle_flag is 1, a uni-prediction candidate list is generated, and motion information is determined from the uni-prediction candidate list based on the partition information of the triangle prediction unit mode and the uni-directional motion information candidate list index determined from the lookup table using merge_triangle_idx.

[0184] If the current block is in Inter MVP mode, the motion information of the current block is determined based on the inter prediction direction, the reference frame index, the motion vector predictor index, and the motion vector difference transmitted in the bitstream. The HEVC or VTM method may be used, or another method for generating a motion vector predictor candidate list may be used. This is not limited in this embodiment of the present invention.

[0185] Step 1103: Perform a motion compensation process based on the motion information to obtain inter prediction samples.

[0186] In step 1103, a motion compensation process and a weighted prediction process are performed based on the motion information of the current block to obtain an inter-prediction sample.

[0187] Step 1104: Obtain intra prediction samples according to mh_intra_flag.

[0188] If mh_intra_flag is 1, then one of the following methods may be specifically used:

[0189] Method 1: Obtain the intra coding mode based on mh_intra_luma_mpm_flag and mh_intra_luma_mpm_idx, and call the intra prediction module to generate the intra prediction block.

[0190] Method 2: Obtain an intra coding mode based on mh_intra_luma_mpm_flag and mh_intra_luma_mpm_idx, and call an intra prediction module to generate an intra prediction block, where the intra prediction module does not use the following two intra coding tools: boundary filtering and PDPC.

[0191] Method 3: Set the intra coding mode to planar mode and call the intra prediction module to generate intra prediction blocks.

[0192] Method 4: Set the intra coding mode to planar mode, and call the intra prediction module to generate intra prediction blocks, where the intra prediction module does not use the following two intra coding tools: boundary filtering and PDPC.

[0193] Step 1105: Combine the inter-prediction samples and the intra-prediction samples to obtain a final predicted block / predicted picture for the current block.

[0194] More specifically, the decoder uses an adaptive weighting method, a fixed coefficient weighting method, or an equal ratio weighting method.

[0195] Step 1106: Add the final inter-predicted picture and the residual picture to obtain a reconstructed picture of the current block.

[0196] Specifically, at the decoder, if the current block has a residual, the residual information and the predicted picture are added to obtain a reconstructed picture of the current block, or if the current block does not have a residual, the predicted picture is the reconstructed picture of the current block.

[0197] It should be noted that the order of steps 1102 and 1103 may be interchanged, which is not a limitation of this embodiment of the present invention.

[0198] EMBODIMENT 4 The fourth embodiment relates to a method of inter-encoding and filtering, and a decoding process for a picture block using the method. The main difference from the third embodiment is in step 1201.

[0199] Step 1201: Parse the inter-prediction mode of the current block to obtain a triangular prediction unit mode identifier, a combined intra and inter prediction mode identifier, and mode-related syntax elements.

[0200] In triangular prediction unit mode, a coding block is divided into two triangular coding blocks along a diagonal. The diagonal division can be a diagonal division or an inverse diagonal division, as shown in Figure 3. A diagonal division is a division from the upper left corner to the lower right corner of the current coding block, and an inverse diagonal division is a division from the upper right corner to the lower left corner of the current coding block.

[0201] When the current coded block / CU is in merge mode, the joint analysis method for the triangle prediction unit mode identifier merge_triangle_flag and the combined intra and inter prediction mode identifier mh_intra_flag mainly constrains whether and the execution mode of the combined intra and inter prediction modes are executed based on the partition information of the triangle prediction unit mode.

[0202] Specifically, this includes one of the following methods:

[0203] Method 1: If the current coded block / CU satisfies the separate constraints of the triangular prediction unit mode identifier merge_triangle_flag and the transmission of merge_triangle_flag is allowed, merge_triangle_flag is parsed. If merge_triangle_flag is 1, merge_triangle_idx is parsed and the split direction TriangleDir of the triangular prediction unit mode is obtained from a lookup table based on merge_triangle_idx. If TriangleDir indicates that the triangular prediction unit is split along a diagonal, the identifier mh_intra_flag of the combined intra and inter prediction mode is not sent and the combined intra and inter prediction mode is not used. If TriangleDir indicates that the triangular prediction unit is split along the inverse diagonal, mh_intra_flag is parsed if the current coded block / CU satisfies the separate constraints of the combined intra and inter prediction mode identifier mh_intra_flag and the transmission of mh_intra_flag is allowed. If mh_intra_flag indicates that a combined intra and inter prediction mode is performed, the bitstream is parsed to obtain intra coding mode related syntax. The intra coding mode related syntax may include a most likely mode identifier mh_intra_luma_mpm_flag and a most likely mode index mh_intra_luma_mpm_idx. Alternatively, a fixed mode such as PLANAR mode may be used directly without sending an index. It should be noted that the intra coding mode may be extracted based on mh_intra_luma_mpm_flag and mh_intra_luma_mpm_idx.

[0204] Method 2: If the current coded block / CU satisfies the separate constraints of the triangular prediction unit mode identifier merge_triangle_flag and the transmission of merge_triangle_flag is allowed, merge_triangle_flag is analyzed. If merge_triangle_flag is 1, merge_triangle_idx is analyzed to obtain the division direction TriangleDir of the triangular prediction unit mode. If the current coded block / CU satisfies the separate constraints of the combined intra and inter prediction mode identifier mh_intra_flag and the transmission of mh_intra_flag is allowed, mh_intra_flag is analyzed. If mh_intra_flag is 1, an intra coding mode related index is analyzed, and an intra prediction mode is selected from an intra candidate mode list based on the intra prediction mode related index. When TriangleDir indicates that the triangular prediction unit is divided along a diagonal, the intra candidate mode list includes only horizontal and vertical prediction modes. When TriangleDir indicates that the triangular prediction unit is divided along an inverse diagonal, the structure of the intra candidate mode list is not changed.

[0205] It should be noted that the partition information and motion information of the triangle prediction unit mode may be obtained from the lookup table based on merge_triangle_idx. In addition, each element in the lookup table may include the partition information of the triangle prediction unit mode and a unidirectional motion information candidate list index.

[0206] Note that if merge_triangle_idx is parsed after merge_triangle_flag, then the location of merge_triangle_idx in the bitstream is unspecified in this embodiment of the invention.

[0207] It should be noted that, in the case where the intra-prediction mode related indexes are parsed after mh_intra_flag, the location of the intra-prediction mode related indexes in the bitstream is not specified in this embodiment of the present invention.

[0208] Please note that in this embodiment of the present invention, only the method for using the triangular prediction unit mode identifier merge_triangle_flag and the combined intra and inter prediction mode identifier mh_intra_flag together is used. The parsing method for the triangular prediction unit mode identifier merge_triangle_flag or the combined intra and inter prediction mode identifier mh_intra_flag is not limited. For example, the constraint may be the following: The combined intra and inter prediction mode identifier mh_intra_flag is parsed only when the width and height of the coded block / CU are equal to or greater than 8 and less than 128, otherwise, mh_intra_flag is not parsed and mh_intra_flag is set to not perform combined intra and inter prediction mode. The triangle prediction unit mode identifier merge_triangle_flag is parsed only if the width + height of the coded block / CU is greater than 12, otherwise merge_triangle_flag is not parsed and merge_triangle_flag is set to execute triangle prediction unit mode.

[0209] For other steps, please refer to the description of embodiment 3. Details will not be described repeatedly.

[0210] EMBODIMENT 5 The fifth embodiment relates to a method for inter-encoding and filtering, and a decoding process for a picture block using the method. The main difference from the third embodiment is in steps 1301 and 1302.

[0211] Step 1301: Parse the inter-prediction mode of the current block to obtain a triangular prediction unit mode identifier, a combined intra and inter prediction mode identifier, and mode-related syntax elements.

[0212] In triangular prediction unit mode, a coding block is divided into two triangular coding blocks along a diagonal. The diagonal division can be a diagonal division or an inverse diagonal division, as shown in Figure 3. A diagonal division is a division from the upper left corner to the lower right corner of the current coding block, and an inverse diagonal division is a division from the upper right corner to the lower left corner of the current coding block.

[0213] When the current coded block / CU is in merge mode, the joint analysis method for the triangle prediction unit mode identifier merge_triangle_flag and the combined intra and inter prediction mode identifier mh_intra_flag primarily constrains whether the triangle prediction unit mode is performed and the split information based on the combined intra and inter prediction modes.

[0214] Details are as follows:

[0215] Method 1: If the current coded block / CU satisfies the separate constraints of the combined intra and inter prediction mode identifier mh_intra_flag and the transmission of mh_intra_flag is allowed, mh_intra_flag is parsed. If mh_intra_flag is 1, the intra coding mode related syntax elements (such as mh_intra_luma_mpm_flag and mh_intra_luma_mpm_idx) are parsed, and the intra prediction mode is selected from the intra candidate mode list based on mh_intra_luma_mpm_flag and mh_intra_luma_mpm_idx. The elimination is performed on the lookup table according to a preset policy. The elimination policy is as follows: If the intra prediction mode is DC or PLANAR mode, merge_triangle_flag is not parsed and the triangle prediction unit mode is not performed. If the intra prediction mode is horizontal or vertical mode, and the current coded block / CU satisfies the separate constraint of the identifier of the triangular prediction unit mode, merge_triangle_flag, and the transmission of merge_triangle_flag is allowed, merge_triangle_flag is analyzed. If merge_triangle_flag is 1, merge_triangle_idx is analyzed, and the split direction TriangleDir of the triangular prediction unit mode is obtained based on the lookup table.

[0216] Method 2: If the current coded block / CU satisfies the separate constraints of the combined intra and inter prediction mode identifier mh_intra_flag and the transmission of mh_intra_flag is allowed, mh_intra_flag is parsed. If mh_intra_flag is 1, the intra coding mode related syntax elements (such as mh_intra_luma_mpm_flag and mh_intra_luma_mpm_idx) are parsed, and the intra prediction mode is selected from the intra candidate mode list based on mh_intra_luma_mpm_flag and mh_intra_luma_mpm_idx. If the current coded block / CU satisfies the separate constraints of the triangular prediction unit mode identifier merge_triangle_flag and the transmission of merge_triangle_flag is allowed, merge_triangle_flag is parsed. If merge_triangle_flag is 1, merge_triangle_idx is analyzed, and the split direction TriangleDir of the triangle prediction unit mode is obtained based on the lookup table. Removal may be performed on the lookup table according to a preset policy. The removal policy is as follows: If the intra prediction mode is DC or PLANAR mode, the lookup table does not include elements obtained by splitting along a diagonal. If the intra prediction mode is horizontal or vertical mode, the lookup table may not be changed.

[0217] Step 1302: Obtain an inter-prediction block based on the triangular prediction unit mode identifier merge_triangle_flag and the partition information and motion information of the triangular prediction unit mode obtained based on the triangular prediction unit mode index merge_triangle_idx.

[0218] Step 2.1: Obtain motion information.

[0219] If the current block is in merge / skip mode and merge_triangle_flag is 0, a merge motion information candidate list is generated. Then, the motion information of the current block is determined based on the merge index carried in the bitstream. The HEVC or VTM method may be used, or another method for generating a motion vector predictor candidate list may be used. This is not limited in this embodiment of the present invention.

[0220] When the current block is in merge / skip mode and merge_triangle_flag is 1, a uni-prediction candidate list is generated, and motion information is determined from the uni-prediction candidate list based on the partition information of the triangle prediction unit mode and the uni-directional motion information candidate list index determined from the lookup table using merge_triangle_idx.

[0221] The lookup table is constructed in an existing manner, and the deletion is performed according to a preset policy. The deletion policy is as follows: if the intra prediction mode is DC or PLANAR mode, the lookup table does not include elements obtained by division along a diagonal. If the intra prediction mode is horizontal or vertical mode, the lookup table is constructed in an existing manner.

[0222] For other steps, please refer to the description of embodiment 3. Details will not be described repeatedly.

[0223] The first to fifth embodiments of the present invention can reduce the coding complexity and improve the coding efficiency.

[0224] 11 is a schematic structural diagram of an apparatus 1100 according to an embodiment of the present invention. The apparatus 1100 includes a receiving module 1102, an analyzing module 1104, a determining module 1106, and an acquiring module 1108. In some embodiments, the analyzing module 1104, the determining module 1106, and the acquiring module 1108 may operate in the aforementioned processors, such as the processor 430 of FIG. 4 or the processor 502 of FIG. 5. In some other embodiments, the relevant functions of the analyzing module 1104, the determining module 1106, and the acquiring module 1108 may be implemented in the systems or devices described in the embodiments of FIG. 1A to FIG. 3.

[0225] A first device embodiment is as follows.

[0226] The receiving module 1102 is configured to receive a bitstream.

[0227] The parsing module 1104 is configured to parse the bitstream to obtain a prediction mode of the current block.

[0228] The decision module 1106 is configured to, if the prediction mode of the current block is not a combined intra and inter prediction mode, select an execution mode of the triangular prediction unit mode according to a preset policy, and determine motion information of the current block according to the execution mode of the triangular prediction unit mode.

[0229] The obtaining module 1108 is configured to perform a motion compensation process based on the motion information to obtain a predicted picture of the current block.

[0230] The decision module is configured to: execute a triangular prediction unit mode according to a preset policy if a combined intra- and inter-prediction mode is not used; or, the decision module is configured to analyze an identifier of a triangular prediction unit mode and execute a triangular prediction unit mode according to a preset policy if a combined intra- and inter-prediction mode is not used;

[0231] A second device embodiment is as follows.

[0232] The receiving module 1102 is configured to receive a bitstream.

[0233] The parsing module 1104 is configured to parse the bitstream to obtain the triangular prediction unit mode of the current block.

[0234] The decision module 1106 is configured to, when a triangular prediction unit mode is used for the current block, select an execution mode of the combined intra and inter prediction modes according to a first preset policy.

[0235] The acquisition module 1108 is configured to acquire motion information of the current block according to a triangular prediction unit mode to acquire an inter-prediction sample, and acquire a predicted picture of the current block according to an execution mode and the inter-prediction sample of a combined intra and inter prediction mode.

[0236] In the third device embodiment, the receiving module 1102 and the analyzing module 1104 are optional.

[0237] The determination module 1106 is configured to determine that a triangular prediction unit mode is performed if the prediction mode of the current block is not a combined intra and inter prediction mode, and determine motion information of the current block according to the triangular prediction unit mode.

[0238] The obtaining module 1108 is configured to perform a motion compensation process based on the motion information to obtain a predicted picture of the current block.

[0239] The decision module 1106 is further configured to, when it is determined that the triangular prediction unit mode is to be performed according to a preset policy, obtain a division direction (TriangleDir) of the triangular prediction unit mode, and determine that the triangular prediction unit mode is not to be performed for the current block if the prediction mode of the current block is a combined intra and inter prediction mode.

[0240] The obtaining module 1108 is configured to obtain an identifier mh_intra_flag, where the identifier mh_intra_flag is used to indicate whether the prediction mode of the current block is a combined intra and inter prediction mode.

[0241] The receiving module 1102, the analyzing module 1104, the determining module 1106, and the obtaining module 1108 may be specifically configured to realize the relevant method steps shown in embodiments 1 to 5. For the sake of brevity, the details will not be repeated here.

[0242] In the above embodiments, the description of each embodiment has its own focus. For parts not described in detail in the embodiments, please refer to the relevant descriptions of other embodiments.

[0243] In some embodiments provided in the present application, it should be understood that the disclosed device may be realized in other ways. For example, the described device embodiment is merely an example. For example, the division into units is merely a logical division of functions. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not implemented. In addition, the shown or discussed mutual couplings or direct couplings or communication connections may be realized through some interfaces. Indirect couplings or communication connections between devices or units may be realized in electronic form or other forms.

[0244] The units described as separate parts may or may not be physically separated, and the parts shown as units may or may not be physical units, and may be located in one place or distributed over multiple network units. Some or all of the units may be selected based on actual requirements to achieve the objectives of the solutions of the embodiments.

[0245] In addition, the functional units in the embodiments of the present invention may be integrated into one processing device, or each of the units may exist physically independent, or two or more units may be integrated into one unit. The integrated unit may be implemented in the form of hardware or in the form of a software functional unit.

[0246] The application of the encoding method and the decoding method shown in the above embodiment, and the system using the encoding method and the decoding method will be described below.

[0247] 12 is a block diagram illustrating a content delivery system 3100 configured to realize a content distribution service. The content delivery system 3100 includes an ingestion device 3102, a terminal device 3106, and optionally a display 3126. The ingestion device 3102 communicates with the terminal device 3106 via a communication link 3104. The communication link may include the communication channel 13 described above. The communication link 3104 includes, but is not limited to, Wi-Fi, Ethernet, a wired connection, a wireless (3G / 4G / 5G) connection, USB, or any type of combination thereof.

[0248] The capture device 3102 may generate data and encode the data according to the encoding method shown in the previous embodiment. Alternatively, the capture device 3102 may deliver the data to a streaming server (not shown in the figure), which encodes the data and transmits the encoded data to the terminal device 3106. The capture device 3102 includes, but is not limited to, a camera, a smartphone or tablet computer, a computer or notebook computer, a video conferencing system, a PDA, an in-vehicle device, or any combination thereof. For example, the capture device 3102 may include the source device 12 described above. If the data includes video, a video encoder 20 included in the capture device 3102 may actually encode the video. If the data includes audio (i.e., voice), an audio encoder included in the capture device 3102 may actually encode the audio. In some practical scenarios, the capture device 3102 delivers the encoded video data and the encoded audio data by multiplexing the encoded video data and the encoded audio data. In another practical scenario, for example, in a video conferencing system, the encoded audio data and the encoded video data are not multiplexed. The capture device 3102 delivers the encoded audio data and the encoded video data separately to the terminal device 3106 .

[0249] In the content supply system 3100, the terminal device 310 receives and plays the encoded data. The terminal device 3106 may be a device with data receiving and recovering capabilities, such as a smartphone or tablet computer 3108, a computer or notebook computer 3110, a network video recorder (NVR) / digital video recorder (DVR) 3112, a television 3114, a set top box (STB) 3116, a video conferencing system 3118, a video surveillance system 3120, a personal digital assistant (PDA) 3122, an in-vehicle device 3124, or any combination of the aforementioned devices capable of decoding the encoded data. For example, the terminal device 3106 may include the destination device 14 described above. If the encoded data includes video, the video decoder 30 included in the terminal device is prioritized to perform video decoding. If the encoded data includes audio, the audio decoder included in the terminal device is prioritized to perform audio decoding.

[0250] For terminal devices with a display, such as a smartphone or tablet computer 3108, a computer or notebook computer 3110, a network video recorder (NVR) / digital video recorder (DVR) 3112, a television 3114, a personal digital assistant (PDA) 3122, or an in-vehicle device 3124, the terminal device may provide the decoded data to a display of the terminal device. For terminal devices without a display, such as an STB 3116, a video conferencing system 3118, or a video surveillance system 3120, an external display 3126 is connected to the terminal device to receive and display the decoded data.

[0251] If each device in the system performs encoding or decoding, the picture encoding device or picture decoding device shown in the above embodiments is used.

[0252] 13 is a diagram of an example structure of the terminal device 3106. After the terminal device 3106 receives a stream from the capture device 3102, the protocol processing unit 3202 analyzes the transport protocol of the stream. The protocol includes, but is not limited to, Real Time Streaming Protocol (RTSP), Hyper Text Transfer Protocol (HTTP), HTTP Live streaming protocol (HLS), MPEG-DASH, Real-time Transport protocol (RTP), Real Time Messaging Protocol (RTMP), or any combination thereof.

[0253] After processing the stream, the protocol processing unit 3202 generates a stream file. The file is output to the demultiplexing unit 3204. The demultiplexing unit 3204 may split the multiplexed data into encoded audio data and encoded video data. As mentioned above, in another practical scenario, for example in a video conferencing system, the encoded audio data and the encoded video data are not multiplexed. In this case, the encoded data is sent to the video decoder 3206 and the audio decoder 3208 without being sent through the demultiplexing unit 3204.

[0254] Demultiplexing is performed to generate a video elementary stream (ES), an audio ES, and optional subtitles. The video decoder 3206 includes the video decoder 30 described in the previous embodiment, and decodes the video ES according to the decoding method shown in the previous embodiment to generate video frames, and supplies such data to the synchronization unit 3212. The audio decoder 3208 decodes the audio ES to generate audio frames, and supplies such data to the synchronization unit 3212. Alternatively, the video frames may be stored in a buffer (not shown in FIG. Y) before being supplied to the synchronization unit 3212. Similarly, the audio frames may also be stored in a buffer (not shown in FIG. Y) before being supplied to the synchronization unit 3212.

[0255] The synchronization unit 3212 synchronizes the video and audio frames and provides the video / audio to the video / audio display 3214. For example, the synchronization unit 3212 synchronizes the presentation of video and audio information. The information may be syntactically encoded using timestamps associated with the presentation of the encoded audio and video data and timestamps associated with the transmission of the data stream.

[0256] If the stream contains subtitles, a subtitle decoder 3210 decodes the subtitles to synchronize them with the video and audio frames and provides the video / audio / subtitles to a video / audio / subtitle display 3216 .

[0257] The present invention is not limited to the above-mentioned system, and the picture encoding device or the picture decoding device in the above-mentioned embodiments may be integrated into another system, for example an automobile system.

[0258] It will be understood that a combination of the disclosed contents with the described method may also apply to a corresponding device or system configured to perform the method, and vice versa. For example, when one or more specific method steps are described, the corresponding device may include one or more units, such as functional units, to perform the described one or more method steps (e.g., one unit performing one or more steps, or multiple units each performing one or more of the multiple steps), even if such one or more units are not explicitly described or shown in the accompanying drawings. In addition, for example, when a particular apparatus is described based on one or more units, such as functional units, the corresponding method may include one step used to realize the function of one or more units (e.g., one step used to realize the function of one or more units, or multiple steps each used to realize one or more functions of multiple units), even if such one or more steps are not explicitly described or shown in the accompanying drawings. Furthermore, it should be understood that the features of the various exemplary embodiments and / or aspects described herein may be combined with each other, unless otherwise specified.

[0259] In one or more examples, the described functions may be implemented by hardware, software, firmware, or any combination thereof. If implemented by software, the functions may be stored on or transmitted over a computer-readable medium as one or more instructions or codes and executed by a hardware-based processing device. A computer-readable medium may include a computer-readable storage medium, which corresponds to a tangible medium such as a data storage medium or a communication medium. For example, a communication medium includes any medium that facilitates the transmission of a computer program from one place to another according to a communication protocol. In this manner, a computer-readable medium may generally correspond to (1) a non-transitory tangible computer-readable storage medium, or (2) a communication medium such as a signal or carrier wave. A data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to obtain instructions, codes, and / or data structures for implementing the techniques described in the present invention. A computer program product may include a computer-readable medium.

[0260] By way of example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk memory, magnetic disk memory or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. In addition, any connection may also be properly referred to as a computer-readable medium. For example, if the instructions are transmitted from a website, server, or another remote source using coaxial cable, fiber optic, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio waves, and microwaves, the coaxial cable, fiber optic, twisted pair, DSL, or wireless technologies such as infrared, radio waves, and microwaves are included in the definition of media. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transitory media, and in fact refer to non-transitory tangible storage media. As used herein, disk and disc include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray disc. Disks usually reproduce data magnetically, while discs reproduce data optically using a laser. Combinations of the foregoing items should also be included within the scope of computer-readable media.

[0261] The instructions may be executed by one or more processors. The one or more processors may be, for example, one or more digital signal processors (DSPs), general-purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Thus, the term "processor" as used in this application may be any of the aforementioned structures or any other structure suitable for implementing the techniques described herein. In addition, in some aspects, the functions described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or integrated into a combinatorial codec. In addition, all of the techniques may be implemented in one or more circuit or logic elements.

[0262] The techniques of the present disclosure may be implemented in a variety of devices or apparatuses, including wireless handsets, integrated circuits (ICs) or sets of ICs (e.g., chipsets). Various components, modules, or units are described in this disclosure to highlight functional aspects of devices configured to perform the techniques of the disclosure, but are not necessarily implemented by different hardware units. Rather, as described above, the various units may be integrated into a codec hardware unit in combination with appropriate software and / or firmware, or may be provided by a set of interoperable hardware units. Those hardware units include one or more processors as described above. [Explanation of symbols]

[0263] 10 Coding Systems 12 Source Device 13 Encoded Data 14 Destination Device 16 Picture source, video source 17 Picture Data 18 Pretreatment Equipment 19 Pre-processing picture data 20 Video Encoder 21 Encoded Picture Data 22 Communication interfaces, communication devices 28 Communication interfaces, communication devices 30 Video Decoder 31 Decoded Picture Data 32 Post-processing device 33 Post-processing picture data 34 Display Devices 40 Video Coding System 41 Imaging Device 42 Antenna 43 Processors 44 Memory 45 Display Devices 46 Processing Equipment 47 Logic Circuits 201 Pictures 202 Input 203 Picture Block 204 Residual Calculation Unit 205 Residual Blocks 206 Conversion Processing Unit 207 Conversion Factors 208 Quantization Units 209 Quantized residual coefficients, quantized transform coefficients 210 Inverse Quantization Unit 211 Dequantized Coefficients 212 Inverse Transformation Processing Unit 213 Reconstructed residual block, inverse transform inverse quantization block, inverse transform residual block 214 Reconstruction Unit, Adder 215 reconstructed blocks 216 Buffers 220 Loop Filter Unit 221 Filtered Blocks 230 Decoded Picture Buffer DPB 231 Decoded Pictures 244 Inter Prediction Units 245 Inter Prediction Blocks 246 Motion Compensation Unit 254 intra prediction units 255 Intra-predicted blocks 260 Prediction Processing Unit 262 Mode Selection Unit 265 predicted blocks 270 Entropy Coding Unit 272 Output 304 Entropy Decoding Unit 309 Quantized Coefficients 310 Inverse Quantization Unit 312 Inverse Transformation Processing Unit 313 Reconstructed residual block, inverse transform block 314 Reconstruction Unit, Adder 315 Reconstructed Blocks 316 Buffer 320 Loop Filter Unit 321 Filtered Blocks 330 Decoded Picture Buffer DPB 332 Output 344 Inter Prediction 354 Intra Prediction 360 Prediction Processing Unit 362 Mode Selection Unit 365 predicted blocks 400 Video Encoding Device 410 Inlet Port 420 Receiving Equipment 430 Processor 440 Transmitting Device 450 Exit Port 460 Memory 470 Encoding / Decoding Module 500 devices 502 processor 504 Memory 506 Code and Data 508 Operating Systems 510 Application Program 512 Bus 514 Secondary Memory 518 Display 520 Picture Sensing Device 522 Sound sensing device 1100 equipment 1102 Receiver Module 1104 Analysis Module 1106 Decision Module 1108 Acquisition Module 3100 Contents Supply System 3102 Capture Device 3104 Communication Links 3106 Terminal Device 3108 Smartphone / Tablet Computer 3110 Computers / Notebook Computers 3112 Digital Video Recorder 3114 Television 3116 Set top box (STB) 3118 Video Conference System 3120 Video Surveillance System 3122 Personal Digital Assistant 3124 In-vehicle devices 3126 Display 3202 Protocol Processing Unit 3204 Demultiplexing Unit 3206 Video Decoder 3208 Audio Decoder 3210 Subtitle Decoder 3212 Synchronous Unit 3214 Video / Audio Display 3216 Video / Audio / Subtitle Display

Claims

1. 1. A picture prediction method, comprising: receiving a bitstream; parsing the bitstream to obtain a prediction mode of a current block; selecting an execution mode of a triangular prediction unit mode according to a preset policy when the prediction mode of the current block is a combined intra and inter prediction mode; determining motion information of the current block according to the execution mode of the triangular prediction unit mode; performing a motion compensation process based on the motion information to obtain inter-prediction samples; obtaining intra prediction samples according to the prediction mode of the current block; combining the inter-predicted samples and the intra-predicted samples to obtain a predicted picture of the current block; The method includes:

2. The method of claim 1 , wherein if an intra prediction mode in the combined intra and inter prediction modes is a DC or PLANAR mode, the triangular prediction unit mode is not performed according to the preset policy.

3. The method of claim 1 , wherein if an intra prediction mode in the combined intra and inter prediction modes is DC or PLANAR mode, only inverse diagonal partitioning is used in the triangular prediction unit mode according to the preset policy.

4. The method of claim 1 , wherein when the combined intra and inter prediction modes are used, the triangular prediction unit mode is not performed according to the preset policy.

5. The method of claim 1, wherein if an intra prediction mode in the combined intra and inter prediction modes is DC or PLANAR mode, a triangular prediction unit mode identifier is not analyzed and the triangular prediction unit mode is not executed according to the preset policy.

6. The method of claim 1 , wherein when the combined intra and inter prediction modes are used, a triangular prediction unit mode identifier is not analyzed and the triangular prediction unit mode is not executed according to the preset policy.

7. The method according to claim 5 or 6, wherein the triangular prediction unit mode identifier is merge_triangle_flag.

8. The method of claim 3 , wherein the anti-diagonal division comprises dividing from the upper right corner to the lower left corner.

9. determining motion information of the current block according to the execution mode of the triangular prediction unit mode, and performing a motion compensation process based on the motion information; if the triangular prediction unit mode is implemented, performing the motion compensation process according to the triangular prediction unit mode; or if the triangular prediction unit mode is not implemented, performing the motion compensation process according to a merge mode.

9. The method of any one of claims 1 to 8, comprising:

10. The method of claim 1 , wherein a combined intra and inter prediction mode identifier indicates the combined intra and inter prediction mode.

11. The method of claim 10, wherein the identifier of the combined intra and inter prediction mode is mh_intra_flag.

12. A method according to any one of claims 1 to 11, wherein when it is determined that the triangular prediction unit mode is to be executed according to the preset policy, the bitstream is parsed to obtain a triangular prediction unit mode index (merge_triangle_idx), and a split direction (TriangleDir) of the triangular prediction unit mode is obtained based on merge_triangle_idx.

13. The method of claim 12, wherein a uni-prediction candidate list is generated, and the motion information of the current block is determined from the uni-prediction candidate list based on the triangular prediction unit mode index.

14. When the triangle prediction unit mode is not performed according to the preset policy, the method further comprises:

12. The method according to claim 1, comprising generating a merge motion information candidate list and determining the motion information of the current block based on a merge index carried in the bitstream.

15. The step of obtaining intra-predicted samples according to the prediction mode of the current block comprises:

15. The method of claim 1, further comprising: parsing the bitstream to obtain an intra-coding mode related syntax; and selecting an intra-prediction mode from an intra candidate mode list based on the syntax.

16. The method of claim 15, wherein the intra-coding mode related syntax includes a most probable mode identifier mh_intra_luma_mpm_flag and a most probable mode index mh_intra_luma_mpm_idx.

17. 1. A picture prediction method, comprising: receiving a bitstream; parsing the bitstream to obtain a triangular prediction unit mode of a current block; selecting an execution mode of the combined intra and inter prediction modes according to a first preset policy when the triangular prediction unit mode is used for the current block; obtaining motion information of the current block according to the triangular prediction unit mode to obtain inter-prediction samples; obtaining a predicted picture of the current block according to the running mode and the inter-predicted sample of the combined intra and inter prediction mode; The method includes:

18. The step of obtaining a predicted picture of the current block according to the running mode of the combined intra and inter prediction modes and the inter prediction sample includes: if the combined intra and inter prediction mode is performed according to the first preset policy, obtaining intra prediction samples according to the combined intra and inter prediction mode, and combining the inter prediction samples and the intra prediction samples to obtain the predicted picture of the current block; or if the combined intra and inter prediction modes are not performed according to the first preset policy, obtaining the predicted picture of the current block according to the inter prediction samples.

20. The method of claim 17, comprising:

19. 20. The method of claim 17, wherein when the triangular prediction unit mode is used, an identifier of a combined intra and inter prediction mode is not analyzed and the combined intra and inter prediction mode is not performed according to the first preset policy.

20. 18. The method of claim 17, wherein when the triangular prediction unit mode is used, if the triangular prediction unit is split along an inverse diagonal, an identifier of the combined intra and inter prediction mode is analyzed according to a second pre-set rule, or if the triangular prediction unit is split along a diagonal, an identifier of the combined intra and inter prediction mode is not analyzed and the combined intra and inter prediction mode is not executed.

21. The method of claim 19 or 20, wherein the identifier of the combined intra and inter prediction mode is mh_intra_flag.

22. 20. The method of claim 17, wherein when the triangular prediction unit mode is used, the combined intra and inter prediction modes are not performed according to the first preset policy.

23. 18. The method of claim 17, wherein, when the triangular prediction unit mode is used, if the triangular prediction unit is split along an inverse diagonal, the combined intra and inter prediction mode is performed, or if the triangular prediction unit is split along a diagonal, the combined intra and inter prediction mode is not performed.

24. 20. The method of claim 17, wherein when the triangular prediction unit mode is used, if the triangular prediction unit is split along a diagonal, only horizontal and vertical prediction modes are used when the combined intra and inter prediction mode is used.

25. 25. The method of claim 17, wherein a triangular prediction unit mode identifier indicates an execution mode of the triangular prediction unit mode.

26. The method of claim 25, wherein the triangular prediction unit mode identifier is merge_triangle_flag.

27. 27. The method according to claim 17, wherein when the combined intra and inter prediction modes are performed, the bitstream is parsed to obtain an intra coding mode related syntax, and an intra prediction mode is selected from an intra candidate mode list based on the syntax.

28. The method of claim 27, wherein the intra-coding mode related syntax includes a most probable mode identifier mh_intra_luma_mpm_flag and a most probable mode index mh_intra_luma_mpm_idx.

29. A method according to any one of claims 17 to 28, wherein the bitstream is parsed to obtain a triangle prediction unit mode index merge_triangle_idx, and a split direction TriangleDir of the triangle prediction unit mode is obtained from a lookup table based on merge_triangle_idx.

30. 30. The method of claim 29, wherein a uni-prediction candidate list is generated, and the motion information of the current block is determined from the uni-prediction candidate list based on the triangular prediction unit mode index.

31. 1. A picture prediction method, comprising: determining that a triangular prediction unit mode is to be performed when the prediction mode of the current block is not a combined intra and inter prediction mode; determining motion information of the current block according to the triangular prediction unit mode; performing a motion compensation process based on the motion information to obtain a predicted picture of the current block; The method includes:

32. The method of claim 31 , wherein the identifier of the triangular prediction unit mode is merge_triangle_flag.

33. The method of claim 31 or 32, wherein the method further comprises the step of obtaining a division direction (TriangleDir) of the triangle prediction unit mode.

34. The method further comprising: A method according to any one of claims 31 to 33, further comprising a step of obtaining an identifier mh_intra_flag, the identifier mh_intra_flag being used to indicate whether the prediction mode of the current block is the combined intra and inter prediction mode.

35. 35. The method of claim 31, wherein when the prediction mode of the current block is the combined intra and inter prediction mode, it is determined that the triangular prediction unit mode is not performed for the current block.

36. An apparatus for decoding a video stream, comprising a processor and a memory, the memory storing instructions that enable the processor to perform a method according to any one of claims 1 to 16 or claims 31 to 35.

37. 36. An apparatus for decoding a video stream, comprising: a processor and a memory, the memory storing instructions that enable the processor to execute a method according to any one of claims 17 to 35.

38. A decoding device comprising a non-volatile memory and a processor coupled to each other, the memory configured to store program instructions, the program instructions enabling the processor to execute a method according to any one of claims 1 to 16 or claims 31 to 35.

39. 39. A decoding device comprising: a non-volatile memory and a processor coupled to each other, the memory configured to store program instructions, the program instructions enabling the processor to execute a method according to any one of claims 17 to 38.

40. A picture prediction device, comprising: A determination module configured to determine that a triangular prediction unit mode is performed when a prediction mode of a current block is not a combined intra and inter prediction mode, and to determine motion information of the current block according to the triangular prediction unit mode; an acquisition module configured to perform a motion compensation process based on the motion information to acquire a predicted picture of the current block; An apparatus comprising:

41. The apparatus of claim 40 , wherein the obtaining module is further configured to obtain a division direction (TriangleDir) of the triangle prediction unit mode.

42. The apparatus of claim 40 or 41, wherein the acquisition module is configured to acquire an identifier mh_intra_flag, the identifier mh_intra_flag being used to indicate whether the prediction mode of the current block is the combined intra and inter prediction mode.

43. 43. The apparatus of claim 40, wherein the determination module is further configured to determine that the triangular prediction unit mode is not performed for the current block when the prediction mode of the current block is the combined intra and inter prediction mode.

Images