Encoder, decoder and corresponding method for sub-block partitioning mode

Abstract

To provide a method and apparatus for encoding and decoding, with improved coding efficiency.SOLUTION: A method of coding implemented by a decoding device, comprises the steps of: obtaining a bitstream; obtaining a value of an indicator for a current block according to the bitstream; obtaining a value of a first parameter for the current block and a value of a second parameter for the current block, according to the value of the indicator and a predefined lookup table; obtaining a value of a sample distance for a sample which is located in the current block, according to the value of the first parameter and the value of the second parameter; and obtaining a prediction value for the sample, according to the value of the sample distance for the sample.SELECTED DRAWING: Figure 15

Description

[Technical Field]

[0001] Embodiments of this application (disclosure) generally relate to the field of picture processing, and more specifically to prediction using subblock segmentation modes. [Background technology]

[0002] Video coding (video encoding and decoding) is used in a wide range of digital video applications, such as digital television broadcasting, video transmission over the internet and mobile networks, real-time conversation applications like video chat, video conferencing, DVD and Blu-ray® discs, video content acquisition and editing systems, and camcorders for security applications.

[0003] Even relatively short videos can require a considerable amount of video data to render, which can be problematic when the data needs to be streamed or otherwise communicated over a communication network with limited bandwidth. Therefore, video data is generally compressed before being transmitted over modern telecommunication networks. Video size can also be an issue when video is stored on a storage device, as memory resources may be limited. Video compression devices often use software and / or hardware at the source to encode the video data before transmission or storage, thereby reducing the amount of data required to represent the digital video image. The compressed data is then received at the destination by a video decompression device that decodes the video data. Due to limited network resources and the ever-increasing demand for higher video quality, improved compression and decompression techniques that improve the compression ratio with little to no sacrifice of picture quality are desirable. [Overview of the project] [Means for solving the problem]

[0004] Embodiments of this application provide apparatus and methods for encoding and decoding according to independent claims.

[0005] The aforementioned and other objectives are achieved by the subject matter of the independent claims. Further implementations are evident from the dependent claims, description, and drawings.

[0006] A first aspect of the present invention provides a coding method performed by a decoding device, the method comprising the steps of: acquiring a bitstream; acquiring an indicator value for the current block according to the bitstream; acquiring a first parameter value and a second parameter value for the current block according to the indicator value and a predetermined lookup table; acquiring a sample distance value for a sample located in the current block according to the first parameter value and the second parameter value; and acquiring a predicted value for a sample according to the sample distance value for the sample.

[0007] According to embodiments of the present invention, parameters for dividing blocks (e.g., angle parameters, distance parameters, etc.) are stored in a predetermined lookup table, so that the actual values ​​of these parameters do not need to be transmitted in the bitstream, and the values ​​of these parameters are obtained according to the indicated values ​​coded in the bitstream. Thus, coding efficiency can be improved.

[0008] As shown in Figure 15, a coding method performed by a decoding device is disclosed, which includes the step of S1501: obtaining a bitstream.

[0009] The bitstream may be obtained via a wireless or wired network. The bitstream may be transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, microwave, Wi-Fi, Bluetooth®, LTE, or 5G.

[0010] In one embodiment, the bitstream is a sequence of bits, for example, in the form of a network abstraction layer (NAL) unit stream or byte stream, representing a sequence of access units (AUs) that form one or more coded video sequences (CVS).

[0011] In some embodiments, the decoding process involves the decoder reading the bitstream and deriving the decoded picture from the bitstream, while the encoding process involves the encoder generating the bitstream.

[0012] Typically, a bitstream contains syntax elements formed by a syntax structure. Syntax elements are the elements of data represented in a bitstream.

[0013] Syntax structure: Zero or more syntax elements that exist together in a bitstream in a specified order.

[0014] In a concrete example, a bitstream format specifies the relationship between a network abstraction layer (NAL) unit stream and a byte stream, both of which are referred to as bitstreams.

[0015] The bitstream can be of one of two formats, for example, the NAL unit stream format or the byte stream format. The NAL unit stream format is conceptually the more "basic" type. The NAL unit stream format contains a sequence of syntax structures called NAL units. This sequence is arranged in decoding order. There are constraints imposed on the decoding order (and content) of the NAL units within the NAL unit stream.

[0016] The byte stream format can be constructed from the NAL unit stream format by arranging the NAL units in decoding order and adding a start code prefix and zero or more bytes of value 0 at the beginning of each NAL unit to form a byte stream. The NAL unit stream format can be extracted from the byte stream format by searching for the position of the unique start code prefix pattern within this byte stream.

[0017] This clause specifies one form of the relationship between the source and the decoded picture given via the bitstream.

[0018] The video source represented by the bitstream is a sequence of pictures in decoding order.

[0019] [[ID=I5]] The source and the decoded picture each consist of one or more sample arrays. - Only luma (Y) (monochrome) - Luma and two chromas (YCbCr or YCgCo) - Green, blue, and red (also known as GBR, RGB) - Arrays representing other unspecified monochrome or trichromatic sampling (e.g., also known as YZX, XYZ)

[0020] The variables and terms associated with these arrays are called luma (or L or Y) and chroma, and the two chroma arrays are called Cb and Cr, regardless of the actual color representation method being used. The actual color representation method can be indicated in the syntax specified in the VUI parameters, as specified in ITU-T H.SEI|ISO / IEC 23002-7.

[0021] Depending on the chroma format sampling structure specified through sps_chroma_format_idc and sps_separate_colour_plane_flag, the variables SubWidthC and SubHeightC are specified in Table 2 (Table 1).

[0022]

Table 1

[0023] In monochrome sampling, there is only one sample array, which is usually considered to be the luma array.

[0024] In 4:2:0 sampling, each of the two chroma arrays has half the height and half the width of the luma array.

[0025] In 4:2:2 sampling, each of the two chroma arrays has the same height as the luma array and half the width of the luma array.

[0026] In 4:4:4 sampling, depending on the value of sps_separate_colour_plane_flag, the following applies. - When sps_separate_colour_plane_flag is equal to 0, each of the two chroma arrays has the same height and width as the luma array. - Otherwise (sps_separate_colour_plane_flag is equal to 1), the three color planes are processed separately as monochrome sampled pictures.

[0027] S1502: Get the indicator value for the current block according to the bitstream.

[0028] In one embodiment, indicator values ​​are used to specify the partition shape for the geometric partition merge mode. For example, the indicator may be merge_gpm_partition_idx[x0][y0], where merge_gpm_partition_idx[x0][y0] specifies the partition shape for the geometric partition merge mode. The array indices x0, y0 specify the relative position (x0, y0) of the top-left luma sample of the coding block being considered with respect to the top-left luma sample of the picture.

[0029] Normally, the value of merge_gpm_partition_idx[x0][y0] is decoded from the bitstream. In one example, the range of values ​​for merge_gpm_partition_idx[][] is from 0 to 63, including 0 and 63. In another example, the decoding process for merge_gpm_partition_idx[][] is "bypassed".

[0030] When merge_gpm_partition_idx[x0][y0] does not exist, it is presumed to be equal to 0.

[0031] S1503: Retrieve the value of the first parameter for the current block and the value of the second parameter for the current block according to the indicator value and a predetermined lookup table.

[0032] In one implementation, the first parameter represents the angle (or corner) for the current block division.

[0033] In one implementation, the second parameter represents the distance for the current block division.

[0034] In one implementation, a predetermined lookup table includes pairs of first and second parameters. If the first parameter of a pair represents an angle corresponding to one of the angles 0, 45, 90, 135, 180, 225, 270, or 315 degrees, then the corresponding second parameter of the pair does not represent a distance of 0 samples from the center of the current block.

[0035] In one implementation, a predetermined lookup table includes pairs of first and second parameters. The first parameter of the pair represents an angle that does not correspond to one of the angles 0, 45, 90, 135, 180, 225, 270, or 315 degrees, and the corresponding second parameter of the pair represents a distance of 0 samples from the center of the current block.

[0036] In one implementation, a predetermined lookup table includes pairs of first and second parameters. The first parameter in a pair represents an angle corresponding to one of the following angles: 0, 45, 90, 135, 180, 225, 270, or 315 degrees, and the corresponding second parameter in the pair represents a distance of 0 samples from the center of the current block.

[0037] In one implementation, the second parameter specifies the distance index of the geometric division, or the second parameter represents the distance from the separator line to the center of the current block.

[0038] In one embodiment, the partition angle variable angleIdx (parameter 1) and distance variable distanceIdx (parameter 2) of the geometric partition mode are set according to the values ​​of merge_gpm_partition_idx[xCb][yCb] (indicator) as specified in the following table. In this implementation, it can be understood that this relationship may be implemented according to table 1 (table 2) or according to a function.

[0039] [Table 2]

[0040] S1504: Obtain the sample distance value for the sample located in the current block according to the values ​​of the first and second parameters.

[0041] In one embodiment, this step includes the following:

[0042] Step 3.1: Obtain the index value of the angle parameter (alphaN or angleIdx) for the current block, the width value (W) of the current block, and the height value (H) of the current block. W and H are the width and height of the current block in units of the number of samples. For example, a coding block with both width and height equal to 8 is a square block containing 64 samples. In another example, W and H are the width and height of the current block in units of the number of samples. The index value of the angle parameter can be obtained as described above, referring to Table 1 (Table 2).

[0043] Step 3.2: Obtain the value of the ratio whRatio according to the values ​​of W and H, where the value of whRatio represents the ratio of the width to the height of the current coding block.

[0044] In some cases, whRatio=H / W or whRatio=W / H.

[0045] In another example, the two variables nCbW and nCbH specify the width and height of the current coding block, and the variable cIdx specifies the color component index.

[0046] The variables nW, nH, and whRatio are derived as follows: nW=(cIdx==0)?nCbW:nCbW*SubWidthC nH=(cIdx==0)?nCbH:nCbH*SubHeightC whRatio=nH / nW

[0047] In one example, the variables SubWidthC and SubHeightC are specified in Table 2 (Table 1).

[0048] Step 3.3: Obtain the shiftHor value according to the lookup table, where the values ​​of alpha and whRatio, in some examples, alpha(alphaN or angleIdx) and whRatio, are used as index values ​​in the lookup table. The shiftHor value may also be obtained according to a function, where the values ​​of alpha(alphaN or angleIdx) and whRatio are inputs to the function, and the shiftHor value is the output of the function. In some examples, the result of the function is similar to or identical to the lookup table. In some examples, the shiftHor value represents the quantization step size for the sample distance calculation process.

[0049] In another example, the function can be represented by shiftHor=(angleIdx%16==8||(angleIdx%16!=0&&whRatio>0))?0:1.

[0050] Step 3.4: The value of sample_dist is calculated according to the shiftHor value.

[0051] In one example, to calculate the sample distance (sample_dist), the variables offsetX and offsetY are first derived as follows:

[0052] If shiftHor is equal to 0, the following is true: offsetX=(-nW)>>1 offsetY=((-nH)>>1)+ (angleIdx<16?(distanceIdx*nH)>>3:-((distanceIdx*nH)>>3))

[0053] Otherwise (when shiftHor is equal to 1), the following applies: offsetX=((-nW)>>1)+ (angleIdx<16?(distanceIdx*nW)>>3:-((distanceIdx*nW)>>3)) offsetY=(-nH)>>1

[0054] The variables xL and yL are derived as follows: xL=(cIdx==0)?x:x*SubWidthC yL=(cIdx==0)?y:y*SubHeightC sample_dist=(((xL+offsetX)<<1)+1)*disLut[displacementX]+ (((yL+offsetY)<<1)+1))*disLut[displacementY]

[0055] The variables displacementX and displacementY are derived as follows: hwRatio=cbHeight / cbWidth Purchase X = angleIdx displacementY=(angleIdx+8)%32

[0056] The array disLut is specified in Table 3 (Table 6) below.

[0057] In some embodiments, according to a certain geometric model, a sample in a coding block is considered to be located within two subblocks. Subblock A or subblock B may contain a portion (but not all) of the sample in the current coding block. Subblock A or subblock B may be represented according to the sign of sample_dist for each sample. sample_dist may be obtained according to the examples and embodiments of other paragraphs.

[0058] S1505: Obtain the predicted value for the sample according to the sample distance value for the sample.

[0059] In one implementation, the step of obtaining a predicted value for a sample according to the sample distance value for the sample is: The steps involve calculating two weight coefficients according to the sample distance values, The process includes the step of obtaining a predicted value for a sample according to a first predicted value, a second predicted value, and two weight coefficients.

[0060] In one implementation, the sample distance value represents the horizontal distance, vertical distance, or combination of horizontal and vertical distance from the sample to the separator line, and the separator line is used to divide a coding block into two subblocks.

[0061] In one example, the calculated sample_dist is used to calculate weight coefficients, which are used for the combination of first and second predicted values ​​corresponding to the sample. In one example, the weight coefficients are denoted as sampleWeight1 and sampleWeight2, referring to the weights corresponding to the first predicted value and the weights corresponding to the second predicted value.

[0062] In one example, the weight coefficients are calculated according to the following function: weightIdxL=partFlip?32+sample_dist:32-sample_dist wValue=Clip3(0,8,(weightIdxL+4)>>3)

[0063] In this example, wValue is sampleWeight1 and 8-wValue is sampleWeight2. The variable partFlip is determined according to the value of angleIdx. In one example, partFlip = (angleIdx>=13&&angleIdx<=27)?0:1 or partFlip = (angleIdx>=13&&angleIdx<=27)?1:0.

[0064] In one example, the combined value of the predicted sample at sample coordinates (x,y) is calculated according to the first predicted value at coordinates (x,y), the second predicted value at coordinates (x,y), sampleWeight1, and sampleWeight2.

[0065] In one example, the predicted sample values ​​are derived as follows: pbSamples[x][y]=Clip3(0,(1< <BitDepth)-1,(predSamplesLA[x][y]*wValue+ predSamplesLB[x][y]*(8-wValue)+offset1)>>shift1)

[0066] Here, bitDepth represents the sample bit depth, the variable shift1 is obtained according to bitDepth, and in one example, shift1 = Max(5, 17 - BitDepth). The variable offset1 is obtained according to shift1, and in one example, offset1 = 1 << (shift1 - 1), and predSamplesLA and predSamplesLB are two (nCbW) x (nCbH) arrays.

[0067] In one implementation, a coding method performed by an encoding device is disclosed, and this method is: The steps include selecting the value of the first parameter and the value of the second parameter, The steps include obtaining the value of the first parameter, the value of the second parameter, and the index value according to the lookup table, The process includes the step of encoding the index value into a bitstream.

[0068] The details of each step on the encoder side correspond to the example above on the decoder side.

[0069] As shown in Figure 16, a second aspect of the present invention provides a decoding device 1600, the decoding device is A receiving module 1601 is configured to receive a bitstream and to obtain indicator values ​​for the current block according to the bitstream. A segmental parameter processing module 1602 is configured to obtain the value of a first parameter and the value of a second parameter for the current block according to the indicator value and a predetermined lookup table, A calculation module 1603 is configured to obtain the sample distance value for a sample located in the current block according to the values ​​of the first and second parameters, The system includes a prediction module 1604 configured to obtain a predicted value for a sample according to the sample distance value for the sample.

[0070] A method according to the first aspect of the present invention may be carried out by an apparatus according to the second aspect of the present invention. Further features and implementations of the above method correspond to features and implementations of the apparatus according to the second aspect of the present invention.

[0071] In one embodiment, a decoder (30) or encoder (20) is disclosed, comprising a processing circuit for performing the method according to any one of the embodiments and implementations described above.

[0072] In one embodiment, a computer program product is disclosed that includes program code for performing a method according to any one of the above embodiments and implementations.

[0073] In one embodiment, One or more processors, A decoder or encoder is disclosed, comprising a non-temporary computer-readable storage medium connected to a processor and storing a program for execution by the processor, wherein the decoder or encoder is configured to perform a method according to any one of the above embodiments and implementations when the program is executed by the processor.

[0074] In one embodiment, a non-temporary storage medium is disclosed, which includes an encoded bitstream decoded by an image decoding device, wherein the bitstream is generated by dividing a frame of a video signal or image signal into a plurality of blocks and including a plurality of syntax elements, the plurality of syntax elements including an indicator (syntax) according to any one of the embodiments and implementations described above.

[0075] Details of one or more embodiments are described in the accompanying drawings and the following description. Other features, purposes, and advantages will be apparent from the description, drawings, and claims.

[0076] Embodiments of the present invention will be described in more detail below with reference to the accompanying figures and drawings. [Brief explanation of the drawing]

[0077] [Figure 1A] This is a block diagram showing an example of a video coding system configured to implement embodiments of the present invention. [Figure 1B] This is a block diagram showing another example of a video coding system configured to implement embodiments of the present invention. [Figure 2] This is a block diagram showing an example of a video encoder configured to implement an embodiment of the present invention. [Figure 3] This is a block diagram illustrating an exemplary structure of a video decoder configured to implement embodiments of the present invention. [Figure 4] This is a block diagram showing examples of encoding or decoding devices. [Figure 5] This is a block diagram showing another example of an encoding or decoding device. [Figure 6a] This diagram illustrates an example of blocks in the same location. [Figure 6b] This diagram illustrates an example of spatially adjacent blocks. [Figure 7] This diagram illustrates several examples of triangular prediction modes. [Figure 8] This figure illustrates some examples of subblock prediction modes. [Figure 9] This figure shows some examples of block divisions. [Figure 10] This figure shows some examples of block divisions. [Figure 11] This figure shows some examples of block divisions. [Figure 12] This figure shows some examples of block divisions. [Figure 13] This is a block diagram showing an exemplary structure of a content supply system 3100 that realizes a content distribution service. [Figure 14] This is a block diagram showing the structure of an example terminal device. [Figure 15] This flowchart shows how embodiments of the method refer to the present invention. [Figure 16] This block diagram indicates that an embodiment of the apparatus refers to the present invention. [Modes for carrying out the invention]

[0078] In the following, unless otherwise explicitly specified, the same reference numeral refers to the same, or at least functionally equivalent, feature.

[0079] In the following description, references are made to the accompanying drawings, which form part of this disclosure and illustrate specific embodiments of the present invention or specific ways in which embodiments of the present invention may be used. Embodiments of the present invention may be used in other embodiments and may include structural or logical changes not shown in the drawings. Accordingly, the following detailed description should not be taken as limiting, and the scope of the present invention is defined by the appended claims.

[0080] For example, disclosures relating to a described method may also apply to a corresponding device or system configured to perform the method, and vice versa. For example, if one or more specific method steps are described, the corresponding device may include one or more units for performing the described one or more method steps (e.g., one unit performing one or more steps, or multiple units each performing one or more of the steps), even if such one or more units are not explicitly described or shown in the drawings. On the other hand, if a specific device is described based on one or more units (e.g., a functional unit), the corresponding method may include one step for performing the function of one or more units (e.g., one step performing the function of one or more units, or multiple steps each performing one or more of the functions of multiple units), even if such one or more steps are not explicitly described or shown in the drawings. Furthermore, it is understood that the various exemplary embodiments and / or features of the aspects described herein may be combined with each other unless otherwise specifically stated.

[0081] Video coding typically refers to the processing of a sequence of pictures that make up a video or video sequence. In the field of video coding, the terms "frame" or "image" may be used as synonyms instead of the term "picture." Video coding (or coding in general) consists of two parts: video encoding and video decoding. Video encoding is performed on the source side and typically involves processing the original video picture (e.g., by compression) to reduce the amount of data required to represent the video picture (for more efficient storage and / or transmission). Video decoding is performed on the destination side and typically involves the reverse processing compared to encoding, in order to reconstruct the video picture. Embodiments that refer to "coding" a video picture (or picture in general) should be understood as relating to the "encoding" or "decoding" of the video picture or each video sequence. The combination of the encoding and decoding parts is also called a codec (coding and decoding).

[0082] In lossless video coding, the original video picture can be reconstructed, meaning the reconstructed video picture will have the same quality as the original video picture (assuming there is no transmission loss or other data loss during storage or transmission). In lossy video coding, further compression is performed, for example by quantization, to reduce the amount of data representing the video picture, which cannot be fully reconstructed in the decoder, meaning the quality of the reconstructed video picture is lower or worse compared to the quality of the original video picture.

[0083] Several video coding standards belong to the group of “lossy hybrid video codecs” (i.e., combining spatial and temporal predictions in the sample domain with 2D transform coding to apply quantization in the transform domain). Each picture in a video sequence is typically divided into a set of non-overlapping blocks, and coding is typically performed at the block level. In other words, at the block (video block) level, video is typically processed, i.e., coded, in the encoder, for example, by generating predicted blocks using spatial (intra-picture) predictions and / or temporal (inter-picture) predictions, subtracting the predicted blocks from the current block (the block currently being processed / will be processed) to obtain a residual block, and transforming the residual block to quantize the residual block in the transform domain and reduce (compress) the amount of data that will be transmitted. Meanwhile, in the decoder, the reverse processing compared to the encoder is applied to the coded or compressed block in order to reconstruct the current block for representation. Furthermore, the encoder replicates the decoder processing loop so that both the encoder and decoder generate identical predictions (e.g., intra-predictions and inter-predictions) and / or reconstructions for processing subsequent blocks, i.e., coding.

[0084] Hereinafter, embodiments of the video coding system 10, video encoder 20, and video decoder 30 will be described with reference to Figures 1 to 3.

[0085] Figure 1A is a schematic block diagram illustrating an exemplary coding system 10, such as a video coding system 10 (or short coding system 10), which may utilize the technique of this application. The video encoder 20 (or short encoder 20) and video decoder 30 (or short decoder 30) of the video coding system 10 represent examples of devices that may be configured to perform the technique according to the various examples described herein.

[0086] As shown in Figure 1A, the coding system 10 includes a source device 12 configured to provide encoded picture data 21 to a destination device 14 for decoding the encoded picture data 13.

[0087] The source device 12 includes an encoder 20 and may additionally, i.e., optionally, include a picture source 16, a preprocessor (or preprocessing unit) 18 such as a picture preprocessor 18, and a communication interface or communication unit 22.

[0088] The picture source 16 may include, or may include, any kind of picture capture device, such as a camera for taking real-world photographs, and / or any kind of picture generation device, such as a computer graphics processor for generating computer-animated pictures, or any kind of other device for acquiring and / or providing real-world pictures, computer-generated pictures (e.g., screen content, virtual reality (VR) pictures), and / or any combination thereof (e.g., augmented reality (AR) pictures). The picture source may also include any kind of memory or storage for storing any of the aforementioned pictures.

[0089] To distinguish it from the processing performed by the preprocessor 18 and the preprocessing unit 18, the picture or picture data 17 may also be called the raw picture or raw picture data 17.

[0090] The preprocessor 18 is configured to receive (raw) picture data 17, perform preprocessing on the picture data 17 to obtain a preprocessed picture 19 or preprocessed picture data 19. The preprocessing performed by the preprocessor 18 may include, for example, cropping, color format conversion (e.g., conversion from RGB to YCbCr), color correction, or noise reduction. It can be understood that the preprocessing unit 18 may be an optional component.

[0091] The video encoder 20 is configured to receive pre-processed picture data 19 and provide encoded picture data 21 (further details are described below, for example, based on Figure 2).

[0092] The communication interface 22 of the source device 12 may be configured to receive the encoded picture data 21 and transmit the encoded picture data 21 (or any further processed version thereof) to another device, such as the destination device 14 or any other device, via the communication channel 13 for storage or direct reconstruction.

[0093] The destination device 14 includes a decoder 30 (for example, a video decoder 30) and may additionally, or optionally, include a communication interface or communication unit 28, a post-processor 32 (or post-processing unit 32), and a display device 34.

[0094] The communication interface 28 of the destination device 14 is configured to receive encoded picture data 21 (or any further processed version thereof) from, for example, the source device 12 directly, or from any other source, such as a storage device for encoded picture data, and to provide the encoded picture data 21 to the decoder 30.

[0095] Communication interfaces 22 and 28 may be configured to transmit or receive encoded picture data 21 or encoded data 13 via a direct communication link between the source device 12 and the destination device 14, such as a direct wired or wireless connection, or via any type of network, such as a wired network, a wireless network, or any combination thereof, or any type of private network, public network, or any combination thereof.

[0096] The communication interface 22 may be configured to, for example, package the encoded picture data 21 into an appropriate format such as a packet, and / or process the encoded picture data using any kind of encoding or processing for transmission over a communication link or communication network.

[0097] The communication interface 28, which is the counterpart to the communication interface 22, may be configured, for example, to receive transmitted data, process the transmitted data using any kind of corresponding decoding or processing and / or depackaging of the transmission, and obtain encoded picture data 21.

[0098] Both communication interfaces 22 and 28 may be configured as unidirectional interfaces, as indicated by the arrows to the communication channel 13 in Figure 1A pointing from source device 12 to destination device 14, or as bidirectional communication interfaces, and may be configured to send and receive messages, for example, to set up a connection, in order to exchange and acknowledge any other information regarding the communication link and / or data transmission, such as the transmission of encoded picture data.

[0099] The decoder 30 is configured to receive the encoded picture data 21 and provide the decoded picture data 31 or the decoded picture 31 (further details are described below, for example, based on Figure 3 or Figure 5).

[0100] The post-processor 32 of the destination device 14 is configured to post-process the decoded picture data 31 (also called reconstructed picture data), such as the decoded picture 31, to obtain post-processed picture data 33, such as the post-processed picture 33. Post-processing performed by the post-processing unit 32 may include, for example, color format conversion (e.g., conversion from YCbCr to RGB), color correction, cropping, or resampling, or any other processing to prepare the decoded picture data 31 for display by, for example, the display device 34.

[0101] The display device 34 of the destination device 14 is configured to receive post-processed picture data 33 for displaying the picture to, for example, a user or viewer. The display device 34 may be, or comprise, any type of display for representing the reconstructed picture, such as an integrated or external display or monitor. The display may include, for example, a liquid crystal display (LCD), an organic light-emitting diode (OLED) display, a plasma display, a projector, a microLED display, an LCoS (liquid crystal on silicon), a digital optical processor (DLP), or any other type of display.

[0102] Figure 1A illustrates the source device 12 and the destination device 14 as separate devices, but the device embodiment may include both the source device 12 and the destination device 14, or both the functionality corresponding to the source device 12 and the functionality corresponding to the destination device 14. In such embodiments, the source device 12 or its corresponding functionality and the destination device 14 or its corresponding functionality may be implemented using the same hardware and / or software, or by separate hardware and / or software, or any combination thereof.

[0103] As will be apparent to those skilled in the art based on the description, the functions of the various units, or the presence and (strict) division of functions within the source device 12 and / or destination device 14, as shown in Figure 1A, may vary depending on the actual device and application.

[0104] An encoder 20 (e.g., a video encoder 20) or a decoder 30 (e.g., a video decoder 30), or both an encoder 20 and a decoder 30, may be implemented via processing circuits, such as those shown in Figure 1B, including one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic circuits, hardware, dedicated to video coding, or any combination thereof. The encoder 20 may be implemented via processing circuits 46 to embody various modules, as discussed with respect to the encoder 20 in Figure 2 and / or any other encoder systems or subsystems described herein. The decoder 30 may be implemented via processing circuits 46 to embody various modules, as discussed with respect to the decoder 30 in Figure 3 and / or any other decoder systems or subsystems described herein. The processing circuits may be configured to perform various operations, as will be discussed later. As shown in Figure 5, when the technique is partially implemented in software, the device can store instructions for the software in a suitable non-temporary computer-readable storage medium and execute the instructions in hardware using one or more processors for performing the technique of the present disclosure. Either the video encoder 20 or the video decoder 30 may be integrated as part of a synthesized encoder / decoder (codec) in a single device, for example, as shown in Figure 1B.

[0105] The source device 12 and destination device 14 may include any type of handheld or stationary device, encompassing a wide range of devices such as notebook or laptop computers, mobile phones, smartphones, tablets or tablet computers, cameras, desktop computers, set-top boxes, televisions, display devices, digital media players, video game consoles, video streaming devices (such as content service servers or content distribution servers), broadcast receiving devices, broadcast transmitting devices, etc., and may or may not use an operating system, or may use any type of operating system. In some cases, the source device 12 and destination device 14 may support wireless communication. Therefore, the source device 12 and destination device 14 may be wireless communication devices.

[0106] In some cases, the video coding system 10 shown in Figure 1A is merely an example, and the techniques of this application may be applied to video coding configurations (e.g., video coding or video decoding) that do not necessarily involve data communication between an coding device and a decoding device. In other examples, data may be retrieved from local memory or streamed over a network. A video coding device can code the data and store it in memory, and / or a video decoding device can retrieve the data from memory and decode it. In some examples, coding and decoding are performed by devices that do not communicate with each other, but simply code the data into memory and / or retrieve the data from memory and decode it.

[0107] For the sake of explanation, embodiments of the present invention will be described herein with reference to, for example, HEVC (High-Efficiency Video Coding) or VVC (Versatile Video Coding), a next-generation video coding standard developed by the ITU-T VCEG (Video Coding Experts Group) and the ISO / IEC MPEG (Motion Picture Experts Group) JCT-VC (Joint Collaboration Team on Video Coding). Those skilled in the art will understand that embodiments of the present invention are not limited to HEVC or VVC.

[0108] Encoder and encoding method Figure 2 shows a schematic block diagram of an exemplary video encoder 20 configured to carry out the technique of the present application. In the example of Figure 2, the video encoder 20 comprises an input 201 (or input interface 201), a residual calculation unit 204, a transformation unit 206, a quantization unit 208, an inverse quantization unit 210, and an inverse transformation unit 212, a reconstruction unit 214, a loop filter unit 220, a decoded picture buffer (DPB) 230, a mode selection unit 260, an entropy coding unit 270, and an output 272 (or output interface 272). The mode selection unit 260 may include an inter-prediction unit 244, an intra-prediction unit 254, and a segmentation unit 262. The inter-prediction unit 244 may include a motion estimation unit and a motion compensation unit (not shown). The video encoder 20 as shown in Figure 2 may also be called a hybrid video encoder or a video encoder according to a hybrid video codec.

[0109] The residual calculation unit 204, the conversion processing unit 206, the quantization unit 208, and the mode selection unit 260 are sometimes referred to as forming the forward signal path of the encoder 20, while the inverse quantization unit 210, the inverse conversion processing unit 212, the reconstruction unit 214, the buffer 216, the loop filter 220, the decoding picture buffer (DPB) 230, the inter-prediction unit 244, and the intra-prediction unit 254 are sometimes referred to as forming the reverse signal path of the video encoder 20, and the reverse signal path of the video encoder 20 corresponds to the signal path of the decoder (see video decoder 30 in Figure 3). The inverse quantization unit 210, the inverse conversion processing unit 212, the reconstruction unit 214, the loop filter 220, the decoding picture buffer (DPB) 230, the inter-prediction unit 244, and the intra-prediction unit 254 are also referred to as forming the “built-in decoder” of the video encoder 20.

[0110] Pictures and picture sections (pictures and blocks) The encoder 20 may be configured to receive, for example, a picture 17 (or picture data 17) via input 201, such as a picture in a sequence of pictures that make up a video or video sequence. The received picture or picture data may also be a pre-processed picture 19 (or pre-processed picture data 19). For brevity, the following description will refer to picture 17. Picture 17 may also be called the current picture or the picture to be coded (particularly in video coding to distinguish the current picture from other pictures, such as previously encoded and / or decoded pictures in the same video sequence, i.e., the video sequence that also contains the current picture).

[0111] A (digital) picture is, or can be considered as, a two-dimensional array or matrix of samples having intensity values. A sample in an array may also be called a pixel (a shortened form of picture element) or pel. The number of samples in the horizontal and vertical (or axis) directions of an array or picture defines the size and / or resolution of the picture. For color representation, typically three color components are used; that is, a picture is represented by, or may contain, three sample arrays. In the RGB format or color space, a picture contains corresponding red, green, and blue sample arrays. However, in video coding, each pixel is typically represented in a luminance and chrominance format or color space, for example, YCbCr, which includes a luminance component represented by Y (sometimes L is used instead) and two chrominance components represented by Cb and Cr. The luminance (or abbreviated as lumin) component Y represents brightness or gray level intensity (for example, in a grayscale picture), while the two chroma (or chromatic) components Cb and Cr represent chromaticity or color information components. Therefore, a picture in YCbCr format contains a luminance sample array of luminance sample values ​​(Y) and two chroma sample arrays of chroma difference values ​​(Cb and Cr). A picture in RGB format may be converted to or transformed to YCbCr format, and vice versa; this process is also known as color transformation or color conversion. If a picture is monochrome, it may contain only a luminance sample array. Therefore, a picture may, for example, be an array of lumina samples in a monochrome format, or two corresponding arrays of lumina samples and chroma samples in 4:2:0, 4:2:2, and 4:4:4 color formats.

[0112] Embodiments of the video encoder 20 may include a picture partitioning unit (not shown in Figure 2) configured to partition a picture 17 into multiple (typically non-overlapping) picture blocks 203. These blocks may also be called root blocks, macroblocks (H.264 / AVC), coding tree blocks (CTB), or coding tree units (CTU) (H.265 / HEVC and VVC). The picture partitioning unit may use the same block size for all pictures in the video sequence and a corresponding grid that defines that block size, or it may be configured to vary the block size between pictures or subsets or groups of pictures, partitioning each picture into a corresponding block.

[0113] In a further embodiment, the video encoder may be configured to directly receive blocks 203 of picture 17, such as one, some, or all of the blocks that make up picture 17. The picture blocks 203 may also be called the current picture block or the picture block to be coded.

[0114] Like picture 17, picture block 203 is, or can be considered, a two-dimensional array or matrix of samples having intensity values ​​(sample values), but with fewer dimensions than picture 17. In other words, block 203 can contain, for example, one sample array (e.g., a lumar array for monochrome picture 17, or a lumar or chromar array for color picture) or three sample arrays (e.g., a lumar array and two chromar arrays for color picture 17), or any other number and / or type of arrays depending on the color format applied. The number of samples in the horizontal and vertical (or axis) directions of block 203 defines the size of block 203. Thus, a block can be, for example, an MxN (M columns vs. N rows) array of samples, or an MxN array of transformation coefficients.

[0115] An embodiment of the video encoder 20, as shown in Figure 2, may be configured to encode the picture 17 block by block, for example, encoding and prediction may be performed for each block 203.

[0116] Residual calculation The residual calculation unit 204 may be configured to calculate the residual block 205 (also called residual 205) based on the picture block 203 and the prediction block 265 (further details about the prediction block 265 will be given later), for example, by subtracting the sample values ​​of the prediction block 265 from the sample values ​​of the picture block 203 and obtaining the residual block 205 in the sample region for each sample (for each pixel).

[0117] conversion The transformation processing unit 206 may be configured to apply a transformation, such as a discrete cosine transform (DCT) or discrete sine transform (DST), to the sample values ​​of the residual block 205 in order to obtain transformation coefficients 207 in the transformation domain. The transformation coefficients 207, also called transformation residual coefficients, may represent the residual block 205 in the transformation domain.

[0118] The conversion processing unit 206 may be configured to apply an integer approximation of DCT / DST, such as the conversion specified for H.265 / HEVC. Compared to the orthogonal DCT conversion, such an integer approximation is typically scaled by a certain coefficient. An additional scaling coefficient is applied as part of the conversion process to maintain the norm of the residual blocks processed by the forward and reverse conversions. The scaling coefficient is typically chosen based on some constraint, such as the scaling coefficient being a power of 2 for shift operations, the bit depth of the conversion coefficient, and a trade-off between accuracy and implementation cost. A specific scaling coefficient is specified, for example, for the reverse conversion by the reverse conversion processing unit 212 (and the corresponding reverse conversion by the reverse conversion processing unit 312 in the video decoder 30, for example), and the corresponding scaling coefficient for the forward conversion in the encoder 20 by the conversion processing unit 206 may be specified accordingly.

[0119] Embodiments of the video encoder 20 (each a conversion processing unit 206) may be configured to output conversion parameters, such as the type of conversion or multiple conversions, either directly or in an encoded or compressed state via the entropy coding unit 270, so that, for example, the video decoder 30 can receive and use the conversion parameters for decoding.

[0120] quantization The quantization unit 208 may be configured to quantize the transformation coefficient 207 to obtain a quantized coefficient 209, for example by applying scalar quantization or vector quantization. The quantized coefficient 209 may also be called the quantized transformation coefficient 209 or the quantized residual coefficient 209.

[0121] The quantization process can reduce the bit depth associated with some or all of the 207 conversion coefficients. For example, n-bit conversion coefficients may be truncated to m-bit conversion coefficients during quantization, where n is greater than m. The degree of quantization can be modified by adjusting the quantization parameter (QP). For example, in scalar quantization, different scalings can be applied to achieve finer or coarser quantization. Smaller quantization step sizes correspond to finer quantization, while larger quantization step sizes correspond to coarser quantization. Applicable quantization step sizes can be indicated by the quantization parameter (QP). The quantization parameter can be, for example, an index to a predetermined set of applicable quantization step sizes. For example, a small quantization parameter may correspond to finer quantization (smaller quantization step size), a large quantization parameter may correspond to coarser quantization (larger quantization step size), or vice versa. Quantization may involve a partition by a certain quantization step size, and the corresponding inverse quantization and / or inverse inverse quantization by, for example, the inverse quantization unit 210 may involve multiplication by that quantization step size. Embodiments conforming to several standards, such as HEVC, may be configured to use quantization parameters to determine the quantization step size. Generally, the quantization step size may be calculated based on the quantization parameters using a fixed-point approximation of the equations involving the partition. Additional scaling factors may be introduced for quantization and inverse quantization to restore the norm of the residual block, which may be modified due to the scaling used in the fixed-point approximation of the equations for the quantization step size and quantization parameters. In one exemplary implementation, the scaling of the inverse transform and inverse quantization may be combined. Alternatively, a customized quantization table may be used to signal from encoder to decoder, for example, in a bitstream. Quantization is a lossy operation, and the loss increases as the quantization step size increases.

[0122] Embodiments of the video encoder 20 (each a quantization unit 208) may be configured to output quantization parameters (QP) either directly or encoded via the entropy coding unit 270, for example, so that a video decoder 30 can receive and apply the quantization parameters for decoding.

[0123] inverse quantization The inverse quantization unit 210 is configured to apply the inverse quantization of the quantization unit 208 to the quantized coefficients in order to obtain the inverse quantized coefficients 211, for example, by applying the inverse of the quantization scheme applied by the quantization unit 208, based on or using the same quantization step size as the quantization unit 208. The inverse quantized coefficients 211 are also sometimes called the inverse quantized residual coefficients 211, and are not usually identical to the transformation coefficients due to quantization losses, but may correspond to the transformation coefficients 207.

[0124] Inverse Transform The inverse transform processing unit 212 is configured to obtain a reconstructed residual block 213 (or the corresponding inverse quantized coefficient 213) in the sample region by applying the inverse transform of the transform applied by the transform processing unit 206, such as the inverse discrete cosine transform (DCT) or the inverse discrete sine transform (DST) or other inverse transforms. The reconstructed residual block 213 may also be called the transform block 213.

[0125] Rebuild The reconstruction unit 214 (for example, an adder or subtractor 214) is configured to obtain the reconstructed block 215 in the sample region by adding the sample value of the reconstructed residual block 213 to the sample value of the prediction block 265, for example, sample by sample, thereby adding the transformed block 213 (i.e., the reconstructed residual block 213) to the prediction block 265.

[0126] filtering The loop filter unit 220 (or simply "loop filter" 220) is configured to filter the reconstructed block 215 to obtain a filtered block 221, or more generally, to filter the reconstructed samples to obtain filtered samples. The loop filter unit is configured, for example, to smooth pixel transitions or to improve video quality in other ways. The loop filter unit 220 may include one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or one or more other filters such as a bilateral filter, an adaptive loop filter (ALF), a sharpening filter, a smoothing filter, or a co-filter, or any combination thereof. In Figure 2, the loop filter unit 220 is shown as a loop filter, but in other configurations, the loop filter unit 220 may be implemented as a post-loop filter. The filtered block 221 may also be called the filtered reconstructed block 221.

[0127] Embodiments of the video encoder 20 (each a loop filter unit 220) may be configured to output loop filter parameters (such as sample adaptive offset information) either directly or encoded via the entropy coding unit 270, for example, so that a decoder 30 can receive and apply the same loop filter parameters or the respective loop filters for decoding.

[0128] Decode picture buffer The decoded picture buffer (DPB) 230 may be a memory that stores a reference picture, or generally reference picture data, for encoding video data by the video encoder 20. The DPB 230 may be formed from any of various memory devices, such as dynamic random access memory (DRAM) including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM®), or other types of memory devices. The decoded picture buffer (DPB) 230 may be configured to store one or more filtered blocks 221. The decoded picture buffer 230 may further be configured to store other previously filtered blocks, such as previously reconstructed and filtered blocks 221 of the same current picture or of different pictures, such as previously reconstructed pictures, for example, to provide a fully reconstructed, i.e., decoded picture (and corresponding reference blocks and samples) and / or a partially reconstructed current picture (and corresponding reference blocks and samples) for interpretation. The decoded picture buffer (DPB) 230 may also be configured to store one or more unfiltered reconstructed blocks 215, or, in cases where the reconstructed blocks 215 are not filtered by the loop filter unit 220, generally unfiltered reconstructed samples, or any other further processed versions of the reconstructed blocks or samples.

[0129] Mode selection (category and prediction) The mode selection unit 260 comprises a segmentation unit 262, an inter-prediction unit 244, and an intra-prediction unit 254, and is configured to receive or acquire, for example, the original picture data, e.g., the original block 203 (the current block 203 of the current picture 17), and the reconstructed picture data, e.g., filtered and / or unfiltered reconstructed samples or blocks from the same (current) picture and / or from one or more previously decoded pictures, from, for example, the decoded picture buffer 230 or other buffers (e.g., line buffers not shown). The reconstructed picture data is used as reference picture data for prediction, such as inter-prediction or intra-prediction, in order to acquire the prediction block 265 or predictor 265.

[0130] The mode selection unit 260 may be configured to determine or select a prediction mode (including no distinction) and a prediction mode (e.g., intra or inter prediction mode) for the current block to generate a corresponding prediction block 265, which is used for calculating the residual block 205 and for reconstructing the reconstructed block 215.

[0131] Embodiments of the mode selection unit 260 may be configured to select a partition and prediction mode (for example, from those supported by or available to the mode selection unit 260) that yields the best match or, in other words, the smallest residual (the smallest residual means better compression for transmission or storage), or the smallest signaling overhead (the smallest signaling overhead means better compression for transmission or storage), or both, or a balance between the two. The mode selection unit 260 may be configured to determine the partition and prediction modes based on rate distortion optimization (RDO), i.e., to select a prediction mode that yields the smallest rate distortion. In this context, terms such as “best,” “smallest,” and “optimal” do not necessarily refer to the overall “best,” “smallest,” and “optimal,” but may also refer to the satisfaction of termination criteria or selection criteria, such as values ​​exceeding or falling below a threshold or other constraint, which may lead to a “suboptimal selection” but reduce complexity and processing time.

[0132] In other words, the partitioning unit 262 may be configured to partition block 203 into smaller block partitions or subblocks (which then form blocks) by repeatedly using, for example, quadtree partitions (QT), binary tree partitions (BT), or ternary tree partitions (TT), or any combination thereof, and to perform, for example, predictions for each of the block partitions or subblocks, where mode selection includes selecting the tree structure of the partitioned block 203 and prediction modes are applied to each of the block partitions or subblocks.

[0133] The following describes in more detail the segmentation (for example, by the segmentation unit 260) and prediction processing (by the inter-prediction unit 244 and the intra-prediction unit 254) performed by the exemplary video encoder 20.

[0134] classification The partitioning unit 262 may partition (or divide) the current block 203 into smaller partitions, such as smaller blocks of square or rectangular size. These smaller blocks (also called subblocks) may be further partitioned into even smaller partitions. This is also called tree partitioning or hierarchical tree partitioning, and for example, the root block at root tree level 0 (hierarchy level 0, depth 0) may be recursively partitioned into two or more blocks at the next lower tree level, such as a node at tree level 1 (hierarchy level 1, depth 1), and these blocks may be further partitioned into two or more blocks at the next lower level, such as tree level 2 (hierarchy level 2, depth 2), until the partitioning ends when a termination criterion is met, such as reaching the maximum tree depth or the minimum block size. Blocks that are not further partitioned are also called tree leaf blocks or leaf nodes. A tree that uses partitioning into two partitions is called a binary tree (BT), a tree that uses partitioning into three partitions is called a ternary tree (TT), and a tree that uses partitioning into four partitions is called a quadtree (QT).

[0135] As previously mentioned, the term “block” as used herein can refer to a portion of a picture, particularly a square or rectangular portion. For example, with respect to HEVC and VVC, a block may be a coding tree unit (CTU), coding unit (CU), prediction unit (PU), and transformation unit (TU), and / or a corresponding block such as a coding tree block (CTB), coding block (CB), transformation block (TB), or prediction block (PB).

[0136] For example, a coding tree unit (CTU) may be a CTB for a luminous sample, two corresponding CTBs for a chroma sample of a picture having three sample arrays, or a CTB for a sample of a picture coded using three separate color planes and syntax structures used to code a monochrome picture or sample, or may comprise these. Correspondingly, a coding tree block (CTB) may be an NxN block of samples for some value of N such that the division of components into the CTB is a partition. A coding unit (CU) may be a coding block for a luminous sample, two corresponding coding blocks for a chroma sample of a picture having three sample arrays, or a coding block for a sample of a picture coded using three separate color planes and syntax structures used to code a monochrome picture or sample, or may comprise these. Correspondingly, a coding block (CB) may be an MxN block of samples for some values ​​of M and N such that the division of the CTB into the coding block is a partition.

[0137] In embodiments, for example, according to HEVC, a coding tree unit (CTU) may be divided into CUs by using a quadtree structure denoted as a coding tree. The decision of whether to code a picture area using interpicture (time) prediction or intrapicture (spatial) prediction is made at the CU level. Each CU may be further divided into one, two, or four PUs, depending on the PU division type. Within a single PU, the same prediction process is applied, and the relevant information is sent to the decoder for each PU. After obtaining residual blocks by applying the prediction process based on the PU division type, the CU may be divided into transformation units (TUs) according to another quadtree structure, similar to the coding tree for the CU.

[0138] In some embodiments, for example, according to the latest video coding standard currently under development called VVC (Versatile Video Coding), a QTBT (Quad-tree and binary tree) is used to segment coding blocks. In the QTBT block structure, CUs can have either a square or rectangular shape. For example, a coding tree unit (CTU) is first segmented by a quad-tree structure. The quad-tree leaf nodes are further segmented by a binary or ternary (or ternary) tree structure. The segmented tree leaf nodes are called coding units (CUs), and their segmentation is used for prediction and transformation processing without further segmentation. This means that CUs, PUs, and TUs have the same block size in the QTBT coding block structure. In parallel, it has been proposed that multiple segmentations, such as ternary tree segmentations, may also be used with the QTBT block structure.

[0139] In one example, the mode selection unit 260 of the video encoder 20 may be configured to perform any combination of the segmentation techniques described herein.

[0140] As described above, the video encoder 20 is configured to determine or select the best or most optimal prediction mode from a set of (predetermined) prediction modes. The set of prediction modes may include, for example, an intra-prediction mode and / or an inter-prediction mode.

[0141] Intra Prediction The set of intra-prediction modes may include 35 different intra-prediction modes, such as DC (or average) mode and omnidirectional modes like planar mode, or directional modes, such as those defined in HEVC, or it may include 67 different intra-prediction modes, such as DC (or average) mode and omnidirectional modes like planar mode, or directional modes, such as those defined for VVC.

[0142] The intra-prediction unit 254 is configured to use reconstructed samples of adjacent blocks of the same current picture to generate an intra-prediction block 265 according to one of the intra-prediction modes in a set of intra-prediction modes.

[0143] The intra-prediction unit 254 (or generally the mode selection unit 260) is further configured to output intra-prediction parameters (or generally information indicating the selected intra-prediction mode for a block) to the entropy coding unit 270 in the form of syntax elements 266 for inclusion in the encoded picture data 21, for example, so that the video decoder 30 can receive and use the prediction parameters for decoding.

[0144] Interpretation The set of interprediction modes (or possible interprediction modes) depends on the available reference picture (i.e., a previously at least partially decoded picture stored in DBP230, for example) and other interprediction parameters, for example whether the entire reference picture is used to search for the best matching reference block, or only a portion of the reference picture is used, such as a search window area around the area of ​​the current block, and / or whether pixel interpolation is applied, for example, half / semipel interpolation and / or quarter-pel interpolation.

[0145] In addition to the prediction modes described above, skip mode and / or direct mode may be applied.

[0146] The interpretation unit 244 may include motion estimation (ME) units and motion compensation (MC) units (neither of which are shown in Figure 2). The motion estimation unit may be configured to receive or acquire for motion estimation picture block 203 (the current picture block 203 of the current picture 17) and the decoded picture 231, or at least one or more previously reconstructed blocks, such as one or more other / different previously decoded picture 231 reconstructed blocks. For example, a video sequence may include the current picture and the previously decoded picture 231, or in other words, the current picture and the previously decoded picture 231 may be part of, or may form, a sequence of pictures that make up the video sequence.

[0147] The encoder 20 may be configured, for example, to select a reference block from multiple reference blocks of the same or different pictures of multiple other pictures, and provide the motion estimation unit with the reference picture (or reference picture index), and / or the offset (spatial offset) between the location (x,y coordinates) of the reference block and the location of the current block, as interpretation parameters. This offset is also called the motion vector (MV).

[0148] The motion compensation unit is configured to receive interprediction parameters and perform interprediction based on or using the interprediction parameters to obtain interprediction blocks 265. Motion compensation performed by the motion compensation unit may involve fetching or generating prediction blocks based on the motion / block vector determined by motion estimation, and possibly performing interpolation to sub-pixel precision. Interpolation filtering may generate additional pixel samples from known pixel samples, thus increasing the number of prediction block candidates that can be used to code picture blocks. Upon receiving the motion vector for the current picture block's PU, the motion compensation unit can find the prediction block pointed to by the motion vector in one of the reference picture lists.

[0149] The motion compensation unit may also generate syntax elements associated with blocks and video slices for use by the video decoder 30 when decoding picture blocks of video slices.

[0150] Entropy coding The entropy coding unit 270 is configured to apply, for example, an entropy coding algorithm or scheme (e.g., variable-length coding (VLC) scheme, context-adaptive VLC scheme (CALVC), arithmetic coding scheme, binarization, context-adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval-partitioned entropy (PIPE) coding, or another entropy coding method or technique) or bypass (no compression) to quantized coefficients 209, inter-prediction parameters, intra-prediction parameters, loop filter parameters, and / or other syntax elements to obtain encoded picture data 21 which can be output via output 272 in the form of an encoded bitstream 21, for example, so that the video decoder 30 can receive and use the parameters for decoding. The encoded bitstream 21 can be transmitted to the video decoder 30 or stored in memory for later transmission or retrieval by the video decoder 30.

[0151] Other structural variations of the video encoder 20 may be used to encode video streams. For example, a non-conversion-based encoder 20 can directly quantize residual signals for some blocks or frames without a conversion processing unit 206. In another implementation, the encoder 20 may have a quantization unit 208 and an inverse quantization unit 210 that are combined into a single unit.

[0152] Decoder and decoding method Figure 3 shows an example of a video decoder 30 configured to implement the technique of the present application. The video decoder 30 is configured to receive encoded picture data 21 (e.g., encoded bitstream 21), which has been encoded by, for example, an encoder 20, in order to obtain a decoded picture 331. The encoded picture data or bitstream contains information for decoding the encoded picture data, such as data representing picture blocks and associated syntax elements of an encoded video slice.

[0153] In the example shown in Figure 3, the decoder 30 comprises an entropy decoding unit 304, an inverse quantization unit 310, an inverse transformation processing unit 312, a reconstruction unit 314 (e.g., an adder 314), a loop filter 320, a decoded picture buffer (DBP) 330, an interpretation unit 344, and an intraprediction unit 354. The interpretation unit 344 may be or may include a motion compensation unit. In some examples, the video decoder 30 may perform a decoding path that is generally the reverse of the encoding path described with respect to the video encoder 100 from Figure 2.

[0154] As described with respect to encoder 20, the inverse quantization unit 210, inverse processing unit 212, reconstruction unit 214, loop filter 220, decoding picture buffer (DPB) 230, inter-prediction unit 344, and intra-prediction unit 354 are also referred to as the “built-in decoder” of video encoder 20. Thus, the inverse quantization unit 310 may have the same function as the inverse quantization unit 110, the inverse processing unit 312 may have the same function as the inverse processing unit 212, the reconstruction unit 314 may have the same function as the reconstruction unit 214, the loop filter 320 may have the same function as the loop filter 220, and the decoding picture buffer 330 may have the same function as the decoding picture buffer 230. Therefore, the descriptions given for each unit and function of video encoder 20 similarly apply to each unit and function of video decoder 30.

[0155] Entropy decoding The entropy decoding unit 304 is configured to analyze the bitstream 21 (or generally the encoded picture data 21) and, for example, perform entropy decoding on the encoded picture data 21 to obtain, for example, quantized coefficients 309 and / or decoded coding parameters (not shown in Figure 3), such as inter-prediction parameters (e.g., reference picture index and motion vector), intra-prediction parameters (e.g., intra-prediction mode or index), transformation parameters, quantization parameters, loop filter parameters, and / or other syntax elements, or any or all of them. The entropy decoding unit 304 may be configured to apply a decoding algorithm or scheme corresponding to an encoding scheme such as those described with respect to the entropy coding unit 270 of the encoder 20. The entropy decoding unit 304 may be further configured to provide the inter-prediction parameters, intra-prediction parameters, and / or other syntax elements to the mode selection unit 360, and the other parameters to other units of the decoder 30. The video decoder 30 may receive syntax elements at the video slice level and / or video block level.

[0156] inverse quantization The inverse quantization unit 310 may be configured to receive quantization parameters (QP) (or generally information about inverse quantization) and quantized coefficients from encoded picture data 21 (for example, by an entropy decoding unit 304, for example, by parsing and / or decoding), and to apply inverse quantization to the decoded quantized coefficients 309 based on the quantization parameters to obtain inverse quantized coefficients 311, which may also be called transformed coefficients 311. The inverse quantization process may include using quantization parameters determined by the video encoder 20 for each video block in the video slice to determine the degree of quantization and, similarly, the degree of inverse quantization to be applied.

[0157] Inverse Transform The inverse transformation processing unit 312 may be configured to receive the inversely quantized coefficients 311, also called the transformation coefficients 311, and to apply a transformation to the inversely quantized coefficients 311 in order to obtain the reconstructed residual block 213 in the sample region. The reconstructed residual block 213 may also be called the transformation block 313. The transformation may be an inverse transformation, such as an inverse DCT, inverse DST, inverse integer transformation, or a conceptually similar inverse transformation process. The inverse transformation processing unit 312 may be further configured to receive transformation parameters or corresponding information from the encoded picture data 21 (for example, by the entropy decoding unit 304, for example, by parsing and / or decoding) in order to determine the transformation to be applied to the inversely quantized coefficients 311.

[0158] Rebuild The reconstruction unit 314 (for example, an adder or summer 314) may be configured to add the reconstructed residual block 313 to the predicted block 365 by, for example, adding the sample values ​​of the reconstructed residual block 313 to the sample values ​​of the predicted block 365, thereby obtaining the reconstructed block 315 in the sample region.

[0159] filtering The loop filter unit 320 (either within or after the coding loop) is configured to filter the reconstructed block 315 to obtain a filtered block 321, for example, to smooth pixel transitions or to improve video quality in a different way. The loop filter unit 320 may include one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or one or more other filters, or any combination thereof, such as a bilateral filter, an adaptive loop filter (ALF), a sharpening filter, a smoothing filter, or a co-filter. In Figure 3, the loop filter unit 320 is shown as a loop filter, but in other configurations, the loop filter unit 320 may be implemented as a post-loop filter.

[0160] Decode picture buffer Next, the decoded video block 321 of the picture is stored in the decoded picture buffer 330, which stores the decoded picture 331 as a reference picture for subsequent motion compensation for other pictures and / or for output display, respectively.

[0161] The decoder 30 is configured to output the decoded picture 311, for example via the output 312, for presentation or viewing to the user.

[0162] prediction The inter-prediction unit 344 may be identical to the inter-prediction unit 244 (specifically, the motion compensation unit), and the intra-prediction unit 354 may have the same functionality as the inter-prediction unit 254, and performs division or division determination and prediction based on division and / or prediction parameters or their respective information received from the encoded picture data 21 (for example, by the entropy decoding unit 304, for example, by analysis and / or decoding). The mode selection unit 360 may be configured to obtain a predicted block 365 by performing a prediction (intra-prediction or inter-prediction) block by block based on the reconstructed picture, block, or each sample (filtered or unfiltered).

[0163] When a video slice is coded as an intra-coded (I) slice, the intra-prediction unit 354 of the mode selection unit 360 is configured to generate a prediction block 365 for the picture block of the current video slice based on the signaled intra-prediction mode and data from a previously decoded block of the current picture. When a video picture is coded as an inter-coded (i.e., B or P) slice, the inter-prediction unit 344 (e.g., a motion compensation unit) of the mode selection unit 360 is configured to generate a prediction block 365 for the video block of the current video slice based on motion vectors and other syntax elements received from the entropy decoding unit 304. In inter-prediction, the prediction block may be generated from one of the reference pictures in one of the reference picture lists. The video decoder 30 may construct reference frame lists List0 and List1 using default construction techniques based on the reference pictures stored in the DPB 330.

[0164] The mode selection unit 360 is configured to determine prediction information for video blocks in the current video slice by analyzing motion vectors and other syntax elements, and to use the prediction information to generate prediction blocks for the current video block being decoded. For example, the mode selection unit 360 uses some of the received syntax elements to determine the prediction mode (e.g., intra-prediction or inter-prediction) used to code the video blocks in the video slice, the inter-prediction slice type (e.g., B-slice, P-slice, or GPB-slice), construction information for one or more of the reference picture lists for the slice, motion vectors for each intercoded video block in the slice, the inter-prediction status for each intercoded video block in the slice, and other information for decoding the video blocks in the current video slice.

[0165] Other variations of the video decoder 30 may be used to decode the encoded picture data 21. For example, the decoder 30 can generate an output video stream without a loop filtering unit 320. For example, a non-transformation-based decoder 30 can directly dequantize the residual signal for some blocks or frames without an inverse transformation processing unit 312. In another implementation, the video decoder 30 may have an inverse quantization unit 310 and an inverse transformation processing unit 312 combined into a single unit.

[0166] It should be understood that in encoder 20 and decoder 30, the result of processing in the current step may be further processed and output to the next step. For example, after interpolation filtering, motion vector derivation, or loop filtering, further operations such as clipping or shifting may be performed on the result of interpolation filtering, motion vector derivation, or loop filtering.

[0167] Note that further operations can be applied to the derived motion vectors of the current block (including, but not limited to, affine mode control point motion vectors, affine, planar, sub-block motion vectors in ATMVP mode, temporal motion vectors, etc.). For example, the values of the motion vectors are constrained to a predefined range according to their representation bits. When the representation bits of the motion vector are bitDepth, the range is -2^(bitDepth-1) to 2^(bitDepth-1)-1, where "^" means exponentiation. For example, when bitDepth is set equal to 16, the range is -32768 to 32767. When bitDepth is set equal to 18, the range is -131072 to 131071. Here, two methods for constraining the motion vectors are provided.

[0168] Method 1: Remove the overflow MSB (Most Significant Bit) by a flow operation. ux=(mvx+2 bitDepth )%2 bitDepth (1) mvx=(ux>=2 bitDepth-1 )?(ux-2 bitDepth ):ux (2) uy=(mvy+2 bitDepth )%2 bitDepth (3) mvy=(uy>=2 bitDepth-1 )?(uy-2 bitDepth ):uy (4) For example, when the value of mvx is -32769, after applying equations (1) and (2), the resulting value is 32767. In a computer system, decimal numbers are stored as two's complements. The two's complement of -32769 is 1,0111,1111,1111,1111 (17 bits), and then the MSB is discarded, so the resulting two's complement is 0111,1111,1111,1111 (decimal 32767), which is the same as the output by applying equations (1) and (2). ux=(mvpx+mvdx+2 bitDepth )%2 bitDepth (5) mvx=(ux>=2bitDepth-1 )?(ux - 2 bitDepth ): ux (6) uy = (mvpy + mvdy + 2 bitDepth ) % 2 bitDepth (7) mvy = (uy >= 2 bitDepth-1 )?(uy - 2 bitDepth ): uy (8) As shown in equations (5) to (8), an operation can be applied between the sum of mvp and mvd.

[0169] Method 2: Remove the overflow MSB by clipping the value. vx = Clip3(-2 bitDepth-1 , 2 bitDepth-1 -1, vx) vy = Clip3(-2 bitDepth-1 , 2 bitDepth-1 -1, vy) Here, the definition of the function Clip3 is as follows.

[0170]

Number

[0171] The operation "?" is usually used as a shorthand for expressing an "if... else" condition. For example, "X < K? X = 1: X = 0" can be interpreted as "if X is less than K, X is set equal to 1, and otherwise (if X is not less than K), X is set equal to 0".

[0172] FIG. 4 is a schematic diagram of a video coding device 400 according to an embodiment of the present disclosure. The video coding device 400 is suitable for implementing the disclosed embodiments as described herein. In one embodiment, the video coding device 400 can be a decoder such as the video decoder 30 of FIG. 1A, or an encoder such as the video encoder 20 of FIG. 1A.

[0173] The video coding device 400 comprises an inlet port 410 (or input port 410) and a receiver unit (Rx) 420 for receiving data, a processor, logic unit, or central processing unit (CPU) 430 for processing data, a transmitter unit (Tx) 440 and an exit port 450 (or output port 450) for transmitting data, and memory 460 for storing data. The video coding device 400 may also include optical-to-electrical (OE) and electrical-to-optical (EO) components connected to the inlet port 410, receiver unit 420, transmitter unit 440, and exit port 450 for the entry and exit of optical or electrical signals.

[0174] The processor 430 is implemented by hardware and software. The processor 430 may be implemented as one or more CPU chips, cores (e.g., as a multi-core processor), FPGAs, ASICs, and DSPs. The processor 430 communicates with an inlet port 410, a receiver unit 420, a transmitter unit 440, an exit port 450, and memory 460. The processor 430 includes a coding module 470. The coding module 470 implements the disclosed embodiments described above. For example, the coding module 470 performs, processes, prepares, or provides various coding operations. Thus, the inclusion of the coding module 470 significantly improves the functionality of the video coding device 400, resulting in the transformation of the video coding device 400 to different states. Alternatively, the coding module 470 is implemented as instructions stored in memory 460 and executed by the processor 430.

[0175] Memory 460 may include one or more disks, tape drives, and solid-state drives, and may be used as an overflow data storage device to store the program when such a program is selected for execution, and to store instructions and data read during program execution. Memory 460 may be, for example, volatile and / or non-volatile, and may be read-only memory (ROM), random access memory (RAM), tri-level associative memory (TCAM), and / or static random access memory (SRAM).

[0176] Figure 5 is a simplified block diagram of the apparatus 500, which can be used as either or both of the source device 12 and destination device 14 from Figure 1, according to an exemplary embodiment.

[0177] The processor 502 of the device 500 may be a central processing unit. Alternatively, the processor 502 may be any other type of device, or multiple devices, capable of manipulating or processing information that is currently existing or will be developed later. The disclosed implementation may be practiced as a single processor, e.g., processor 502, as shown, but advantages in terms of speed and efficiency may be obtained by using more than one processor.

[0178] The memory 504 of the device 500 may be a read-only memory (ROM) device or a random access memory (RAM) device in a given implementation. Any other suitable type of storage device may be used as memory 504. Memory 504 may include code and data 506 accessed by the processor 502 using the bus 512. Memory 504 may further include an operating system 508 and an application program 510, the application program 510 including at least one program that allows the processor 502 to perform the method described herein. For example, the application program 510 may include applications 1 through N, which further include video coding applications that perform the method described herein.

[0179] The device 500 may also include one or more output devices, such as a display 518. In one example, the display 518 may be a touch-sensitive display combining a display with a touch-sensing element capable of sensing touch input. The display 518 may be connected to the processor 502 via a bus 512.

[0180] Although illustrated here as a single bus, the bus 512 of device 500 may consist of multiple buses. Furthermore, the secondary storage 514 may be directly connected to other components of device 500, or accessed via a network, and may include a single integrated unit such as a memory card or multiple units such as multiple memory cards. Thus, device 500 can be implemented in a wide variety of configurations.

[0181] In an example of constructing a merge candidate list according to ITU-T H.265, the merge candidate list is constructed based on the following candidates: 1. Up to 4 candidate spaces derived from 5 adjacent spatial blocks. 2. One time candidate derived from two temporally identical blocks. 3. Additional candidates, including synthesized bidirectional prediction candidates. 4. Candidate zero motion vectors

[0182] Spatial candidates First, motion information of spatially adjacent blocks is added to the merge candidate list as motion information candidates (in some examples, the merge candidate list may be an empty list before the first motion vector is added to the merge candidate list). The adjacent blocks that are considered for insertion into the merge list are shown in Figure 6b. In the interprediction block merging, up to four candidates are inserted into the merge list by sequentially checking A1, B1, B0, A0, and B2 in this order.

[0183] Motion information may include all motion data, including information on whether one or two reference picture lists are used, as well as the reference index and motion vector for each reference picture list.

[0184] In one example, after checking whether an adjacent block is available and contains motion information, several additional redundancy checks are performed before adopting all motion data from the adjacent block as a candidate for motion information. These redundancy checks can be divided into two categories for two different purposes. Category 1: Ensure that candidates with redundant motion data are not included in the list. Category 2: Prevents the merging of two divisions that could be expressed by other means, which would result in redundant syntax.

[0185] Time candidates Figure 6a illustrates the coordinates of the block from which time-motion information candidates are extracted. Co-located blocks are blocks that have the same x,y coordinates as the current block but are located on a different picture (one of the reference pictures). If the list is not filled, time-motion information candidates are added to the merge list (in some examples, the merge list is not filled when the number of candidates in the merge list is less than a threshold, for example, the threshold could be 4, 5, or 6).

[0186] Generated candidates If the merge list is not yet filled after the insertion of spatial and temporal motion information candidates, the generated candidates are added to fill the list. The list size is indicated in the sequence parameter set and is constant throughout the coded video sequence.

[0187] Two-way prediction A spatial mode with interpretation is called "bidirectional prediction," and two motion vectors are used to predict a block. The motion vectors can point to the same or different reference pictures, which may be indicated by a reference picture list ID and a reference picture index. For example, the first motion vector may point to a first picture in reference picture list L0, and the second motion vector may point to a first picture in reference picture list L1. Two reference picture lists (e.g., L0 and L1) may be maintained, and the picture pointed to by the first motion vector is selected from list L0, and the picture pointed to by the second motion vector is selected from list L1.

[0188] In one example, if motion information indicates bidirectional prediction, the motion information includes two parts. • L0 section: Motion vectors and reference picture indexes pointing to entries in the reference picture list L0. • L1 section: Motion vector and reference picture index pointing to an entry in the reference picture list L1.

[0189] Picture Order Count (POC): A variable associated with each picture, uniquely identifying the relevant picture among all pictures in the CVS (Coded Video Sequence). When the relevant picture is to be output from the decoded picture buffer, it indicates the location of the relevant picture in the output order relative to the output order locations of other pictures in the same CVS that will be retrieved from the decoded picture buffer.

[0190] Each of the reference picture lists L0 and L1 may contain one or more reference pictures, each identified using a POC. The association between each reference index and a POC value may be signaled in the bitstream. For example, the L0 and L1 reference picture lists may contain the following reference pictures:

[0191] [Table 3]

[0192] In the example above, the first entry in the reference picture list L1 (indicated by reference index 0) is a reference picture with a POC value of 13. The second entry in the reference picture list L1 (indicated by reference index 1) is a reference picture with a POC value of 14.

[0193] The merge list construction process in ITU-T H.265 and VVC outputs a list of motion information candidates. The VVC merge list construction process is described in section "8.3.2.2 Derivation process for luma motion vectors for merge mode" of the document JVET-L1001_v2 Versatile Video Coding (Draft 3), which is publicly available at http: / / phenix.it-sudparis.eu / jvet / . The term motion information refers to the motion data required to perform a motion-compensated prediction process. Motion information typically refers to the following information: • Whether the block applies unidirectional or bidirectional prediction. • The IDs of the reference picture used in the prediction (two IDs if the block applies bidirectional prediction) • Motion vectors (two motion vectors if the block is predicted to move in both directions) • Additional information

[0194] In VVC and H.265, the list of candidates that is the output of the merge list construction contains motion information for N candidates. The number N is typically contained in the bitstream and can be a positive integer such as 5 or 6. The candidates included in the constructed merge list may contain either unidirectional or bidirectional prediction information. This means that the candidates selected from the merge list may represent bidirectional prediction operations.

[0195] Triangulation prediction mode The concept of triangular prediction mode introduces a new triangular division for motion-compensated predictions. As shown in Figure 7, two triangular prediction units are used for the CU, either diagonally or inversely diagonally. Each triangular prediction unit for the CU is interpreted using a unidirectional prediction motion vector and reference frame index derived from a unidirectional prediction candidate list. After the samples associated with each triangular prediction unit have been predicted, for example, by motion-compensated prediction or intrapicture prediction, an adaptive weighting process is performed on the ends of the diagonal. Then, transformation and quantization processes are applied to the entire CU. Note that this mode applies only to skip mode and merge mode.

[0196] In triangular prediction mode, a block is divided into two triangular segments (as shown in Figure 7), and each segment can be predicted using one motion vector. The motion vector used to predict one triangular segment (denoted by PU1) may be different from the motion vector used to predict the other triangular segment (denoted by PU2). Note that in some examples, each segment may be predicted using only a single motion vector (unidirectional prediction) to reduce the complexity of performing triangular prediction mode. In other words, PU1 and PU2 may not be predicted using bidirectional prediction involving two motion vectors.

[0197] Subblock prediction mode The triangular prediction mode is a special case of subblock prediction where a block is divided into two blocks. In the example above, the directions of the two block divisions are illustrated (45-degree and 135-degree divisions). Other division angles and division ratios are also possible for subblock prediction (see example in Figure 8).

[0198] In some examples, a block is divided into two subblocks, and each part (subblock) is predicted using unidirectional prediction. In one example, following the use of subblock partitioning mode, the following steps are applied to obtain prediction samples for the block. Step 1: Divide the coding block into two subblocks according to the geometric model. This model may result in the division of the block by a separating line (e.g., a straight line), as illustrated in Figures 9-12. It should be understood that the result of Step 1 may be merely conceptual. For example, after Step 1 there is no clear concept of "subblock 1" or "subblock 2," and parameters are generated or obtained by referring to the separating line. The samples belong to different subblocks according to the parameters that refer to the separating line. In this step, according to the geometric model, the samples in the coding block are considered to be located within two subblocks. Subblock A or subblock B contains some (but not all) of the samples in the current coding block. Subblock A or subblock B may be represented according to the sign of sample_dist for each sample. sample_dist may be obtained according to the examples and embodiments in other paragraphs. Step 2: Obtain the first prediction mode for the first subblock and the second prediction mode for the second subblock. In some examples, the first prediction mode is not identical to the second prediction mode. In some examples, the prediction mode (either the first or second prediction mode) may be an interprediction mode, and the information for the interprediction mode may include a reference picture index and a motion vector. In other examples, the prediction mode may be an intraprediction mode, and the information for the intraprediction mode may include an intraprediction mode index. Step 3: Obtain the first and second predicted values ​​using the first and second prediction modes, respectively. Step 4: Obtain the composite value of the predicted sample according to the splitting disclosed in Step 1, and according to the synthesis of the first and second predicted values. The process of synthesizing samples from the first and second predictions to obtain the composite value of the predicted sample of the block may include filtering operations, masking operations, or sample duplication.

[0199] In one example, in step 1, the coding block is divided into two subblocks in various ways. Figure 9 shows an example of the division of a coding block, where a separator line 1250 divides the block into two subblocks. Two parameters are signaled to describe the line 1250, one parameter being the angle alpha 1210 and the other being the distance dist 1230.

[0200] In some embodiments, the angle, as shown in Figure 9, is measured between the x-axis and the separator line, while the distance is measured by the length of a vector perpendicular to the separator line and passing through the center of the current block.

[0201] In another example, Figure 10 shows an alternative way of representing the separation line, where the examples of angles and distances differ from those shown in Figure 9.

[0202] In some examples, in step 4, the split disclosed in step 1 is used as a composite of the two aforementioned predictions to obtain the final prediction. In some examples, a blending operation is applied in step 4 to remove any artifacts (sharp or jagged appearance along the separation line). The blending operation can be described as a filtering operation along the separation line.

[0203] In one example, on the encoder side, the separation line (parameters defining the line, e.g., angle and distance) is determined based on a rate-distortion-based cost function. The determined line parameters are encoded into a bitstream. On the decoder side, the line parameters are decoded (obtained) according to the bitstream. On the decoder side, the line parameters are used to combine the first and second predicted values ​​in order to obtain the final predicted value. Step 1 does not require the concept of two subcoding blocks; the decoder can use coding blocks and line parameters to represent the concept of two subcoding blocks.

[0204] Since there are many possibilities in splitting a coding block into two sub-blocks, the signaling (coding) of the split requires too many bits. Furthermore, since angle and distance values ​​can have many different values, too much side information must be signaled in the bitstream.

[0205] Embodiments of the present invention relate to the signaling of segmentation parameters. Efficient coding is achieved by the following features: • Parameters for block division (e.g., angle and distance) are stored in a predetermined table so that the actual values ​​of angle and distance do not need to be transmitted. • The table contains a partitioning parameter that is more likely to include a sufficient number of variants. • Indexes on a table are encoded into (or retrieved from) a bitstream.

[0206] Embodiments of the present invention achieve a method for joint coding parameters that describe the division of coding blocks into subblocks.

[0207] Embodiment 1 (from the perspective of the decoder): In this embodiment, the following steps are applied to obtain a composite prediction of the coding block sample.

[0208] Step 1: Obtain the indicator value of the current decrypted block according to the bitstream.

[0209] In one embodiment, the indicator value is used to specify the partition shape in geometric partition merge mode. For example, the indicator may be merge_gpm_partition_idx[x0][y0], where merge_gpm_partition_idx[x0][y0] specifies the partition shape in geometric partition merge mode. The array indices x0, y0 specify the relative position (x0, y0) of the top-left luma sample of the coding block being considered with respect to the top-left luma sample of the picture.

[0210] Normally, the value of merge_gpm_partition_idx[x0][y0] is decoded from the bitstream. In one example, the range of values ​​for merge_gpm_partition_idx[][] is from 0 to 63, including 0 and 63. In another example, the decoded process for merge_gpm_partition_idx[][] is "bypassed".

[0211] When merge_gpm_partition_idx[x0][y0] does not exist, it is inferred to be equal to 0.

[0212] Step 2: Obtain the values ​​of the first and second parameters based on the indicator values ​​and a predetermined lookup table. (Parameter 1, Parameter 2) = lookupTable(indicator)

[0213] In one embodiment, the partition angle variable angleIdx (parameter 1) and the distance variable distanceIdx (parameter 2) of the geometric partition mode are set according to the values ​​of merge_gpm_partition_idx[xCb][yCb] (indicators) as specified in the following table. In implementations, it can be understood that this relationship may be implemented according to table 1 (Table 4) or according to a function.

[0214] [Table 4]

[0215] Step 3: For the samples in the current decoded block, the sample distance (sample_dist) is calculated according to the values ​​of the first and second parameters.

[0216] In one embodiment, step 3 includes the following steps.

[0217] Step 3.1: Obtain the index value of the angle parameter (alphaN or angleIdx) for the current block, the width value (W) of the current block, and the height value (H) of the current block. W and H are the width and height of the current block in units of the number of samples. For example, a coding block with both width and height equal to 8 is a square block containing 64 samples. In another example, W and H are the width and height of the current block in units of the number of samples. The index value of the angle parameter is obtained as described above and can be found in Table 1 (Table 4).

[0218] Step 3.2: Obtain the value of the ratio whRatio according to the values ​​of W and H, where the value of whRatio represents the ratio of the width to the height of the current coding block.

[0219] In some cases, whRatio=H / W or whRatio=W / H.

[0220] In another example, the two variables nCbW and nCbH specify the width and height of the current coding block, and the variable cIdx specifies the color component index.

[0221] The variables nW, nH, and whRatio are derived as follows: nW=(cIdx==0)?nCbW:nCbW*SubWidthC nH=(cIdx==0)?nCbH:nCbH*SubHeightC whRatio=nH / nW

[0222] In one example, the variables SubWidthC and SubHeightC are specified in Table 2 (Table 5) depending on the chroma format sampling structure, which is specified through chroma_format_idc (chroma_format_idc specifies chroma sampling for luma sampling) and separate_colour_plane_flag (separate_colour_plane_flag equal to 1 specifies that the three color components of the 4:4:4 chroma format are coded separately. separate_colour_plane_flag equal to 0 specifies that the color components are not coded separately. When separate_colour_plane_flag does not exist, it is assumed to be equal to 0. When separate_colour_plane_flag is equal to 1, the coded picture consists of three separate components, each consisting of a coded sample of one color plane (Y, Cb, or Cr) and using the monochrome coding syntax. In this case, each color plane is associated with a specific colour_plane_id value). Other values ​​for chroma_format_idc, SubWidthC, and SubHeightC may be specified in the future by ITU-T|ISO / IEC.

[0223] [Table 5]

[0224] In monochromatic sampling, there is only one sample array, which is usually considered a luma array.

[0225] In 4:2:0 sampling, each of the two chroma arrays has half the height and half the width of the chroma array.

[0226] In 4:2:2 sampling, each of the two chroma arrays has the same height as the luma array and half the width of the luma array.

[0227] In 4:4:4 sampling, the following applies depending on the value of separate_colour_plane_flag: - If separate_colour_plane_flag is equal to 0, each of the two chroma arrays has the same height and width as the luma array. - Otherwise (separate_colour_plane_flag is equal to 1), the three color planes are processed separately as monochrome sampled pictures.

[0228] Step 3.3: Obtain the shiftHor value according to the lookup table, where the values ​​of alpha and whRatio, in some examples, alpha(alphaN or angleIdx) and whRatio, are used as index values ​​in the lookup table. The shiftHor value may also be obtained according to a function, where the values ​​of alpha(alphaN or angleIdx) and whRatio are inputs to the function, and the shiftHor value is the output of the function. In some examples, the result of the function is similar to or identical to the lookup table. In some examples, the shiftHor value represents the quantization step size for the sample distance calculation process.

[0229] In another example, the function can be represented by shiftHor=(angleIdx%16==8||(angleIdx%16!=0&&whRatio>0))?0:1.

[0230] Step 3.4: The value of sample_dist is calculated according to the shiftHor value.

[0231] In one example, to calculate the sample distance (sample_dist), first, the variables offsetX and offsetY are derived as follows.

[0232] When shiftHor is equal to 0, the following holds. offsetX = (-nW) >> 1 offsetY = ((-nH) >> 1) + (angleIdx < 16? (distanceIdx * nH) >> 3 : -((distanceIdx * nH) >> 3)) Otherwise (when shiftHor is equal to 1), the following holds. offsetX = ((-nW) >> 1) + (angleIdx < 16? (distanceIdx * nW) >> 3 : -((distanceIdx * nW) >> 3)) offsetY = (-nH) >> 1 - The variables xL and yL are derived as follows. xL = (cIdx == 0)? x : x * SubWidthC yL = (cIdx == 0)? y : y * SubHeightC sample_dist = (((xL + offsetX) << 1) + 1) * disLut[displacementX] + (((yL + offsetY) << 1) + 1)) * disLut[displacementY]

[0233] The variables displacementX and displacementY are derived as follows. hwRatio=cbHeight / cbWidth Purchase X = angleIdx displacementY=(angleIdx+8)%32

[0234] The array disLut specified in Table 3 (Table 6) is as follows:

[0235] [Table 6]

[0236] In another embodiment, in one example, step 3 of the above embodiment includes the following steps.

[0237] Step 3.1: Obtain the angle parameter value (alphaN or angleIdx), distance index value (distanceIdx), current block width value (W), and current block height value (H) for the current block. W and H are the width and height of the current block in units of the number of samples. For example, a coding block with both width and height equal to 8 is a square block containing 64 samples. In another example, W and H are the width and height of the current block in units of the number of samples. The index value of the angle parameter is obtained as described above and can be found in Table 1 (Table 4).

[0238] Step 3.2: Obtain the value of the ratio whRatio according to the values ​​of W and H, where the value of whRatio represents the ratio of the width to the height of the current coding block.

[0239] In some cases, whRatio=H / W or whRatio=W / H.

[0240] In another example, the two variables nCbW and nCbH specify the width and height of the current coding block, and the variable cIdx specifies the color component index.

[0241] The variables nW, nH, and whRatio are derived as follows: nW=(cIdx==0)?nCbW:nCbW*SubWidthC nH=(cIdx==0)?nCbH:nCbH*SubHeightC whRatio=nH / nW

[0242] Step 3.3: Obtain the shiftHor value according to the lookup table, where the values ​​of alpha and whRatio, in some examples, the values ​​of alphaN and whRatio, are used as index values ​​in the lookup table. In some examples, the shiftHor value represents the quantization step size for the sample distance calculation process. The shiftHor value may also be obtained according to a function, where the values ​​of alpha(alphaN or angleIdx) and whRatio are inputs to the function, and the shiftHor value is the output of the function. In some examples, the result of the function is similar to or the same as the lookup table. In some examples, the shiftHor value represents the quantization step size for the sample distance calculation process.

[0243] In another example, the function can be represented by shiftHor=(angleIdx%16==8||(angleIdx%16!=0&&whRatio>0))?0:1.

[0244] Step 3.4: The value of sample_dist is calculated according to the shiftHor value, the distanceIdx value, the angle value (alphaN or angleIdx), the W value, and the H value.

[0245] In one example, to calculate the sample distance (sample_dist), the variables offsetX and offsetY are first derived as follows:

[0246] If shiftHor is equal to 0, the following is true: offsetX=(-nW)>>1 offsetY = ((-nH) >> 1) + (angleIdx < 16? (distanceIdx * nH) >> 3 : -((distanceIdx * nH) >> 3)) In other cases (where shiftHor is equal to 1), the following applies. offsetX = ((-nW) >> 1) + (angleIdx < 16? (distanceIdx * nW) >> 3 : -((distanceIdx * nW) >> 3)) offsetY = (-nH) >> 1 - Variables xL and yL are derived as follows. xL = (cIdx == 0)? x : x * SubWidthC yL = (cIdx == 0)? y : y * SubHeightC sample_dist = (((xL + offsetX) << 1) + 1) * disLut[displacementX] + (((yL + offsetY) << 1) + 1)) * disLut[displacementY]

[0247] Variables displacementX and displacementY are derived as follows. hwRatio = cbHeight / cbWidth displacementX = angleIdx displacementY = (angleIdx + 8) % 32

[0248] The array disLut is specified in table 3 (Table 6) above.

[0249] Step 4: To calculate the weight coefficients, the calculated sample_dist is used, and the weight coefficients are used for the combination of the first prediction value and the second prediction value corresponding to the said sample. In one example, the weight coefficients are denoted as sampleWeight1 and sampleWeight2, referring to the weight corresponding to the first prediction value and the weight corresponding to the second prediction value.

[0250] In one example, the weight coefficients are calculated according to the following function: weightIdxL=partFlip?32+sample_dist:32-sample_dist wValue=Clip3(0,8,(weightIdxL+4)>>3)

[0251] In this example, wValue is sampleWeight1 and 8-wValue is sampleWeight2. The variable partFlip is determined according to the value of angleIdx. In one example, partFlip = (angleIdx>=13&&angleIdx<=27)?0:1 or partFlip = (angleIdx>=13&&angleIdx<=27)?1:0.

[0252] Step 5: The composite value of the predicted sample at sample coordinates (x,y) is calculated according to the first predicted value at coordinates (x,y), the second predicted value at coordinates (x,y), sampleWeight1, and sampleWeight2.

[0253] In one example, the predicted sample values ​​are derived as follows: pbSamples[x][y]=Clip3(0,(1< <BitDepth)-1,(predSamplesLA[x][y]*wValue+ predSamplesLB[x][y]*(8-wValue)+offset1)>>shift)

[0254] bitDepth represents the sample bit depth, and the variable shift1 is obtained according to bitDepth, where in one example shift1 = Max(5, 17 - BitDepth). The variable offset1 is obtained according to shift1, where in one example offset1 = 1 << (shift1 - 1), and predSamplesLA and predSamplesLB are two (nCbW) x (nCbH) arrays.

[0255] Embodiment 2 (from the perspective of the encoder) In this embodiment, the following steps are applied.

[0256] Step 0: Select the values ​​for the first parameter and the second parameter. In one example, the values ​​for the first and second parameters are obtained according to the rate-distortion scale.

[0257] Step 1: According to the value of the first parameter, the value of the second parameter, and according to the lookup table, (Parameter 1, Parameter 2) = lookupTable(index) The index value is obtained so that it becomes as follows.

[0258] Step 2: Encode the index value into a bitstream.

[0259] Optionally, the following steps are used in this embodiment.

[0260] Step 3: For the samples in the coding block, the sample distance (sample_dist) is calculated according to the values ​​of the first and second parameters.

[0261] Step 4: The calculated sample_dist is used to calculate weight coefficients for the combination of the first and second predicted values ​​corresponding to the sample. The weight coefficients are denoted as sampleWeight1 and sampleWeight2, and refer to the weight values ​​corresponding to the first and second predicted values, respectively.

[0262] Step 5: The combined predicted value for the sample at coordinate (x,y) is calculated according to the first predicted value at coordinate (x,y), the second predicted value at coordinate (x,y), sampleWeight1, and sampleWeight2. Details of each step on the encoder side correspond to the example above that described the decoder side.

[0263] In one example, since the lookup table of Embodiment 1 is the same as lookup table 2 of Embodiment 2, the encoder and decoder can obtain the same results.

[0264] In some examples, the sample distance may represent the horizontal distance from the sample to the separator line, or the vertical distance, or a combination of the horizontal and vertical distances (the separator line is used to divide a coding block into two subblocks). The sample is represented by coordinates (x,y) with respect to the top-left sample of the coding block. Sample coordinates and sample_dist are illustrated in Figures 11 and 12.

[0265] In one example, the first parameter represents a quantized angle value (angleIdx), and the second parameter represents a quantized distance value (distanceIdx). The two parameters describe the equation of a line. In one example, the distance 1230 may be obtained according to distanceIdx (the second parameter), and the angle alpha (1210) may be obtained according to angleIdx (the first parameter). The distance 1230 may be the distance to the center of the coding block, and the angle may be the angle between the separator line and a horizontal (or equivalently vertical) line passing through the center point of the coding block.

[0266] In one example, the lookup table is predetermined.

[0267] In some examples, there may be more than one predetermined lookup table. In one example, one lookup table for a coding block is used when the aspect ratio of the block (aspect ratio = W > H ? W / H : H / W) is equal to 1. In another example, if the aspect ratio is not equal to 1, one or more lookup tables (not identical to the aforementioned one) are used, where W and H are the width and height of the coding block.

[0268] In another example, there may be more than one predetermined lookup table, one of which is for a coding block, and the block's W*H is greater than a threshold. If the block's W*H is less than or equal to the threshold, one or more lookup tables (not identical to the one) are used for the block. For example, if the expected size of the current block is less than or equal to 32x32 luminance samples, the first lookup table is used. Otherwise, if the size of the current coding block is greater than 32x32 luminance samples, the second lookup table is used. The two lookup tables may contain different sets of parameters for various characteristics of different block sizes. For example, the second lookup table may contain more parameters than the first lookup table, providing greater partitioning flexibility for larger blocks.

[0269] In some examples, if there are more than one predetermined lookup table to use for a block, a function f(W,H) may be used to determine one lookup table for the block, where W and H are the width and height of the coding block.

[0270] In another example, the sample distance sample_dist is calculated according to the following formula: sample_dist=((x<<1)+1)*Dis[angleIdx1]+((y<<1)+1))*Dis[angleIdx2]-offset(distanceIdx) Here, The values ​​of angleIdx1 and angleIdx2 are derived / calculated from the bitstream or based on other information obtained from the bitstream, and angleIdx1 and angleIdx2 represent the quantized trigonometric parameters of the separation line, i.e., the first parameter of type cosine and the second parameter of type sine. In one example, x and y are the -x and -y coordinates of the sample with respect to the top-left sample of the coding block. • Dis[] is a lookup table. Dis[angleIdx1] describes the change in sample distance (sample_dist) for a unit increase (an increase of 1 value) in the -x coordinate of the sample. Dis[angleIdx2] describes the change in sample distance (sample_dist) for a unit increase (an increase of 1 value) in the -y coordinate of the sample. In one example, angleIdx1 is equal to angleIdx, and angleIdx2 is equal to (displacementX+8)%32. • offset(distanceIdx) is an offset value that is a function of the index value(distanceIdx), which is obtained from the bitstream or derived / calculated based on other information obtained from the bitstream.

[0271] Build a lookup table For example, the range of the value of the first parameter may be from 0 to 4K-1, where K is a non-negative integer. The range of the value of the second parameter may be from 0 to N. In this case, the rows of the lookup table may include all combinations of the first and second parameters, except for the following combinations. • First set of pairs: The value of the second parameter is equal to 0, and the value of the first parameter is equal to one of {0, K, 2K, 3K}. • The second set of pairs: The value of the second parameter is 0, and the value of the first parameter is equal to one of {K / 2, 3K / 2, 5K / 2, 7K / 2}. • Third set of parameters: The value of the second parameter is 0, and the value of the first parameter is 2K or greater.

[0272] In one example, the value of K is 8 or 4. The number K describes how many discrete angle values ​​are used to divide a 90-degree angle. In one example, if the number K is 8, the dividing line could have the following angle values: 0, 11.25, 22.5, 33.75, 45, ... degrees. In another example, if the value of K is 4, the angle values ​​are 0, 22.5, 45, 67.5, ... degrees.

[0273] In one example, the value of N is 4. In general, N can be an integer greater than 0.

[0274] In some embodiments, the second parameter describes the distance from the separator line to the center of the coding block. If the value of the second parameter is 0, this means that the separator line passes through the center of the coding block. If the value of the first parameter is equal to K, this means that the separator line is either horizontal or vertical (the angle value is equal to 0, 90, 270, or 360 degrees). The first set of pairs is removed from the lookup table because it creates a redundant partition, which can also be achieved by a binary partition of the coding block using a binary tree. A binary partition is a mechanism that divides a coding block into two coding blocks of equal size (and not into subblocks) along a vertical or horizontal axis. Thus, according to embodiments of the present invention, the first set of pairs is removed from the lookup table.

[0275] The second set of pairs is removed from the lookup table because it creates a partition that can be achieved by the triangular prediction mode. Therefore, removing the second set of pairs prevents the partitioning of redundant subblocks.

[0276] When the angle is equal to X or X+180, if the second parameter is equal to 0, the same category is created, so the third set of pairs is removed from the table.

[0277] In another example, the following pairs may be removed from the lookup table: The value of the second parameter is equal to N, and the value of the first parameter is equal to one of {K / 2, 3K / 2, 5K / 2, 7K / 2}. The value of the second parameter is equal to N, and the value of the first parameter is equal to one of {K / 2-1, 3K / 2-1, 5K / 2-1, 7K / 2-1, K / 2+1, 3K / 2+1, 5K / 2+1, 7K / 2+1}.

[0278] In some embodiments, the above pairs may be removed from the lookup table because they create two subblocks, one of which may be too small. When the value of the first parameter is equal to K / 2, 3k / 2, ..., this means that the angle of the separation line is 45 degrees, 135 degrees, 225 degrees, or 315 degrees. When the value of the second parameter is equal to N (which is the highest value), one of the subblocks contains a small portion of one corner of the coding block, which is too small for the splitting operation.

[0279] The order of pairs in a lookup table For example, if the second parameter is equal to N (where N is the highest possible value for the second parameter), then a pair (first parameter, second parameter) may be added to the end of the lookup table. In other words, if there are P pairs where the value of the second parameter is equal to N, then the last P entries in the lookup table may contain P pairs. For example, if N is equal to 4 and K is equal to 8, then there are P = 4K * 1 = 32 parameter pairs where the second parameter is equal to 4.

[0280] In another example, parameter pairs are ordered according to a heuristic process. This heuristic process can be determined from the training data, for example, by reflecting the occurrence or use of parameter pairs in the training dataset. As a result, parameter pairs that appear more frequently are placed at the beginning of the list, and those that appear less frequently are placed at the end.

[0281] When the second parameter is equal to N, one subblock is smaller than the other. The probability of selecting such a division is lower (compared to when the number of samples in each subblock is more similar). Furthermore, index values ​​(in the lookup table) with higher values ​​typically require more bits to encode into a bitstream. Thus, this increases compression efficiency, and the beginning of the table can be allocated to more likely pairs (e.g., pairs that result in a more favorable subblock division) because less likely pairs are included at the end of the table.

[0282] An example of an indicator (geo_partition_idx) and a lookup table are given below. The indicator has a value between 0 and 139, which is used to select the first and second parameters using the lookup table. An example of the lookup table is Table 4 (Table 7).

[0283] [Table 7A] [Table 7B] [Table 7C] [Table 7D]

[0284] Table 4 (Table 7) is merely an example to show possible combinations of the first and second parameters, and it should be understood that in other implementations, parts of Table 4 (Table 7) may be used as a lookup table to retrieve the first and second parameters.

[0285] Example 1. A coding method performed by a decoding device, Steps to obtain a bitstream, Steps include: obtaining the indicator value for the current block according to the bitstream, The steps include obtaining the value of a first parameter for the current block and the value of a second parameter for the current block according to the indicator value and a predetermined lookup table, The steps include obtaining the sample distance value for the sample located in the current block according to the values ​​of the first parameter and the second parameter, A method comprising the steps of obtaining a predicted value for a sample according to the value of the sample distance to the sample.

[0286] Example 2. The method of Example 1, where the first parameter represents the angle for the current block division.

[0287] Example 3. The method of Example 1 or 2, where the second parameter represents the distance for the current block division.

[0288] Example 4. A predetermined lookup table consists of a first parameter pair and a second parameter pair (in one example, each entry in the table is a unique pair of the first and second parameters), and the following pairs are in the predetermined lookup table: One of the methods in Examples 1 to 3, where the first parameter represents an angle corresponding to one of the angles 0, 45, 90, 135, 180, 225, 270, or 315 degrees, and the corresponding second parameter of the pair does not represent a distance of 0 samples from the center of the current block.

[0289] Example 5. A predetermined lookup table consists of a first parameter pair and a second parameter pair (in one example, each entry in the table is a unique pair of the first and second parameters), and the following pairs are in the predetermined lookup table: One of the methods in Examples 1 through 4, where the first parameter represents an angle that does not correspond to one of the angles 0, 45, 90, 135, 180, 225, 270, or 315 degrees, and the corresponding second parameter of the pair represents a distance of 0 samples from the center of the current block.

[0290] Example 6. A predetermined lookup table consists of a first parameter pair and a second parameter pair (in one example, each entry in the table is a unique pair of the first and second parameters), and the following pairs are not included in the predetermined lookup table: One of the methods from Examples 1 to 5, where the first parameter represents an angle corresponding to one of the angles 0, 45, 90, 135, 180, 225, 270, or 315 degrees, and the corresponding second parameter of the pair represents a distance of 0 samples from the center of the current block.

[0291] Example 7. A decoder (30) comprising a processing circuit for performing a method according to any one of Examples 1 to 6.

[0292] Example 8. A computer program containing program code to perform a method according to any one of Examples 1 through 6.

[0293] Example 9. Decoder, One or more processors, A decoder comprising a non-temporary computer-readable storage medium connected to a processor and storing a program for execution by the processor, wherein the decoder is configured to perform, when the program is executed by the processor, in a manner according to any one of Examples 1 to 6.

[0294] The following describes the application of the encoding and decoding methods shown in the embodiments described above, and the systems using them.

[0295] Figure 13 is a block diagram showing a content supply system 3100 for realizing a content distribution service. This content supply system 3100 includes a capture device 3102, a terminal device 3106, and optionally a display 3126. The capture device 3102 communicates with the terminal device 3106 via a communication link 3104. The communication link may include the communication channel 13 described above. The communication link 3104 may include, but is not limited to, Wi-Fi, Ethernet®, cable, wireless (3G / 4G / 5G), USB, or any combination thereof.

[0296] The capture device 3102 may generate data and encode it using an encoding method as shown in the above embodiment. Alternatively, the capture device 3102 may deliver the data to a streaming server (not shown in the figure), which encodes the data and transmits the encoded data to the terminal device 3106. The capture device 3102 includes, but is not limited to, a camera, a smartphone or tablet, a computer or laptop, a video conferencing system, a PDA, an in-vehicle device, or any combination thereof. For example, the capture device 3102 may include a source device 12 as described above. When the data includes video, the video encoder 20 included in the capture device 3102 may actually perform the video encoding process. When the data includes audio (i.e., voice), the audio encoder included in the capture device 3102 may actually perform the audio encoding process. In some practical scenarios, the capture device 3102 delivers the encoded video and audio data by multiplexing them together. In other practical scenarios, for example in a video conferencing system, the encoded audio data and encoded video data are not multiplexed. The capture device 3102 distributes the encoded audio data and encoded video data separately to the terminal device 3106.

[0297] In the content supply system 3100, the terminal device 310 receives and reproduces the encoded data. The terminal device 3106 may be a device capable of receiving and restoring data, such as a smartphone or tablet 3108, a computer or laptop 3110, a network video recorder (NVR) / digital video recorder (DVR) 3112, a TV 3114, a set-top box (STB) 3116, a video conferencing system 3118, a video surveillance system 3120, a personal digital assistant (PDA) 3122, an in-vehicle device 312, or any combination thereof, that is capable of decoding the encoded data described above. For example, the terminal device 3106 may include the destination device 14 as described above. When the encoded data includes video, the video decoder 30 included in the terminal device is preferred for performing video decoding. When the encoded data includes audio, the audio decoder included in the terminal device is preferred for performing audio decoding.

[0298] In terminal devices with displays, such as a smartphone or tablet 3108, a computer or laptop 3110, a network video recorder (NVR) / digital video recorder (DVR) 3112, a TV 3114, a personal digital assistant (PDA) 3122, or an in-vehicle device 3124, the terminal device can feed the decoded data to the display. In terminal devices without displays, such as an STB 3116, a video conferencing system 3118, or a video surveillance system 3120, an external display 3126 is connected to them to receive and display the decoded data.

[0299] When each device in this system performs encoding or decoding, a picture encoding device or a picture decoding device, as shown in the embodiments described above, may be used.

[0300] Figure 14 shows the structure of an example terminal device 3106. After the terminal device 3106 receives the stream from the capture device 3102, the protocol progression unit 3202 analyzes the transmission protocol of the stream. The protocol may include, but is not limited to, Real-Time Streaming Protocol (RTSP), Hypertext Transfer Protocol (HTTP), HTTP Live Streaming Protocol (HLS), MPEG-DASH, Real-Time Transport Protocol (RTP), Real-Time Messaging Protocol (RTMP), or any combination thereof.

[0301] After the protocol processing unit 3202 processes the stream, a stream file is generated. The file is output to the demultiplexing unit 3204. The demultiplexing unit 3204 can separate the multiplexed data into encoded audio data and encoded video data. As described above, in some real-world scenarios, for example in a video conferencing system, the encoded audio data and encoded video data are not multiplexed. In this situation, the encoded data is sent to the video decoder 3206 and audio decoder 3208 without passing through the demultiplexing unit 3204.

[0302] Through demultiplexing, video element streams (ES), audio ES, and optionally subtitles are generated. A video decoder 3206, including a video decoder 30 as described in the above embodiments, decodes the video ES by the decoding method shown in the above embodiments to generate video frames and feeds this data to the synchronization unit 3212. An audio decoder 3208 decodes the audio ES to generate audio frames and feeds this data to the synchronization unit 3212. Alternatively, video frames may be stored in a buffer (not shown in Figure 14) before feeding their data to the synchronization unit 3212. Similarly, audio frames may be stored in a buffer (not shown in Figure 14) before feeding them to the synchronization unit 3212.

[0303] The synchronization unit 3212 synchronizes video and audio frames and supplies video / audio to the video / audio display 3214. For example, the synchronization unit 3212 synchronizes the presentation of video and audio information. Information can be coded in syntax using timestamps related to the presentation of coded audio and visual data, as well as timestamps related to the delivery of the data stream itself.

[0304] If subtitles are included in the stream, the subtitle decoder 3210 decodes the subtitles and synchronizes them with the video and audio frames to supply the video / audio / subtitles to the video / audio / subtitle display 3216.

[0305] The present invention is not limited to the system described above, and either the picture encoding / decoding device or the picture decoding device in the above embodiments may be incorporated into other systems, such as vehicle systems.

[0306] Mathematical operators The mathematical operators used in this application are similar to those used in the C programming language. However, the results of integer division and arithmetic shift operations are more precisely defined, and additional operations such as exponentiation and real-valued division are defined. The numbering and counting conventions generally start from 0, for example, “first” is equivalent to the 0th, “second” is equivalent to the 1st, and so on.

[0307] Arithmetic operators The following arithmetic operators are defined as follows: + Addition - Subtraction (as an operator for two independent variables) or negation (as a unary prefix operator) * Multiplication including matrix multiplication x yExponentiation. Specifies x raised to the power of y. Such notation is used for superscripts that are not intended to be interpreted as exponentiation in other contexts. This is integer division where the decimal part of the result is truncated. For example, 7 / 4 and -7 / -4 are truncated to 1, and -7 / 4 and 7 / -4 are truncated to -1. The division symbol (÷) is used to indicate division in mathematical expressions where rounding or truncation is not intended.

number

number

[0308] Logical operators The following logical operators are defined as follows: x&&y: Boolean logic "product" of x and y Boolean logic "sum" of x||yx and y ! Boolean logic "negation" If x?y:zx is not true or equal to 0, it evaluates to the value y; otherwise, it evaluates to the value z.

[0309] Relational operators The following relational operators are defined as follows: > larger >= Above < Smaller <= Below == equal != Not equal

[0310] When a relational operator is applied to a syntactic element or variable to which the value "na" (not applicable) is assigned, the value "na" is treated as a distinct value for that syntactic element or variable. The value "na" is considered not equal to any other value.

[0311] Bitwise operators The following bitwise operators are defined as follows. & Bitwise "product". When operating on an integer independent variable, it operates on the two's complement representation of the integer value. When operating on a binary independent variable that contains fewer bits than another independent variable, the shorter independent variable is extended by adding higher-order bits equal to 0. | Bitwise "sum". When operating on an integer independent variable, it operates on the two's complement representation of the integer value. When operating on a binary independent variable that contains fewer bits than another independent variable, the shorter independent variable is extended by adding higher-order bits equal to 0. ^ Bitwise "exclusive OR". When operating on an integer independent variable, it operates on the two's complement representation of the integer value. When operating on a binary independent variable that contains fewer bits than another independent variable, the shorter independent variable is extended by adding higher-order bits equal to 0. x>>y Arithmetic right shift of y bits in binary of the two's complement integer representation of x. This function is defined only for non-negative integer values of y. The bit shifted into the most significant bit (MSB) as a result of the right shift has a value equal to the MSB of x before the shift operation. x<<y Arithmetic left shift of y bits in binary of the two's complement integer representation of x. This function is defined only for non-negative integer values of y. The bit shifted into the least significant bit (LSB) as a result of the left shift has a value equal to 0.

[0312] Assignment operators The following arithmetic operators are defined as follows. = Assignment operator The increment operator ++, i.e., x++, is equivalent to x = x + 1. When used in array indices, it evaluates to the value of the variable before the increment operation. -- Decrement, i.e., x -- is equivalent to x = x - 1. When used in array indices, it evaluates to the value of the variable before the decrement operation. += increments by the specified amount; that is, x+=3 is equivalent to x=x+3, and x+=(-3) is equivalent to x=x+(-3). -= Decrements by the specified amount; that is, x-=3 is equivalent to x=x-3, and x-=(-3) is equivalent to x=x-(-3).

[0313] Range notation The following notation is used to specify a range of values. x = y . . zx takes integer values ​​from y to z, including both endpoints, where x, y, and z are integers, and z is greater than y.

[0314] mathematical function The following mathematical function is defined.

number

number

number

number

number

number

number

number

[0315] Order of operations When the order of precedence in an expression is not explicitly indicated by the use of parentheses, the following rules apply: - Higher-priority operations are evaluated before any lower-priority operations. - Operations with the same priority are evaluated sequentially from left to right.

[0316] The following table (Table 8) specifies the order of operations from highest to lowest, with higher positions in Table 8 indicating higher priority.

[0317] With respect to those operators, which are also used in the C programming language, the order of precedence used herein is the same as that used in the C programming language.

[0318] [Table 8]

[0319] Textual explanation of logical operations In text, logical operation statements are mathematically described in the following format. if (condition 0) Statement 0 else if (condition 1) Statement 1 ... else / *Notes for reference regarding the remaining conditions* / statement n This can be described in the following way: The following applies... / ... - If condition 0, statement 0 - Instead, if condition 1 is true, then statement 1 - … - Otherwise (note for reference regarding the remaining conditions), statement n

[0320] Each of the "if..., otherwise..., otherwise..." statements in the text is introduced with "..., otherwise..." or "..., otherwise...", followed immediately by "..., otherwise...". The last condition in "..., otherwise..., otherwise..." is always "..., otherwise...". Interrupted "if..., otherwise..., otherwise..." statements can be identified by matching "..., otherwise..." or "..., otherwise..." with the final "..., otherwise...".

[0321] In text, logical operation statements are mathematically described in the following format. if(condition 0a &&condition 0b) Statement 0 else if(condition 1a||condition 1b) Statement 1 ... else statement n This can be described in the following way: The following applies... / ... - If all of the following conditions are true, then statement 0: - Condition 0a - Condition 0b - Instead, if one or more of the following conditions are true, then statement 1: - Condition 1a - Condition 1b - … - Otherwise, statement n

[0322] In text, logical operation statements are mathematically described in the following format. if (condition 0) Statement 0 if (condition 1) Statement 1 This can be described in the following way: When condition 0, statement 0 When condition 1 is met, statement 1

[0323] While embodiments of the present invention have been described primarily in relation to video coding, it should be noted that embodiments of the coding system 10, encoder 20, and decoder 30 (and correspondingly system 10) and other embodiments described herein can also be configured, as in video coding, for the processing or coding of still pictures, i.e., for the processing or coding of individual pictures independent of any preceding or consecutive pictures. Generally, when picture processing coding is limited to a single picture 17, only the interpretation units 244 (encoder) and 344 (decoder) may not be available. All other functions (also called tools or techniques) of the video encoder 20 and video decoder 30, such as residual calculation 204 / 304, transformation 206, quantization 208, inverse quantization 210 / 310, (inverse) transformation 212 / 312, segmentation 262 / 362, intra prediction 254 / 354, and / or loop filtering 220, 320, and entropy coding 270, and entropy decoding 304, can equally be used for still picture processing.

[0324] For example, embodiments of encoder 20 and decoder 30, and functions described herein with reference to encoder 20 and decoder 30, may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, functions may be stored in a computer-readable medium or transmitted as one or more instructions or codes over a communication medium and executed by a hardware-based processing unit. Computer-readable medium may include computer-readable storage medium, which corresponds to tangible media such as data storage media, or communication media including any medium that facilitates the transfer of computer programs from one place to another according to a communication protocol, for example. Thus, computer-readable medium generally corresponds to (1) non-transient tangible computer-readable storage media, or (2) communication media such as signals or carrier waves. Data storage media may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementing the techniques described herein. Computer program products may include computer-readable medium.

[0325] As an example, and not an limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, or any other media accessible by a computer that can be used to store desired program code in the form of instructions or data structures. Also, any connection is appropriately called computer-readable media. For example, if instructions are transmitted from a website, server or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of media. However, it should be understood that computer-readable storage media and data storage media do not include connections, carriers, signals, or other temporary media, but instead refer to non-temporary, tangible storage media. As used herein, the terms "disk" and "disc" include Compact Disc (CD), LaserDisc®, Optical Disc, Digital Multipurpose Disc (DVD), Floppy Disk®, and Blu-ray® Disc, where a disk typically reproduces data magnetically, and a disc uses a laser to reproduce data optically. Any combination of the above should also be included within the scope of computer-readable media.

[0326] Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, the term “processor” as used herein may refer to any of the aforementioned structures or any other structure suitable for implementing the techniques described herein. In addition, in some embodiments, the functions described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into a synthesized codec. Furthermore, the techniques may be fully implemented in one or more circuits or logic elements.

[0327] The techniques of this disclosure can be implemented in a wide variety of devices or apparatus, including wireless handsets, integrated circuits (ICs), or sets of ICs (e.g., chipsets). Various components, modules, or units are described in this disclosure to highlight functional aspects of devices configured to perform the techniques disclosed, but implementation by different hardware units is not necessarily required. Rather, as described above, the various units may be combined in a codec hardware unit, or they may be provided by a collection of interoperable hardware units, including one or more processors as described above, together with appropriate software and / or firmware. [Explanation of Symbols]

[0328] 12 Source Devices 13 Communication Channels 14 Destination device 16 Picture Sources 17. Picture data (picture) 18 Preprocessing Unit (Preprocessor) 19. Pre-processed picture data (pre-processed picture) 20 Video Encoders 21 Encoded picture data 22. Communication Unit (Communication Interface) 28. Communication Unit (Communication Interface) 30 video decoders 31. Decrypted picture data (decrypted picture) 32 Post-processing unit (post-processor) 33 Post-processed picture data (post-processed picture) 34 Display Devices 46 Processing Circuit 201 Input Interface 203 Picture Block 204 Residual Calculation Unit 205 Residual Block 206 Conversion Processing Unit 207 Conversion coefficient 208 Quantization Units 209 Quantized transformation coefficients 210 Inverse Quantization Unit 211 Inversely quantized residual coefficients 212 Inverse Transform Processing Unit 213 Reconstructed residual block 214 Reconstruction Unit (Adder or Subtractor) 215 Reconstructed Blocks 220 Loop Filter Unit (Loop Filter) 221 Filtered and reconstructed blocks 230 Decoded picture buffer 231 Decrypted picture 244 Interpretation Units 254 Intra Prediction Units 260 Mode Selection Unit 262 division units 265 Prediction Blocks (Predictors) 266 Syntax Elements 270 Entropy coding unit (entropy coding) 272 outputs, output interface 304 Entropy Decoding Unit 309 Quantized coefficients 310 Inverse Quantization Unit 311 Inversely quantized coefficients 312 Inverse Transform Processing Unit 313 Reconstructed residual block 314 Reconstruction Unit (Adder) 315 Reconstructed Blocks 320 Loop Filter Unit (Loop Filter) 321 filtered blocks 330 Decode picture buffer 331 Decrypted picture 344 Interpretation Units 354 Intra Prediction Units 365 Prediction Block 400 video coding devices 410 Inlet port (input port) 420 Receiver Unit 430 Central Processing Unit (Processor) 440 Transmitter Unit 450 Output Ports 460 memory 470 coding modules 502 Processors 504 memory 506 data 508 Operating Systems 510 Application Programs 512 Bus 518 displays 1210 angle alpha 1230 distance 1250 Separation line 1600 decryption devices 1601 Receiver Module 1602 Classification Parameter Processing Module 1603 Computation Module 1604 Prediction Module 3102 Capture Device 3106 Terminal device 3108 Smartphone or tablet 3110 Computer or laptop 3112 Network Video Recorder / Digital Video Recorder 3114 TV 3116 Set-Top Box 3118 Video conferencing system 3120 Video Surveillance System 3122 Mobile Information Terminal 3124 In-vehicle devices 3126 External Display 3202 Protocol Progress Unit 3204 Reverse Multiplexing Unit 3206 Video Decoder 3208 Audio Decoder 3210 Subtitle Decoder 3212 Synchronization Unit 3214 Video / Audio Display 3216 Video / Audio / Subtitle Display

Claims

1. 1. A method of coding implemented by a decoding device, comprising: obtaining a value of an indicator for a current block according to the bitstream; obtaining a value of a first parameter for the current block and a value of a second parameter for the current block according to the value of the indicator and a predetermined lookup table; obtaining a sample distance value for a sample located in the current block according to a value of the first parameter and a value of the second parameter; obtaining a predicted value for the sample according to the sample distance value for the sample; having said step of obtaining a predicted value for said sample according to said sample distance value for said sample, calculating two weighting coefficients corresponding to two sub-blocks according to the sample distance values; obtaining the predicted value for the sample according to a first predicted value for a first sub-block of the two sub-blocks based on a first prediction mode, a second predicted value for a second sub-block of the two sub-blocks based on a second prediction mode, and the two weighting factors; A method comprising:

2. The method of claim 1 , wherein the first parameter represents an angle for a partition of the current block.

3. The method of claim 1 or 2, wherein the second parameter represents a distance for a partition of the current block.

4. 4. The method of claim 1, wherein the second parameter specifies a distance index of a geometric partition or the second parameter describes a distance from a separation line to a center of the current block.

5. 5. The method according to claim 1, wherein the sample distance value represents a horizontal distance, or a vertical distance, or a combination of the horizontal and vertical distances, from the sample to a separation line, the separation line being used to divide a coding block into two sub-blocks.

6. the predetermined lookup table includes pairs of a first parameter and a second parameter; 6. The method of claim 1, wherein if the first parameter of a pair represents an angle corresponding to one of the angles 0, 45, 90, 135, 180, 225, 270 or 315 degrees, the corresponding second parameter of the pair does not represent a distance of 0 samples from the center of the current block.

7. the predetermined lookup table includes pairs of a first parameter and a second parameter; 7. The method of claim 1, wherein the first parameter of a pair represents an angle that does not correspond to one of the following angles: 0, 45, 90, 135, 180, 225, 270, or 315 degrees, and the corresponding second parameter of the pair represents a distance of 0 samples from the center of the current block.

8. the predetermined lookup table includes pairs of a first parameter and a second parameter; 8. The method of claim 1, wherein the first parameter of a pair represents an angle corresponding to one of the following angles: 0, 45, 90, 135, 180, 225, 270, or 315 degrees, and the corresponding second parameter of the pair represents a distance of 0 samples from the center of the current block.

9. A decoder comprising processing circuitry for carrying out the method according to any one of claims 1 to 8.

10. A decoder comprising: one or more processors; a non-transitory computer-readable storage medium coupled to said processor and storing a program for execution by said processor; Equipped with A decoder, the program being configured, when executed by the processor, to perform the method of any one of claims 1 to 8.

11. 1. A method of coding implemented by a coding device, comprising: obtaining a value of an indicator for the current block; obtaining a value of a first parameter for the current block and a value of a second parameter for the current block according to the value of the indicator and a predetermined lookup table; obtaining a sample distance value for a sample located in the current block according to a value of the first parameter and a value of the second parameter; obtaining a predicted value for the sample according to the sample distance value for the sample; having said step of obtaining a predicted value for said sample according to said sample distance value for said sample, calculating two weighting coefficients corresponding to two sub-blocks according to the sample distance values; obtaining the prediction value for the sample according to a first prediction value for a first sub-block of the two sub-blocks based on a first prediction mode, a second prediction value for a second sub-block of the two sub-blocks based on a second prediction mode, and the two weighting factors; encoding said indicator into a bitstream; A method comprising:

12. 1. An encoder comprising: one or more processors; a non-transitory computer-readable storage medium coupled to said processor and storing a program for execution by said processor; Equipped with The program, when executed by the processor, configures the encoder to perform the method of claim 11.

13. A computer program comprising a program code for carrying out the method according to any one of claims 1 to 8 and 11.

14. A terminal, One or more processors, a memory, and a communication interface; The memory and the communication interface are connected to the one or more processors, and the terminal communicates with other devices via the communication interface; the memory is configured to store computer program code, the computer program code including instructions: A terminal, wherein when the one or more processors execute the instructions, the terminal performs the method of any one of claims 1 to 8 and 11.

15. A data structure for use by a decoder, the data structure including an encoded bitstream, the bitstream including an indicator; a processor of the decoder, decoding the bitstream to obtain the indicator; obtaining a value of a first parameter for a current block and a value of a second parameter for the current block according to the value of the indicator and a predetermined lookup table; obtaining a sample distance value for a sample located in the current block according to a value of the first parameter and a value of the second parameter; obtaining a predicted value for the sample according to the sample distance value for the sample; Do the following: obtaining a predicted value for the sample according to the sample distance value for the sample; Calculating two weighting factors corresponding to two sub-blocks according to the sample distance values; obtaining the predicted value for the sample according to a first predicted value for a first sub-block of the two sub-blocks based on a first prediction mode, a second predicted value for a second sub-block of the two sub-blocks based on a second prediction mode, and the two weighting factors; A data structure including:

16. A device for storing a bitstream, comprising: At least one storage medium and at least one communication interface; the at least one communication interface is configured to receive or transmit a bitstream; the at least one storage medium is configured to store the bitstream; the bitstream includes an indicator of a current block, the indicator and a predefined lookup table are used to obtain a first parameter value for the current block and a second parameter value for the current block, the first parameter value and the second parameter value are used to obtain a sample distance value for a sample located in the current block, and the sample distance value for the sample is used to obtain a predicted value for the sample; the sample distance value for the sample is used to obtain a predicted value for the sample; Calculating two weighting factors corresponding to two sub-blocks according to the sample distance values; obtaining the predicted value for the sample according to a first predicted value for a first sub-block of the two sub-blocks based on a first prediction mode, a second predicted value for a second sub-block of the two sub-blocks based on a second prediction mode, and the two weighting factors; Including, the device.

17. A method for storing a bitstream, comprising the steps of: receiving or transmitting a bitstream via a communication interface; storing the bitstream in one or more storage media, the bitstream including an indicator of a current block, the indicator and a predefined lookup table being used to obtain a first parameter value for the current block and a second parameter value for the current block, the first parameter value and the second parameter value being used to obtain a sample distance value for a sample located in the current block, and the sample distance value for the sample being used to obtain a predicted value for the sample; having the sample distance value for the sample is used to obtain a predicted value for the sample; Calculating two weighting factors corresponding to two sub-blocks according to the sample distance values; obtaining the predicted value for the sample according to a first predicted value for a first sub-block of the two sub-blocks based on a first prediction mode, a second predicted value for a second sub-block of the two sub-blocks based on a second prediction mode, and the two weighting factors; A method comprising:

18. A device for transmitting a bitstream, comprising: at least one storage medium configured to store at least one bitstream, the bitstream including a current block indicator; at least one processor configured to retrieve one or more bitstreams from a storage medium of the at least one storage medium and transmit the at least one bitstream to a destination device; Equipped with The at least one processor: obtaining a value of a first parameter for the current block and a value of a second parameter for the current block according to the value of the indicator and a predetermined lookup table; obtaining a sample distance value for a sample located in the current block according to a value of the first parameter and a value of the second parameter; obtaining a predicted value for the sample according to the sample distance value for the sample; configured to: obtaining a predicted value for the sample according to the sample distance value for the sample; Calculating two weighting factors corresponding to two sub-blocks according to the sample distance values; obtaining the predicted value for the sample according to a first predicted value for a first sub-block of the two sub-blocks based on a first prediction mode, a second predicted value for a second sub-block of the two sub-blocks based on a second prediction mode, and the two weighting factors; Including, the device.

19. A method for transmitting a bitstream, comprising the steps of: storing at least one bitstream on at least one storage medium, the bitstream including an indicator of a current block, the indicator and a predefined lookup table being used to obtain a first parameter value for the current block and a second parameter value for the current block, the first parameter value and the second parameter value being used to obtain a sample distance value for a sample located in the current block, the sample distance value for the sample being used to obtain a predicted value for the sample; the sample distance value for the sample is used to obtain a predicted value for the sample; Calculating two weighting factors corresponding to two sub-blocks according to the sample distance values; Obtaining the prediction value for the sample according to a first prediction value for a first sub-block of the two sub-blocks based on a first prediction mode, a second prediction value for a second sub-block of the two sub-blocks based on a second prediction mode, and the two weighting factors; obtaining one or more bitstreams from a storage medium of the at least one storage medium; transmitting the at least one bitstream to a destination device; A method comprising:

20. A system for processing a bitstream, comprising: an encoding device, one or more storage media, and a decoding device; the encoding device is configured to obtain a video signal and encode the video signal to obtain one or more bitstreams; the bitstream includes an indicator of a current block, the indicator and a predefined lookup table are used to obtain a first parameter value for the current block and a second parameter value for the current block, the first parameter value and the second parameter value are used to obtain a sample distance value for a sample located in the current block, and the sample distance value for the sample is used to obtain a predicted value for the sample; the sample distance value for the sample is used to obtain a predicted value for the sample; Calculating two weighting factors corresponding to two sub-blocks according to the sample distance values; obtaining the predicted value for the sample according to a first predicted value for a first sub-block of the two sub-blocks based on a first prediction mode, a second predicted value for a second sub-block of the two sub-blocks based on a second prediction mode, and the two weighting factors; Including, the system.

Images