Video decoding method, video decoding device, computer program, and video encoding method

Abstract

To provide a method and device for video encoding / decoding.SOLUTION: Aspects of the disclosure provide methods and apparatuses for video encoding / decoding. In some examples, an apparatus for video decoding includes processing circuit. The processing circuit decodes prediction information for a target block within a target picture. The processing circuit determines whether an area of the target block is smaller than or equal to a threshold. The processing circuit constructs a motion vector prediction list that includes a number of motion vector predictors. The number of motion vector predictors is based on whether the area of the target block is determined to be smaller than or equal to the threshold. The processing circuit reconstructs the target block on the basis of the motion vector predictor list.SELECTED DRAWING: Figure 9

Description

[Technical Field]

[0001] Incorporation by Reference This application claims the benefit of priority to U.S. Provisional Application No. 62 / 777,735, entitled "Constructing a Simplified Merge List for Small Encoding Blocks," filed December 10, 2018, which claims the benefit of priority to U.S. Patent Application No. 16 / 555,549, entitled "Constructing a Simplified Merge List for Small Encoding Blocks," filed August 29, 2019. The entire disclosure of the prior application is incorporated herein by reference in its entirety.

[0002] This disclosure describes embodiments generally related to video coding. [Background technology]

[0003] The description of the background art provided herein is for the purpose of generally presenting the context of the present disclosure. The work of the presently named inventors, to the extent that that work is described in this Background section, and aspects of the description that would not otherwise qualify as prior art at the time of filing, are not admitted, expressly or impliedly, as prior art to the present disclosure.

[0004] Video encoding and decoding can be performed using motion-compensated inter-picture prediction. Uncompressed digital video can include a series of pictures, each with spatial dimensions of, for example, 1920 x 1080 luma samples and associated chroma samples. The series of pictures can have a fixed or variable picture rate (also informally called the frame rate), for example, 60 pictures per second or 60 Hz. Uncompressed video has significant bitrate requirements. For example, at 8 bits per sample, 1080p60 4:2:0 video (1920 x 1080 luma sample resolution at a 60 Hz frame rate) requires a bandwidth approaching 1.5 gigabits per second. One hour of such video requires more than 600 gigabytes of storage space.

[0005] One goal of video encoding and decoding can be to reduce redundancy in the input video signal through compression. Compression can help reduce the aforementioned bandwidth or storage space requirements, sometimes by more than two orders of magnitude. Both lossless and lossy compression, as well as combinations of them, can be used. Lossless compression refers to techniques that allow an exact replica of the original signal to be reconstructed from a compressed version. When using lossy compression, the reconstructed signal may not be identical to the original, but the distortion between the original and reconstructed signal is small enough to make the reconstructed signal useful for its intended application. For video, lossy compression is widely adopted. The amount of acceptable distortion varies by application; for example, users of some consumer streaming applications can tolerate higher distortion than users of television distribution applications. The achievable compression ratio can reflect higher acceptable / tolerable distortion, resulting in higher compression ratios.

[0006] Video encoders and decoders can utilize techniques in several broad categories, such as motion compensation, transforms, quantization, and entropy coding.

[0007] Video codec technology may include a technique called intra-coding, in which sample values ​​are represented without reference to samples or other data from previously reconstructed reference pictures. In some video codecs, a picture is spatially subdivided into blocks of samples. When all blocks of samples are coded in intra mode, the picture may be an intra-picture. Intra-pictures and their derivatives, such as independent decoder refresh pictures, can be used to reset the decoder state and therefore may be used as the first picture of a coded video bitstream and video session or as a still image. Samples in intra-blocks may undergo a transform, and the transform coefficients may be quantized before entropy coding. Intra-prediction may be a technique that minimizes sample values ​​in the pre-transform domain. In some cases, the smaller the DC value after the transform and the smaller the AC coefficients, the fewer bits are required to represent the block after entropy coding for a given quantization step size.

[0008] For example, traditional intra-coding, such as that known from MPEG-2 generation encoding techniques, does not use intra-prediction. However, some newer video compression techniques include techniques that rely on, for example, surrounding sample data and / or metadata obtained during spatially neighboring and preceding encoding / decoding of the data block in decoding order. Such techniques are hereinafter referred to as "intra-prediction" techniques. Note that, at least in some cases, intra-prediction uses only reference data from the current picture being reconstructed, and not from reference pictures.

[0009] Intra prediction can take various forms. If more than one such technique is available for a given video coding technique, the technique used can be coded as an intra prediction mode. In some cases, modes can include sub-modes and / or parameters, which can be coded separately or included in the mode codeword. The codeword used for a given mode / sub-mode / parameter combination can affect the coding efficiency achieved by intra prediction and can also affect the entropy coding technique used to convert the codeword into a bitstream.

[0010] A mode of intra prediction was introduced in H.264, improved in H.265, and further refined in new coding techniques such as the Joint Search Model (JEM), Versatile Video Coding (VVC), and Benchmark Set (BMS). A prediction block may be formed using neighboring sample values ​​belonging to already available samples. The sample values ​​of the neighboring samples are replicated in the prediction block according to a direction. A reference to the direction in use may be coded in the bitstream, or it may itself be predicted.

[0011] Referring to FIG. 1A, shown at the bottom right is a subset of nine prediction directions from the 33 possible prediction directions (corresponding to the 33 angle modes of the 35 intra modes) of H.265. The point where the arrows converge (101) represents the sample being predicted. The arrows represent the direction in which the sample is predicted. For example, arrow (102) indicates that sample (101) is predicted from one or more samples to the upper right, at an angle of 45 degrees from horizontal. Similarly, arrow (103) indicates that sample (101) is predicted from one or more samples to the lower left of sample (101), at an angle of 22.5 degrees from horizontal.

[0012] Referring again to FIG. 1A, a square block of 4×4 samples (104) is shown in the upper left (indicated by the dashed bold line). The square block (104) contains 16 samples. Each sample is labeled with "S" and its position in the Y dimension (e.g., row index) and its position in the X dimension (e.g., column index). For example, sample S21 is the second sample (from the top) in the Y dimension and the first sample (from the left) in the X dimension. Similarly, sample S44 is the fourth sample in the block (104) in both the Y and X dimensions. Because the block size is 4×4 samples, S44 is located in the lower right. Also shown are reference samples that follow a similar numbering scheme. The reference samples are labeled with R and their Y position (e.g., row index) and X position (column index) relative to the block (104). In both H.264 and H.265, since the predicted samples are neighbors of the block being reconstructed, negative values ​​do not need to be used.

[0013] Intra-picture prediction may work by copying reference sample values ​​from neighboring samples according to the signaled prediction direction. For example, assuming that the coded video bitstream includes a signal indicating the prediction direction for this block that matches the arrow (102), the sample is predicted from one or more prediction samples to the upper right and at a 45-degree angle from horizontal. In this case, samples S41, S32, S23, and S14 are predicted from the same reference sample R05. Then, sample S44 is predicted from reference sample R08.

[0014] In some cases, particularly when the direction is not evenly divisible by 45 degrees, the values ​​of multiple reference samples may be combined, for example by interpolation, to calculate the reference sample.

[0015] As video coding technology evolves, the number of possible directions increases. In H.264 (2003), nine different directions could be represented. This increased to 33 in H.265 (2013), and JEM / VVC / BMS can support up to 65 directions at the time of publication. Experiments have been conducted to identify the most likely directions, and some entropy coding techniques are used to represent these possible directions with a small number of bits, accepting some penalty for less likely directions. Furthermore, the direction itself may be predicted from nearby directions used in nearby, already decoded blocks.

[0016] Figure 1B shows a schematic diagram (105) showing 65 intra-prediction directions with JEM to illustrate the increasing number of prediction directions over time.

[0017] The mapping of intra-prediction direction bits in an encoded video bitstream to represent directions may vary from one video coding technique to another, ranging from simple direct mapping of prediction directions to complex adaptive schemes involving intra-prediction modes, codewords, most-likely modes, and similar techniques. However, in all cases, there may be some directions that are statistically less likely to occur in the video content than some other directions. Because the goal of video compression is to reduce redundancy, a well-performing video coding technique will represent these less-likely directions with more bits than more-likely directions.

[0018] Motion compensation may be a lossy compression technique in which blocks of sample data from a previously reconstructed picture or portion thereof (reference picture) are spatially shifted in a direction indicated by a motion vector (MV), and then used to predict a newly reconstructed picture or portion thereof. In some cases, the reference picture may be the same as the picture currently being reconstructed. The MV may have two dimensions, X and Y, or three dimensions, with the third indicating the reference picture in use (the latter may indirectly be the temporal dimension).

[0019] In some video compression techniques, the MV applicable to a region of sample data can be predicted from other MVs, e.g., from MVs associated with other regions of sample data that are spatially adjacent to the region being reconstructed and precede that MV in decoding order. Doing so can significantly reduce the amount of data required to encode the MV, thereby eliminating redundancy and improving compression ratios. For example, when encoding an input video signal derived from a camera (called natural video), MV prediction can work effectively because regions larger than the region to which a single MV is applicable have a statistical likelihood of moving in a similar direction and, therefore, can be predicted, in some cases, using similar motion vectors derived from MVs in nearby regions. As a result, the MV found for a given region is similar or identical to the MV predicted from surrounding MVs, and, after entropy coding, can then be represented using fewer bits than would be used to directly encode the MV. In some cases, MV prediction can be an example of lossless compression of a signal (i.e., MV) derived from the original signal (i.e., sample stream). In other cases, for example, prediction from several surrounding MVs child Due to rounding errors when computing (predictor), the MV prediction itself may be irreversible.

[0020] Various MV prediction mechanisms are described in H.265 / HEVC (ITU-T Recommendation H.265, "High Efficiency Video Coding," December 2016). Among the many MV prediction mechanisms provided by H.265, the advanced motion vector prediction (AMVP) mode and the merge mode are described here.

[0021] In AMVP mode, the motion information of spatially and temporally neighboring blocks of the target block can be used to predict the motion information of the target block, and the prediction residual is further coded. Examples of spatially and temporally neighboring candidates are shown in Figures 1C and 1D, respectively. Motion Vector Prediction of Two Candidates childA list is formed. The first candidate prediction is from the first available motion vector of the two blocks A0 (112) and A1 (113) in the lower left corner of the current block (111), as shown in FIG. 1C. The second candidate prediction is from the first available motion vector of the three blocks B0 (114), B1 (115), and B2 (116) above the current block (111). If no valid motion vector is found from the checked locations, the candidate is not added to the list. If two available candidates have the same motion information, only one candidate is kept in the list. If the list is not full, i.e., if there are no two different candidates in the list, the temporally co-located motion vector (after scaling) from C0 (122) in the lower right corner of the co-located block (121) in the reference picture is used as the other candidate, as shown in FIG. 1D. If motion information for C0 (122) is unavailable, the central position C1 (123) of the co-located block in the reference picture is used instead. The above derivation is still not sufficient for motion vector prediction. child If there are no candidates, a zero motion vector is used to fill the list. Two flags, mvp_l0_flag and mvp_l1_flag, are signaled in the bitstream to indicate the AMVP index (0 or 1) for the MV candidate lists L0 and L1, respectively.

[0022] In inter-picture prediction merge mode, when the merge flag (including the skip flag) is signaled as TRUE, a merge index is signaled to indicate which candidate in the merge candidate list will be used to indicate the motion vector of the current block. At the decoder, the merge candidate list is constructed based on the spatial and temporal neighborhood of the current block. As shown in Figure 1C, up to four motion vectors derived from five spatially neighboring blocks (A0-B2) are added to the merge candidate list. Additionally, as shown in Figure 1D, up to one motion vector from two temporally co-located blocks (C0 and C1) in the reference picture is added to the list. Additional merge candidates include combined bidirectional prediction candidates and zero motion vector candidates. Before selecting a block's motion information as a merge candidate, a redundancy check is performed to check whether it is identical to elements in the current merge candidate list. If it is different from each element in the current merge candidate list, it is added to the merge candidate list as a merge candidate. MaxMergeCandsNum is defined as the size of the merge candidate list in terms of candidate numbers. In HEVC, MaxMergeCandsNum is signaled in the bitstream. Skip mode can be thought of as a special merge mode with zero residual.

[0023] In VVC, the subblock-based temporal motion vector prediction (SbTMVP) method, like HEVC's temporal motion vector prediction (TMVP), can use motion fields from co-located pictures to improve the motion vector prediction and merge mode of CUs in a target picture. The same co-located pictures used in TMVP are used in SbTVMP. SbTMVP differs from TMVP in two main respects: (1) TMVP predicts motion at the CU level, while SbTMVP predicts motion at the sub-CU level; and (2) TMVP fetches temporal motion vectors from co-located blocks in the co-located picture (the co-located block is the bottom-right or center block relative to the target CU). SbTMVP applies a motion shift before fetching temporal motion information from the co-located picture, and the motion shift is obtained from a motion vector from one of the spatial neighboring blocks of the target CU.

[0024] The SbTVMP process is shown in Figures 1E and 1F. SbTMVP predicts motion vectors for sub-CUs within a target CU in two steps. In the first step, as shown in Figure 1E, the spatial neighborhood of the target block (131) is examined in the following order: A1 (132), B1 (133), B0 (134), and A0 (135). If the first available spatial neighboring block with a motion vector that uses the co-located picture as a reference picture is identified, this motion vector is selected as the motion shift to be applied. If no such motion vector is identified from the spatial neighborhood, the motion shift is set to (0, 0).

[0025] In the second step, as shown in FIG. 1F, the motion shift identified in the first step is applied (i.e., added to the coordinates of the current block) to obtain sub-CU-level motion information (e.g., motion vectors and reference indices) from the co-located picture. The example of FIG. 1F assumes that the motion shift (149) is set to the motion vector of the spatial neighboring block A1 (143). Next, for a current sub-CU (e.g., sub-CU (144)) in the current block (142) of the current picture (141), the motion information of the corresponding co-located sub-CU (e.g., co-located sub-CU (154)) in the co-located block (152) of the co-located picture (151) is used to derive the motion information of the current sub-CU. The motion information of the corresponding co-located sub-CU (e.g., co-located sub-CU (154)) is converted into a motion vector and reference index for the target sub-CU (e.g., sub-CU (144)) in a manner similar to HEVC's TMVP process, which applies temporal motion scaling to align the reference picture of the temporal motion vector with the reference picture of the target CU.

[0026] In VVC, a combined subblock-based merge list containing both SbTVMP candidates and affine merge candidates can be used in the subblock-based merge mode. The SbTVMP mode is enabled / disabled by the Sequence Parameter Set (SPS) flag. If the SbTMVP mode is enabled, the SbTMVP prediction child is added as the first entry of the sub-block-based merge list, followed by the affine merge candidates. In some applications, the maximum allowed size of the sub-block-based merge list is 5. The sub-CU size used in SbTMVP is fixed, e.g., 8x8. As with the affine merge mode, SbTMVP mode can only be applied to a CU if both its width and height are 8 or greater.

[0027] The coding logic for the additional SbTMVP merge candidate is the same as that for other merge candidates: for each CU in a P or B slice, an additional rate-distortion (RD) check is performed to determine whether to use the SbTMVP candidate.

[0028] In VVC, the history-based MVP (HMVP) method includes HMVP candidates defined as the motion information of previously coded blocks. A table containing multiple HMVP candidates is maintained during the encoding / decoding process. When a new slice is detected, the table is emptied. Whenever there is an inter-coded non-affine block, the associated motion information is added as a new HMVP candidate to the last entry of the table. The coding flow of the HMVP method is shown in Figure 1G.

[0029] The table size,S,is set to 6, which indicates that up to six HMVP candidates,can be added to the table.,When inserting a new motion candidate into the table, a constrained FIFO,rule is used so that a redundancy check is first applied to,determine whether an identical HMVP exists in the table.,If found, the identical HMVP is removed from the table, and,then all HMVP candidates are moved forward, i.e., their indexes are,decremented by one. Figure 1H shows an example of inserting a new motion candidate into the,HMVP table.

[0030] HMVP candidates may be used in the process of building a merge candidate list. The most recent few HMVP candidates in the table are checked in order and inserted into the candidate list after the TMVP candidate. Pruning is applied to HMVP candidates for spatial or temporal merge candidates, excluding sub-block motion candidates (i.e., ATMVP).

[0031] To reduce the number of pruning operations, the number of HMVP candidates to be checked (denoted by L) is set as L = (N <= 4) ≡ M : (8 - N), where N denotes the number of available non-subblock merge candidates in the table and M denotes the number of available HMVP candidates. Furthermore, the process of constructing a merge candidate list from the HMVP list terminates when the total number of available merge candidates reaches the advertised maximum allowable merge candidate minus one. Furthermore, the number of pairs of combined bidirectional predictive merge candidate derivations is reduced from 12 to 6.

[0032] HMVP candidates can also be used in the AMVP candidate list construction process. The motion vectors of the last K HMVP candidates in the table are inserted after the TMVP candidate. Only HMVP candidates with the same reference picture as the AMVP target reference picture are used to construct the AMVP candidate list. Pruning is applied to HMVP candidates. In some applications, K is set to 4, but the size of the AMVP list is not changed and is 2.

[0033] Pairwise average candidates are generated by averaging pairs of predefined candidates in the current merge candidate list. In VVC, the number of pairwise average candidates is six, and the predefined pairs are defined as {(0,1), (0,2), (1,2), (0,3), (1,3), (2,3)}, where the numbers indicate the merge index into the merge candidate list. The averaged motion vector is calculated separately for each reference list. If both motion vectors are available in one list, these two motion vectors are averaged even if they point to different reference pictures. If only one motion vector is available, that motion vector is used directly. If no motion vector is available, this list is considered invalid. Pairwise average candidates can be replaced by combined candidates in the HEVC standard.

[0034] Multi-hypothesis prediction can be used to improve the single prediction of AMVP mode. A flag is signaled to enable or disable multi-hypothesis prediction. Furthermore, when the flag is true, one additional merge index is signaled. In this way, multi-hypothesis prediction transforms single prediction into dual prediction, where one prediction is obtained using the original syntax elements of AMVP mode and the other prediction is obtained using merge mode. The final prediction combines these two predictions using a 1:1 weight, as in bidirectional prediction. A merge candidate list is first derived from the merge mode, with sub-CU candidates (such as affine, alternative temporal motion vector prediction (ATMVP)) excluded. Next, the merge candidate list is split into two separate lists: one for list 0 (L0) containing all L0 motion from the candidates, and one for list 1 (L1) containing all L1 motion. After removing redundancies and filling gaps, two merge lists are generated for L0 and L1, respectively. There are two constraints when applying multi-hypothesis prediction to improve AMVP mode. First, it is enabled for CUs with a Luma Coded Block (CB) region of 64 or more, and second, it only applies to L1 for low latency B pictures. Summary of the Invention [Problem to be solved by the invention]

[0035]

[0009] Aspects of the present disclosure provide a method and apparatus for video encoding / decoding. In some examples, an apparatus for video decoding includes a receiving circuit and a processing circuit.

[0036] The processing circuit decodes prediction information for a current block in a current picture that is part of an encoded video sequence. The processing circuit determines whether an area of ​​the current block is less than or equal to a threshold. The processing circuit performs motion vector prediction. child motion vector predictions, including the number of child Building a list: Motion vector prediction child The number of motion vector predictions is based on whether the area of ​​the current block is determined to be equal to or less than the threshold. child Reconstruct the target block based on the list. [Means for solving the problem]

[0037] According to aspects of the present disclosure, motion vector prediction child Motion vector predictions included in the list child is a first number if it is determined that the area of ​​the target block is equal to or smaller than the threshold value, and is a second number if it is determined that the area of ​​the target block is greater than the threshold value. child Motion vector predictions included in the list child is a second number greater than the first number.

[0038] According to an aspect of the present disclosure, the prediction information indicates that the current block is to be further divided into a plurality of smaller blocks. The processing circuit divides the current block into the plurality of smaller blocks. The processing circuit determines whether an area of ​​one of the plurality of smaller blocks is equal to or less than a threshold. The processing circuit performs a motion vector prediction based on whether the area of ​​one of the plurality of smaller blocks is determined to be equal to or less than the threshold. child Building a list: Motion vector prediction child Motion vector predictions included in the list child is a first number if the area of ​​one of the plurality of smaller blocks is determined to be less than or equal to the threshold, and is a second number if the area of ​​one of the plurality of smaller blocks is determined to be greater than the threshold. child Motion vector predictions included in the list child The number is the second number.

[0039] According to an aspect of the present disclosure, the prediction information indicates a partition size of the current block. The processing circuit determines whether the partition size is less than or equal to a threshold. The processing circuit performs motion vector prediction based on whether the partition size is determined to be less than or equal to the threshold. child Building a list: Motion vector prediction child Motion vector predictions included in the list child is a first number if the partition size is determined to be less than or equal to the threshold, and is a second number if the partition size is determined to be greater than the threshold. child Motion vector predictions included in the list child The number is the second number.

[0040] In one embodiment, the threshold is a preset number of luminance samples.

[0041] In one embodiment, the threshold corresponds to the picture resolution of the target picture.

[0042] In one embodiment, the threshold is signaled in the encoded video sequence.

[0043] In one embodiment, the first number of motion vector predictions child Motion vector prediction including child The list includes spatial and temporal motion vector predictions. child The spatial or temporal motion vector prediction is constructed based on the first set of child The first set of motion vector predictions child Motion vector prediction including child Spatial or temporal motion vector predictions used to construct the list child smaller than the second set.

[0044] In one embodiment, the first number of motion vector predictions child Motion vector prediction including child The list includes spatial and temporal motion vector predictions. child Does not include.

[0045] In one embodiment, a first number of redundancy checks are performed to generate a first number of motion vector predictions. child Motion vector prediction including child Constructing a list and performing a second number of redundancy checks to generate a second number of motion vector predictions. child Motion vector prediction including childBuild a list. The first number of redundancy checks is less than the second number of redundancy checks.

[0046] In one embodiment, the first number of redundancy checks includes: (i) possible spatial motion vector predictions; child and motion vector prediction child the first existing spatial motion vector prediction in the list child and (ii) possible temporal motion vector predictions. child and motion vector prediction child the first existing temporal motion vector prediction in the list child and (iii) possible history-based motion vector prediction. child and motion vector prediction child the first or other existing spatial motion vector prediction in the list child and (iv) possible history-based motion vector prediction. child and motion vector prediction child the first or other existing temporal motion vector prediction in the list child does not include at least one of the comparisons between

[0047] In one embodiment, the first number of motion vector predictions child Motion vector prediction including child The list is constructed without redundancy checking.

[0048] In one embodiment, motion vector prediction child At least one of the motion vector prediction child The current picture is used as a reference picture so that at least one reference block of is in the current picture.

[0049] Aspects of the present disclosure also provide a non-transitory computer-readable medium storing instructions that, when executed by a computer, cause the computer to perform any one or combination of methods for video decoding.

[0050] Further features, nature and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings. [Brief explanation of the drawings]

[0051] [Figure 1A] FIG. 2 is a schematic diagram of an example subset of intra-prediction modes. [Figure 1B] FIG. 1 is a diagram of an exemplary intra-prediction direction. [Figure 1C] FIG. 1 is a schematic diagram of a target block and its surrounding spatial merge candidates in one example. [Figure 1D] FIG. 1 is a schematic diagram of co-located blocks and temporal merging candidates in an example. [Figure 1E] 1 is a schematic diagram of a current block and its surrounding spatial merging candidates for sub-block-based temporal motion vector prediction (SbTMVP), according to an example. [Figure 1F] 1 is an exemplary process for deriving SbTMVP candidates according to one example. [Figure 1G] 1 is a decoding flow of a history-based motion vector prediction (HMVP) method in one example. [Figure 1H] 10 is an exemplary process for updating a table in HMVP according to one example. [Figure 2] FIG. 1 is a schematic diagram of a simplified block diagram of a communication system according to one embodiment. [Figure 3] FIG. 1 is a schematic diagram of a simplified block diagram of a communication system according to one embodiment. [Figure 4] FIG. 2 is a schematic diagram of a simplified block diagram of a decoder according to one embodiment. [Figure 5] FIG. 2 is a schematic diagram of a simplified block diagram of an encoder according to one embodiment. [Figure 6] 10 shows a block diagram of an encoder according to another embodiment; [Figure 7] 10 shows a block diagram of a decoder according to another embodiment; [Figure 8] 1 shows a flowchart outlining an exemplary process according to some embodiments of the present disclosure. [Figure 9] 10 shows another flowchart outlining an exemplary process according to some embodiments of the present disclosure. [Figure 10] FIG. 1 is a schematic diagram of a computer system according to one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0052] FIG. 2 shows a simplified block diagram of a communication system (200) according to an embodiment of the present disclosure. The communication system (200) includes, for example, multiple terminal devices capable of communicating with each other via a network (250). For example, the communication system (200) includes a first pair of terminal devices (210) and (220) interconnected via the network (250). In the example of FIG. 2, the first pair of terminal devices (210) and (220) perform unidirectional data transmission. For example, the terminal device (210) may encode video data (e.g., a stream of video pictures captured by the terminal device (210)) for transmission to another terminal device (220) via the network (250). The encoded video data may be transmitted in the form of one or more encoded video bitstreams. The terminal device (220) may receive the encoded video data from the network (250), decode the encoded video data to recover the video pictures, and display the video pictures according to the recovered video data. One-way data transmission can be common, such as in media serving applications.

[0053] In another example, the communication system (200) includes a second pair of terminal devices (230) and (240) that perform bidirectional transmission of encoded video data, such as may occur during a video conference. In the case of bidirectional transmission of data, in one example, each of the terminal devices (230) and (240) can encode video data (e.g., a stream of video pictures captured by the terminal device) for transmission to the other of the terminal devices (230) and (240) over the network (250). Each of the terminal devices (230) and (240) can also receive encoded video data transmitted by the other of the terminal devices (230) and (240), decode the encoded video data to recover video pictures, and display the video pictures on an accessible display device according to the recovered video data.

[0054] In the example of FIG. 2 , terminal devices 210, 220, 230, and 240 may be depicted as a server, a personal computer, and a smartphone, although the principles of the present disclosure need not be so limited. Embodiments of the present disclosure find application in laptop computers, tablet computers, media players, and / or dedicated videoconferencing equipment. Network 250 represents any number of networks that convey encoded video data between terminal devices 210, 220, 230, and 240, including, for example, wireline and / or wireless communication networks. Communications network 250 may exchange data over circuit-switched and / or packet-switched channels. Exemplary networks include telecommunications networks, local area networks, wide area networks, the Internet, and the like. For purposes of this discussion, the architecture and topology of network 250 may not be important to the operation of the present disclosure, unless otherwise described herein.

[0055] 3 shows the arrangement of a video encoder and a video decoder in a streaming environment as an example of an application of the disclosed subject matter. The disclosed subject matter may be equally applicable to other video-enabled applications including, for example, video conferencing, digital TV, and storage of compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0056] The streaming system may include a capture subsystem (313), which may include a video source (301), such as a digital camera, that creates a stream of uncompressed video pictures (302). In one example, the stream of video pictures (302) includes samples captured by the digital camera. The stream of video pictures (302), shown with a thicker line to emphasize its larger amount of data compared to the encoded video data (304) (or encoded video bitstream), may be processed by an electronic device (320) that includes a video encoder (303) coupled to the video source (301). The video encoder (303) may include hardware, software, or a combination thereof for enabling or implementing aspects of the disclosed subject matter, as described in more detail below. The encoded video data (304) (or encoded video bitstream (304)), shown with a thinner line to emphasize its smaller amount of data compared to the stream of video pictures (302), may be stored on a streaming server (305) for future use. One or more streaming client subsystems, such as the client subsystems (306) and (308) of FIG. 3, can access the streaming server (305) to retrieve copies (307) and (309) of the encoded video data (304). The client subsystem (306) may include, for example, a video decoder (310) within an electronic device (330). The video decoder (310) decodes the incoming copy of the encoded video data (307) and creates an output stream of video pictures (311) that can be rendered on a display (312) (e.g., a display screen) or other rendering device (not shown). In some streaming systems, the encoded video data (304), (307), and (309) (e.g., a video bitstream) may be encoded according to several video encoding / compression standards. Examples of these standards include ITU-T Recommendation H.265. As an example, a video encoding standard under development is informally known as VVC.The disclosed subject matter may be used in the context of a VVC.

[0057] It should be noted that the electronic devices (320) and (330) may include other components (not shown). For example, the electronic device (320) may include a video decoder (not shown), and the electronic device (330) may similarly include a video encoder (not shown).

[0058] 4 shows a block diagram of a video decoder (410) according to an embodiment of the present disclosure. The video decoder (410) may be included in an electronic device (430). The electronic device (430) may include a receiver (431) (e.g., a receiving circuit). The video decoder (410) may be used in place of the video decoder (310) in the example of FIG. 3.

[0059] The receiver (431) can receive one or more coded video sequences to be decoded by the video decoder (410), in the same or other embodiments, one coded video sequence at a time, with the decoding of each coded video sequence being independent of the other coded video sequences. The coded video sequences can be received from a channel (401), which can be a hardware / software link to a storage device that stores the coded video data. The receiver (431) can receive the coded video data along with other data, such as coded audio data and / or auxiliary data streams, which can be forwarded to respective using entities (not shown). The receiver (431) can separate the coded video sequences from other data. To combat network jitter, a buffer memory (415) can be coupled between the receiver (431) and the entropy decoder / parser (420) (hereinafter "parser (420)"). In some applications, the buffer memory (415) is part of the video decoder (410). In other cases, it may be external to the video decoder (410) (not shown). In still other cases, there may be a buffer memory (not shown) external to the video decoder (410), for example, to combat network jitter, and there may be another buffer memory (415) internal to the video decoder (410), for example, to handle playback timing. When the receiver (431) is receiving data from a store / forward device with sufficient bandwidth and controllability, or from a synchronous network, the buffer memory (415) may not be required or may be small. For use with best-effort packet networks such as the Internet, the buffer memory (415) may be required, but it may be relatively large and advantageously sizable, and may be implemented at least in part in an operating system or similar element (not shown) external to the video decoder (410).

[0060] The video decoder (410) may include a parser (420) for reconstructing symbols (421) from the encoded video sequence. These symbol categories include information used to manage the operation of the video decoder (410) and, in some cases, information for controlling a rendering device (e.g., a display screen), such as the rendering device (412), which may not be an integral part of the electronic device (430) but may be coupled to it, as shown in FIG. 4. The rendering device control information may be in the form of a supplemental enhancement information (SEI) message or a video usability information (VUI) parameter set fragment (not shown). The parser (420) may parse / entropy decode the received encoded video sequence. The encoding of the encoded video sequence may follow a video coding technique or standard and may follow various principles, including variable-length coding, Huffman coding, context-sensitive or non-context-sensitive arithmetic coding, etc. The parser (420) can extract from the coded video sequence a set of subgroup parameters for at least one of a subgroup of pixels in the video decoder based on at least one parameter corresponding to the group. The subgroup may include a Group of Pictures (GOP), a picture, a tile, a slice, a macroblock, a coding unit (CU), a block, a transform unit (TU), a prediction unit (PU), etc. The parser (420) can also extract information from the coded video sequence, such as transform coefficients, quantization parameter values, motion vectors, etc.

[0061] The parser (420) may perform entropy decoding / parsing operations on the video sequence received from the buffer memory (415) to create symbols (421).

[0062] The reconstruction of the symbols (421) may involve several different units, depending on the type of coded video picture or portion thereof (e.g., inter and intra pictures, inter and intra blocks, etc.), and other factors. Which units are involved and how may be controlled by subgroup control information parsed from the coded video sequence by the parser (420). The flow of such subgroup control information between the parser (420) and the following units is not shown for clarity.

[0063] In addition to the functional blocks already mentioned, the video decoder (410) may be conceptually subdivided into several functional units, as described below. In an actual implementation operating under commercial constraints, many of these units will interact closely with each other and may be at least partially integrated with each other. However, for purposes of describing the disclosed subject matter, the following conceptual subdivision into functional units is appropriate.

[0064] The first unit is a scalar / inverse transform unit (451), which receives control information from the parser (420) including the transform to be used, block size, quantization coefficients, quantization scaling matrix, etc., as well as the quantized transform coefficients as symbols (421). The scalar / inverse transform unit (451) may output blocks containing sample values ​​that can be input to an aggregator (455).

[0065] In some cases, the output samples of the scaler / inverse transform (451) may relate to intra-coded blocks, i.e., blocks that do not use prediction information from a previously reconstructed picture but may use prediction information from a previously reconstructed portion of the current picture. Such prediction information may be provided by an intra-picture prediction unit (452). In some cases, the intra-picture prediction unit (452) generates blocks of the same size and shape as the block being reconstructed using surrounding, already reconstructed information fetched from a current picture buffer (458). The current picture buffer (458), for example, buffers partially reconstructed and / or fully reconstructed current pictures. The aggregator (455) may add, on a sample-by-sample basis, the prediction information generated by the intra-prediction unit (452) to the output sample information provided by the scaler / inverse transform unit (451).

[0066] In other cases, the output samples of the scalar / inverse transform unit (451) may relate to inter-coded, possibly motion-compensated, blocks. In such cases, the motion-compensated prediction unit (453) may access a reference picture memory (457) to fetch samples used for prediction. After motion-compensating the fetched samples according to the symbols (421) associated with the block, these samples may be added by an aggregator (455) to the output of the scalar / inverse transform unit (451) to generate output sample information (in this case, referred to as residual samples or residual signals). The addresses in the reference picture memory (457) from which the motion-compensated prediction unit (453) fetches prediction samples may be controlled by motion vectors available to the motion-compensated prediction unit (453), for example, in the form of symbols (421) that may have X, Y, and reference picture components. Motion compensation may also include interpolation of sample values ​​fetched from the reference picture memory (457) when sub-sample accurate motion vectors are used, motion vector prediction mechanisms, etc.

[0067] The output samples of the aggregator (455) may be subjected to various loop filtering techniques in the loop filter unit (456). Video compression techniques may include in-loop filtering techniques controlled by parameters contained in the coded video sequence (also called coded video bitstream) and available to the loop filter unit (456) as symbols (421) from the parser (420), but may also be responsive to meta-information obtained during decoding of previous (decoding order) parts of the coded picture or coded video sequence, or may be responsive to previously reconstructed and loop-filtered sample values.

[0068] The output of the loop filter unit (456) may be a sample stream that may be output to a rendering device (412) and stored in a reference picture memory (457) for use in future inter-picture prediction.

[0069] Some coded pictures, once fully reconstructed, may be used as reference pictures for future prediction. For example, once a coded picture corresponding to a current picture is fully reconstructed and the coded picture is identified as a reference picture (e.g., by the parser (420)), the current picture buffer (458) may become part of the reference picture memory (457), and a new current picture buffer may be reallocated before beginning reconstruction of the next coded picture.

[0070] The video decoder (410) can perform decoding operations according to a predetermined video compression technique in a standard, such as ITU-T Recommendation H.265. An encoded video sequence may conform to the syntax specified by the video compression technique or standard being used, in the sense that it conforms to both the syntax of the video compression technique or standard and the profile of the video compression technique or standard as its document. Specifically, a profile can select some tools from all available tools in the video compression technique or standard as the only tools usable by that profile. Compliance also requires that the complexity of the encoded video sequence be within a level defined by the level of the video compression technique or standard. In some cases, the level imposes restrictions on the maximum picture size, maximum frame rate, maximum reconstruction sample rate (e.g., measured in megasamples per second), maximum reference picture size, etc. The restrictions set by the level may be further restricted in some cases by the specification of a hypothetical reference decoder (HRD) and HRD buffer management metadata signaled in the encoded video sequence.

[0071] In one embodiment, the receiver (431) may receive additional (redundant) data along with the encoded video. The additional data may be included as part of the encoded video sequence. The additional data may be used by the video decoder (410) to properly decode the data and / or more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0072] 5 shows a block diagram of a video encoder (503) according to an embodiment of the present disclosure. The video encoder (503) is included in an electronic device (520). The electronic device (520) includes a transmitter (540) (e.g., a transmitting circuit). The video encoder (503) may be used in place of the video encoder (303) in the example of FIG. 3.

[0073] The video encoder (503) can receive video samples from a video source (501) (not part of the electronic device (520) in the example of FIG. 5), which can capture video images that are encoded by the video encoder (503). In other examples, the video source (501) is part of the electronic device (520).

[0074] The video source (501) may provide a source video sequence that is encoded by the video encoder (503) in the form of a digital video sample stream, which may be of any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, etc.), any color space (e.g., BT.601 YCrCB, RGB, etc.), and any suitable sampling structure (e.g., YCrCb 4:2:0, YCrCb 4:4:4). In a media serving system, the video source (501) may be a storage device that stores previously prepared video. In a video conferencing system, the video source (501) may be a camera that captures local image information as a video sequence. Video data may be provided as multiple individual pictures that, when viewed sequentially, impart motion. The pictures themselves may be organized as a spatial array of pixels, each of which may contain one or more samples, depending on the sampling structure, color space, etc., in use. Those skilled in the art will readily understand the relationship between pixels and samples. The following discussion will focus on samples.

[0075] According to one embodiment, the video encoder (503) may encode and compress pictures of a source video sequence into an encoded video sequence (543) in real time or under any other time constraint required by the application. Enforcing an appropriate encoding rate is one function of the controller (550). In some embodiments, the controller (550) controls and is functionally coupled to other functional units, as described below. For clarity, coupling is not shown. Parameters set by the controller (550) may include rate control-related parameters (e.g., picture skip, quantization, lambda value for rate-distortion optimization techniques), picture size, group of pictures (GOP) layout, maximum motion vector search range, etc. The controller (550) may be configured with other appropriate functions related to optimizing the video encoder (503) for a particular system design.

[0076] In some embodiments, the video encoder (503) is configured to operate in an encoding loop. As an overly simplistic explanation, in one example, the encoding loop may include a source coder (530) (e.g., responsible for creating symbols, such as a symbol stream, based on an input picture to be encoded and reference pictures) and a (local) decoder (533) embedded in the video encoder (503). The decoder (533) reconstructs the symbols to create sample data in a manner similar to that created by the (remote) decoder (because compression between the symbols and the encoded video bitstream is lossless in the video compression techniques considered in the disclosed subject matter). The reconstructed sample stream (sample data) is input to a reference picture memory (534). Because decoding of the symbol stream yields bit-perfect results regardless of the location of the decoder (local or remote), the contents of the reference picture memory (534) are also bit-perfect between the local and remote encoders. In other words, the prediction part of the encoder "sees" the samples of the reference picture as exactly the same sample values ​​that the decoder "sees" when using the prediction during decoding. This basic principle of reference picture synchrony (and the drift that occurs when synchrony cannot be maintained, for example, due to channel errors) is also used in several related technologies.

[0077] The operation of the "local" decoder (533) may be the same as the operation of a "remote" decoder, such as the video decoder (410), already described in detail in connection with Figure 4. However, with brief reference also to Figure 4, because symbols are available and the encoding / decoding of the symbols into an encoded video sequence by the entropy coder (545) and parser (420) may be lossless, the entropy decoding portion of the video decoder (410), as well as the buffer memory (415) and parser (420), may not be fully implemented in the local decoder (533).

[0078] It may be noted at this point that any decoder technology other than analysis / entropy decoding present in a decoder must necessarily be present in the corresponding encoder in substantially identical functional form. For this reason, the disclosed subject matter focuses on the operation of the decoder. A description of the encoder technology can be omitted, as it is the reverse of the decoder technology described comprehensively. Only a few areas require more detailed description, which are provided below.

[0079] In operation, in some cases, the source coder (530) may perform motion-compensated predictive coding, which predictively codes an input picture with reference to one or more previously coded pictures from a video sequence designated as “reference pictures.” In this way, the coding engine (532) codes differences between pixel blocks of the input picture and pixel blocks of reference pictures that may be selected as predictive references for the input picture.

[0080] The local video decoder (533) may decode the coded video data of pictures that may be designated as reference pictures based on symbols created by the source coder (530). The operation of the coding engine (532) may advantageously be a lossy process. When the coded video data is decoded by a video decoder (not shown in FIG. 5), the reconstructed video sequence may typically be a replica of the source video sequence, possibly with some errors. The local video decoder (533) may replicate the decoding process that may be performed by the video decoder on the reference pictures and store the reconstructed reference pictures in a reference picture cache (534). In this way, the video encoder (503) may locally store copies of reconstructed reference pictures that have common content (without transmission errors) as reconstructed reference pictures retrieved by the far-end video decoder.

[0081] The predictor (535) may perform the prediction search of the coding engine (532). That is, for a new picture to be coded, the predictor (535) may search the reference picture memory (534) for sample data (as candidate reference pixel blocks) or certain metadata of the reference picture, such as motion vectors, block shapes, etc., which may serve as suitable prediction references for the new picture. The predictor (535) may operate pixel block by pixel block of sample blocks to find suitable prediction references. In some cases, as determined by the search results obtained by the predictor (535), the input picture may have prediction references drawn from multiple reference pictures stored in the reference picture memory (534).

[0082] The controller (550) may manage the encoding operations of the source coder (530), including, for example, setting parameters and subgroup parameters used to encode the video data.

[0083] The output of all the aforementioned functional units may undergo entropy coding in an entropy coder (545), which converts the symbols produced by the various functional units into an encoded video sequence by losslessly compressing the symbols according to techniques such as Huffman coding, variable length coding, or arithmetic coding.

[0084] The transmitter (540) may buffer the encoded video sequence created by the entropy coder (545) for transmission over a communication channel (560), which may be a hardware / software link to a storage device that stores the encoded video data. The transmitter (540) may merge the encoded video data from the video coder (503) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (sources not shown).

[0085] The controller (550) may manage the operation of the video encoder (503). During encoding, the controller (550) may assign a coded picture type to each coded picture, which may affect the coding technique that may be applied to the respective picture. For example, in many cases, pictures may be assigned one of the following picture types:

[0086] An intra picture (I-picture) may be one that can be coded and decoded without using other pictures in a sequence as a source of prediction. Some video codecs allow for various types of intra pictures, such as, for example, Independent Decoder Refresh ("IDR") pictures. Those skilled in the art are aware of these variations of I-pictures and their respective uses and characteristics.

[0087] A predicted picture (P picture) may be one that can be coded and decoded using intra- or inter-prediction, which uses at most one motion vector and reference index to predict the sample values ​​of each block.

[0088] Bidirectionally predicted pictures (B-pictures) may be those that can be coded and decoded using intra- or inter-prediction, which uses up to two motion vectors and reference indices to predict the sample values ​​of each block. Similarly, multiple predicted pictures can use more than two reference pictures and associated metadata to reconstruct a single block.

[0089] A source picture is typically spatially divided into multiple sample blocks (e.g., blocks of 4x4, 8x8, 4x8, or 16x16 samples each) and coded block by block. Blocks may be predictively coded with reference to other (already coded) blocks, as determined by the coding assignment applied to the block's respective picture. For example, blocks of an I-picture may be non-predictively coded or predictively coded with reference to already coded blocks of the same picture (spatial or intra-prediction). Pixel blocks of a P-picture may be predictively coded via spatial prediction or via temporal prediction with reference to one previously coded reference picture. Blocks of a B-picture may be predictively coded via spatial prediction or via temporal prediction with reference to one or two previously coded reference pictures.

[0090] The video encoder (503) may perform encoding operations according to a predetermined video encoding technique or standard, such as ITU-T Recommendation H.265. In its operations, the video encoder (503) may perform various compression operations, including predictive encoding operations that exploit temporal and spatial redundancies in the input video sequence. Thus, the encoded video data may conform to the syntax specified by the video encoding technique or standard being used.

[0091] In one embodiment, the transmitter (540) can transmit additional data along with the coded video. The source coder (530) may include such data as part of the coded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant pictures and slices, SEI messages, VUI parameter set fragments, etc.

[0092] Video may be captured as multiple source pictures (video pictures) in a time sequence. Intra-picture prediction (often abbreviated as intra-prediction) exploits spatial correlation within a given picture, while inter-picture prediction exploits correlation (temporal or otherwise) between pictures. In one example, a particular picture being encoded / decoded, called a current picture, is divided into blocks. If a block of the current picture is similar to a reference block of a reference picture in a previously encoded and still buffered video, the block of the current picture may be coded by a vector called a motion vector. A motion vector points to a reference block within the reference picture and may have a third dimension that identifies the reference picture if multiple reference pictures are used.

[0093] In some embodiments, inter-picture prediction may use a bidirectional prediction technique. Bidirectional prediction techniques use two reference pictures, such as a first reference picture and a second reference picture, both of which precede a current picture in a video in decoding order (but may be past and future in display order, respectively). A block in the current picture may be coded by a first motion vector pointing to a first reference block in the first reference picture and a second motion vector pointing to a second reference block in the second reference picture. A block may be predicted by a combination of the first and second reference blocks.

[0094] Furthermore, merge mode techniques can be used in inter-picture prediction to improve coding efficiency.

[0095] According to some embodiments of the present disclosure, prediction, such as inter-picture prediction and intra-picture prediction, is performed in units of blocks. For example, according to the HEVC standard, pictures in a sequence of video pictures are divided into coding tree units (CTUs) for compression, and the CTUs of a picture are of the same size, such as 64x64 pixels, 32x32 pixels, or 16x16 pixels. Generally, a CTU includes three coding tree blocks (CTBs), one luma CTB and two chroma CTBs. Each CTU may be recursively quadtree-decomposed into one or more coding units (CUs). For example, a 64x64 pixel CTU may be divided into one CU of 64x64 pixels, four CUs of 32x32 pixels, or 16 CUs of 16x16 pixels. In one example, each CU is analyzed to determine the prediction type of the CU, such as inter prediction type or intra prediction type. The CU is then divided into one or more prediction units (PUs) according to temporal and / or spatial predictability. Generally, each PU includes a luma prediction block (PB) and two chroma PBs. In one embodiment, the prediction operation in encoding (encoding / decoding) is performed in units of a prediction block. Using a luma prediction block as an example of a prediction block, the prediction block includes a matrix of pixel values ​​(e.g., luma values) of 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, etc.

[0096] 6 shows a diagram of a video encoder (603) according to another embodiment of the present disclosure. The video encoder (603) is configured to receive a processed block (e.g., a predictive block) of sample values ​​in a target video picture in a sequence of video pictures and encode the processed block into an encoded picture that is part of the encoded video sequence. In one example, the video encoder (603) is used in place of the video encoder (303) in the example of FIG. 3.

[0097] In an HEVC example, the video encoder (603) receives a matrix of sample values ​​for a processing block, such as a predictive block, such as 8x8 samples. The video encoder (603) determines, for example, using rate-distortion optimization, whether the processing block is best coded using intra mode, inter mode, or bidirectional prediction mode. If the processing block is coded in intra mode, the video encoder (603) can code the processing block into a coded picture using an intra prediction technique, and if the processing block is coded in inter mode or bidirectional prediction mode, the video encoder (603) can code the processing block into a coded picture using an inter prediction or bidirectional prediction technique, respectively. In some video coding techniques, merge mode is used when a motion vector is coded in a predictive mode. child One or more motion vector predictions without the benefit of coded motion vector components outside the predictors child In some other video encoding techniques, there may be motion vector components applicable to the current block. In one example, the video encoder (603) includes other components, such as a mode decision module (not shown), for determining the mode of the processing block.

[0098] In the example of Figure 6, the video encoder (603) includes an inter-encoder (630), an intra-encoder (622), a residual calculator (623), a switch (626), a residual encoder (624), a general controller (621), and an entropy encoder (625) coupled together as shown in Figure 6.

[0099] The inter-encoder (630) is configured to receive a sample of a current block (e.g., a processing block), compare the block with one or more reference blocks of reference pictures (e.g., blocks of previous and subsequent pictures), generate inter-prediction information (e.g., an inter-coding technique, a motion vector, a description of redundant information through merge mode information), and calculate an inter-prediction result (e.g., a predicted block) based on the inter-prediction information using any suitable technique. In some examples, the reference picture is a decoded reference picture that is decoded based on the coded video information.

[0100] The intra encoder (622) is configured to receive samples of a current block (e.g., a processing block), possibly compare the block with previously coded blocks in the same picture, generate quantized coefficients after transformation, and possibly also generate intra prediction information (e.g., intra prediction direction information according to one or more intra coding techniques). In one example, the intra encoder (622) also calculates intra prediction results (e.g., prediction blocks) based on the intra prediction information and reference blocks in the same picture.

[0101] The general controller (621) is configured to determine general control data and control other components of the video encoder (603) based on the general control data. In one example, the general controller (621) determines the mode of the block and provides a control signal to the switch (626) based on the mode. For example, if the mode is intra mode, the general controller (621) controls the switch (626) to select the intra mode result used by the residual calculator (623) and controls the entropy encoder (625) to select the intra prediction information and include it in the bitstream. If the mode is inter mode, the general controller (621) controls the switch (626) to select the inter prediction result used by the residual calculator (623) and controls the entropy encoder (625) to select the inter prediction information and include it in the bitstream.

[0102] The residual calculator (623) is configured to calculate a difference (residual data) between the received block and a prediction result selected from the intra-encoder (622) or the inter-encoder (630). The residual encoder (624) is configured to operate on the residual data and encode the residual data to generate transform coefficients. In one example, the residual encoder (624) is configured to transform the residual data from the spatial domain to the frequency domain to generate transform coefficients. The transform coefficients are then subjected to a quantization process to obtain quantized transform coefficients. In various embodiments, the video encoder (603) also includes a residual decoder (628). The residual decoder (628) is configured to perform an inverse transform and generate decoded residual data. The decoded residual data can be used by the intra-encoder (622) and the inter-encoder (630), as appropriate. For example, the inter-encoder (630) can generate decoded blocks based on the decoded residual data and inter-prediction information, and the intra-encoder (622) can generate decoded blocks based on the decoded residual data and intra-prediction information. The decoded blocks are processed appropriately to generate decoded pictures, which may be buffered in a memory circuit (not shown) and used as reference pictures in some examples.

[0103] The entropy encoder (625) is configured to format the bitstream to include the encoded block. The entropy encoder (625) is configured to include various information in accordance with an appropriate standard, such as the HEVC standard. In one example, the entropy encoder (625) is configured to include general control data, selected prediction information (e.g., intra-prediction information or inter-prediction information), residual information, and other appropriate information in the bitstream. It should be noted that, according to the disclosed subject matter, there is no residual information when encoding a block in a merged sub-mode of either the inter mode or the bi-prediction mode.

[0104] 7 shows a diagram of a video decoder (710) according to another embodiment of the present disclosure. The video decoder (710) is configured to receive coded pictures that are part of a coded video sequence and decode the coded pictures to generate reconstructed pictures. In one example, the video decoder (710) is used in place of the video decoder (310) in the example of FIG. 3.

[0105] In the example of Figure 7, the video decoder (710) includes an entropy decoder (771), an inter-decoder (780), a residual decoder (773), a reconstruction module (774), and an intra-decoder (772) coupled together as shown in Figure 7.

[0106] The entropy decoder (771) can be configured to reconstruct from the coded picture several symbols representing the syntax elements that make up the coded picture. Such symbols can include, for example, the mode in which the block is coded (e.g., intra mode, inter mode, bidirectional prediction mode, inter mode and bidirectional prediction mode in merged or other submodes), prediction information (e.g., intra prediction information or inter prediction information) that can identify certain samples or metadata used for prediction by the intra decoder (772) or inter decoder (780), respectively, residual information in the form of quantized transform coefficients, etc. In one example, if the prediction mode is an inter prediction mode or bidirectional prediction mode, the inter prediction information is provided to the inter decoder (780), and if the prediction type is an intra prediction type, the intra prediction information is provided to the intra decoder (772). The residual information, which may undergo inverse quantization, is provided to the residual decoder (773).

[0107] The inter decoder (780) is configured to receive the inter prediction information and generate an inter prediction result based on the inter prediction information.

[0108] The intra decoder (772) is configured to receive the intra prediction information and to generate a prediction result based on the intra prediction information.

[0109] The residual decoder (773) is configured to perform inverse quantization to extract inverse quantized transform coefficients and process the inverse quantized transform coefficients to transform the residual from the frequency domain to the spatial domain. The residual decoder (773) may also require some control information (to include quantization parameters (QPs)), which may be provided by the entropy decoder (771) (datapath not shown, as it is only low-volume control information).

[0110] The reconstruction module (774) is configured to combine, in the spatial domain, the residual output by the residual decoder (773) and the prediction result (possibly output by the inter- or intra-prediction module) to form a reconstructed block, which may be part of a reconstructed picture, which may be part of a reconstructed video. It should be noted that other suitable operations, such as a deblocking operation, may be performed to improve visual quality.

[0111] It should be noted that the video encoders (303), (503), and (603) and the video decoders (310), (410), and (710) may be implemented using any suitable technique. In one embodiment, the video encoders (303), (503), and (603) and the video decoders (310), (410), and (710) may be implemented using one or more integrated circuits. In other embodiments, the video encoders (303), (503), and (603) and the video decoders (310), (410), and (710) may be implemented using one or more processors executing software instructions.

[0112] For blocks coded in non-subblock mode, motion vector prediction childThe motion vector predictor list can be constructed as follows: (1) candidate predictors from spatially and temporally neighboring blocks, (2) candidate predictors from the HMVP buffer, and (3) existing motion vector predictors. child (4) a zero motion vector prediction using a different reference picture. Examples of these candidate predictions are given above.

[0113] Motion Vector Prediction child When building the list, a series of operations may be performed, including an availability check and a redundancy check. The availability check checks whether spatial or temporal neighboring blocks are coded in inter prediction mode. The redundancy check checks whether the new motion vector prediction is a motion vector prediction. child Check for overlap with existing motion vector predictions in the list. For example, availability and redundancy checks are performed for spatial and temporal candidates. Redundancy checks are performed for existing candidates. Additionally, redundancy checks are performed for candidates from the HMVP buffer against existing spatial / temporal candidates. Motion Vector Prediction child If the list is long, these operations may take several cycles to perform. In the worst case, the picture is divided into many small blocks. Motion vector prediction for all these small blocks child The total number of cycles required to complete the list construction may be higher than desired. For example, the merge candidate list for each coding block may not be the same as that of its neighboring blocks. Therefore, especially for small blocks, it is necessary to reduce the number of operations and improve the motion vector prediction. child It is desirable to simplify the list building process.

[0114] This disclosure provides a method for motion vector prediction when certain block size constraints are met. childWe provide improved techniques (e.g., merge mode, skip mode, or AMVP mode) to simplify the list construction process. That is, if the current block is considered a small block, we can predict the motion vector of the current block. child Simplify the list building process to include motion vector predictions child This can reduce the number of availability and / or redundancy check operations performed.

[0115] Various thresholds can be used to indicate a small block. In one embodiment, a target block may be considered a small block if its block area is less than or equal to a threshold. In another embodiment, the target block is further divided into multiple smaller blocks, and the target block is considered a small block if one of the multiple smaller blocks is less than or equal to a threshold. The threshold can be a predetermined number of luma samples, such as 32 or 64 luma samples. The threshold can correspond to the picture resolution of the target picture. For example, a higher resolution picture may have a higher threshold than a lower resolution picture. Furthermore, the threshold can be signaled in the bitstream or coded video sequence, such as in a sequence parameter set (SPS), picture parameter set (PPS), slice or tile header, etc.

[0116] According to an aspect of the present disclosure, when the target block is considered to be a small block, a motion vector prediction for the target block is performed. child The list building process is performed by comparing the motion vector predictions included with blocks that are not considered small blocks. child can be simplified to reduce the number of

[0117] The motion vector prediction categories, such as spatial motion vector prediction, temporal motion vector prediction, and history-based motion vector prediction, each include multiple motion vector predictions. childIn one embodiment, if the current block is considered a small block, motion vector predictions from one or more of these categories may be included. child For example, for each motion vector prediction category, the number of respective motion vector predictions can be reduced. child A subset of the motion vector prediction child The resulting motion vector prediction child The list shows all motion vector predictions. child It is shorter than including it in the list. child The subset of may be the first N (e.g., N=1, 2, etc.) candidates from each or some categories, where N is an integer smaller than the number of possible candidates in the respective category. However, other predetermined candidates may be selected in other embodiments. The number N of each or some categories may be the same or different. Furthermore, the motion vector prediction for each or some categories child The selection of the subset of can be the same or different. In one example, only the first N spatial and / or temporal candidates are used as motion vector predictors. child are allowed in the list, where the number N is less than the number of possible spatial and temporal candidates. In some examples, N may be different for spatial and temporal candidates.

[0118] In one embodiment, for the spatial motion vector prediction category and the temporal motion vector prediction category, child Subset of and temporal motion vector prediction child A subset of the motion vector prediction child list, followed by HMVP candidates, etc. In other embodiments, motion vector prediction child The list includes spatial and / or temporal motion vector predictions. child i.e., spatial and / or temporal candidates are not allowed, and motion vector prediction child The list starts with the HMVP candidates.

[0119] According to an aspect of the present disclosure, when the target block is considered to be a small block, a motion vector prediction for the target block is performed. child The process of building the list can be simplified so that fewer operations, such as redundancy check operations, are performed. The redundancy check is performed by comparing the new motion vector predictions with the motion vector predictions. child Existing motion vector predictions in the list child Compare with.

[0120] In one embodiment, one or more redundancy checks between two spatial and / or temporal candidates can be eliminated. For example, motion vector prediction child The list shows possible spatial motion vector predictions. child and motion vector prediction child No redundancy check is performed between the spatial motion vector predictions already in the list. child The list contains possible temporal motion vector predictions. child and motion vector prediction child Existing temporal motion vector predictions in the list child Do not perform redundancy checks between

[0121] In one embodiment, one or more redundancy checks between candidates from the HMVP buffer and existing candidates from spatial and / or temporal neighboring locations can be eliminated. For example, motion vector prediction child The list shows possible history-based motion vector predictions. child and motion vector prediction child Existing spatial motion vector predictions in the list child In another example, the motion vector prediction child The list shows possible history-based motion vector predictions. child and motion vector prediction child Existing temporal motion vector predictions in the list child Do not perform redundancy checks between

[0122] In one embodiment, no redundancy check is performed, i.e., motion vector prediction child The list is constructed without performing any redundancy checks.

[0123] FIG. 8 illustrates a motion vector prediction method in accordance with some embodiments of the present disclosure. child 1 shows a flowchart outlining an exemplary process (800) that simplifies the list construction process. In various embodiments, the process (800) is performed by a processing circuit, such as a processing circuit of a terminal device (210), (220), (230), or (240), a processing circuit that performs the functions of a video encoder (303), a processing circuit that performs the functions of a video decoder (310), a processing circuit that performs the functions of a video decoder (410), a processing circuit that performs the functions of an intra-prediction module (452), a processing circuit that performs the functions of a video encoder (503), a processing circuit that performs the functions of a predictor (535), a processing circuit that performs the functions of an intra-encoder (622), or a processing circuit that performs the functions of an intra-decoder (772). In some embodiments, the process (800) is implemented in software instructions, and thus, when the processing circuit executes the software instructions, the processing circuit performs the process (800).

[0124] The process (800) may generally begin at step (S801), where the process (800) determines whether to further divide the target block. If it is determined not to further divide the target block, the process (800) proceeds to step (S802); otherwise, the process (800) proceeds to step (S803).

[0125] In step (S802), the process (800) determines whether the block area of ​​the target block is equal to or less than a threshold value. If it is determined that the block area of ​​the target block is equal to or less than the threshold value, the process (800) proceeds to step (S805); if not, the process (800) proceeds to step (S806).

[0126] In step (S803), the process (800) divides the current block into multiple smaller blocks and then proceeds to step (S804).

[0127] In step (S804), the process (800) determines whether the block area of ​​one of the plurality of smaller blocks is less than or equal to a threshold value. The threshold value may be a preset number of luma samples, may correspond to the picture resolution of the target picture, or may be signaled in the coded video sequence. If it is determined that the block area of ​​one of the plurality of smaller blocks is less than or equal to the threshold value, the process (800) proceeds to step (S805); otherwise, the process (800) proceeds to step (S806).

[0128] In some embodiments, step (S801) and / or step (S803) are optional and may not be performed. For example, if the decoded prediction information of the current block indicates a partition size of the current block, the partition size indicates that the current block is to be further divided into multiple smaller blocks. Process (800) determines whether the partition size is less than or equal to a threshold value. This procedure is the same as step (S804). If it is determined that the partition size is less than or equal to the threshold value, process (800) proceeds to step (S805); otherwise, the process proceeds to step (S806). In other embodiments, the comparison of the block with the threshold value is performed after block division has already been performed.

[0129] In step S805, the process S800 calculates a first number of motion vector predictions. child Motion vector prediction including child The list is constructed, and then the process proceeds to step (S807).

[0130] In step S806, the process S800 calculates a second number of motion vector predictions. child Motion vector prediction including child Build a list, the second number being greater than the first number, i.e., the motion vector prediction child The list may be generated by, for example, the method described above, to generate a second number of motion vector predictions. childthe first number of motion vector predictions child Then, the process (800) proceeds to step (S807).

[0131] In step (S807), the process (800) performs motion vector prediction. child The target block is reconstructed based on the list, after which the process (800) ends.

[0132] 9 shows a flowchart outlining an example process (900) according to some embodiments of the present disclosure. In various embodiments, the process (900) is performed by a processing circuit, such as a processing circuit of a terminal device (210), (220), (230), or (240), a processing circuit performing the functions of a video encoder (303), a processing circuit performing the functions of a video decoder (310), a processing circuit performing the functions of a video decoder (410), a processing circuit performing the functions of an intra-prediction module (452), a processing circuit performing the functions of a video encoder (503), a processing circuit performing the functions of a predictor (535), a processing circuit performing the functions of an intra-encoder (622), or a processing circuit performing the functions of an intra-decoder (772). In some embodiments, the process (900) is implemented with software instructions, and thus, the processing circuit performs the process (900) when the processing circuit executes the software instructions.

[0133] The process (900) may generally begin at step (S901), where the process (900) decodes prediction information for a current block in a current picture that is part of a coded video sequence. After decoding the prediction information, the process (900) proceeds to step (S902).

[0134] In step (S902), the process (900) determines whether the area of ​​the current block is less than or equal to a threshold. Then, the process (900) proceeds to step (S903). The threshold may be a preset number of luma samples, such as 32 or 64 luma samples. The threshold may correspond to the picture resolution of the current picture. The threshold may be signaled in the coded video sequence.

[0135] In step (S903), the process (900) performs motion vector prediction. child motion vector predictions, including the number of child Building a list: Motion vector prediction child The number of motion vector predictions is based on whether the area of ​​the target block is determined to be less than or equal to the threshold. child Motion vector predictions included in the list child is the first number, otherwise the motion vector prediction child Motion vector predictions included in the list child is a second number greater than the first number.

[0136] In some embodiments, the prediction information indicates that the current block is to be further divided into a plurality of smaller blocks, and the process (900) divides the current block into the plurality of smaller blocks and determines whether an area of ​​one of the plurality of smaller blocks is less than or equal to a threshold. The process (900) performs a motion vector prediction based on whether the area of ​​one of the plurality of smaller blocks is determined to be less than or equal to the threshold. child The list is further constructed. If the area of ​​one of the small blocks is determined to be less than or equal to the threshold, the motion vector prediction is performed. child Motion vector predictions included in the list child is the first number, otherwise the motion vector prediction child Motion vector predictions included in the list child The number is the second number.

[0137] In some embodiments, the prediction information indicates a partition size of the current block. In such embodiments, the process (900) determines whether the partition size is less than or equal to a threshold. The process (900) performs motion vector prediction based on whether the partition size is determined to be less than or equal to the threshold. child If the partition size is determined to be less than or equal to the threshold, the list is further constructed. child Motion vector predictions included in the list child is the first number, otherwise the motion vector prediction child Motion vector predictions included in the list child The number is the second number.

[0138] In one embodiment, the first number of motion vector predictions child Motion vector prediction including child The list includes spatial and temporal motion vector predictions. child The spatial or temporal motion vector prediction is constructed based on the first set of child The first set of motion vector predictions child Motion vector prediction including child Spatial or temporal motion vector predictions used to construct the list child smaller than the second set.

[0139] In one embodiment, the first number of motion vector predictions child Motion vector prediction including child The list includes spatial and temporal motion vector predictions. child Does not include.

[0140] In one embodiment, the first number of motion vector predictions child Motion vector prediction including child To build the list, some redundancy checks are performed, e.g., a first number of redundancy checks are performed to generate a first number of motion vector predictions. child Motion vector prediction including childConstructing a list and performing a second number of redundancy checks to generate a second number of motion vector predictions. child Motion vector prediction including child Build a list. The first number of redundancy checks is less than the second number of redundancy checks.

[0141] In one embodiment, the first number of redundancy checks includes: (i) possible spatial motion vector predictions; child and motion vector prediction child the first existing spatial motion vector prediction in the list child and (ii) possible temporal motion vector prediction. child and motion vector prediction child the first existing temporal motion vector prediction in the list child and (iii) possible history-based motion vector prediction. child and motion vector prediction child the first or other existing spatial motion vector prediction in the list child and (iv) possible history-based motion vector prediction. child and motion vector prediction child the first or other existing temporal motion vector prediction in the list child does not include at least one of the comparisons between

[0142] In one embodiment, the first number of motion vector predictions child Motion vector prediction including child The list is constructed without redundancy checking.

[0143] In one embodiment, motion vector prediction child At least one of the motion vector prediction child The current picture is used as a reference picture so that at least one reference block of is in the current picture.

[0144] In step (S904), the process (900) performs motion vector prediction. child Reconstruct the target block based on the list.

[0145] After the target block is reconstructed, the process (900) ends.

[0146] The above embodiment is a motion vector prediction child In other embodiments, the motion vector prediction is based on reducing the number of redundancy checks in addition to the number of child Note that the number of redundancy checks can be reduced without reducing the number of .

[0147] The motion vector prediction method can use a picture other than the current picture as a reference picture. However, the motion vector prediction or block compensation can be performed from a previously reconstructed region in the current picture. Such motion vector prediction or block compensation is sometimes called intra-picture block compensation, current picture reference (CPR), or intra-block copy (IBC). In the IBC prediction mode, a displacement vector indicating the offset between the current block and the reference block in the current picture is called a block vector (BV). The reference block has already been reconstructed before the current block. It should be noted that the BV can be considered as the MV described in this application.

[0148] The techniques described above can be implemented as computer software using computer-readable instructions and physically stored on one or more computer-readable media. For example, Figure 10 illustrates a computer system (1000) suitable for implementing some embodiments of the disclosed subject matter.

[0149] Computer software can be encoded using any suitable machine code or computer language and can be subjected to assembly, compilation, linking, or similar mechanisms to create code containing instructions that can be executed by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc., directly, or through interpretation, execution of microcode, etc.

[0150] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, Internet of Things devices, and the like.

[0151] 10 for computer system 1000 are exemplary in nature and are not intended to suggest any limitation as to the scope of use or functionality of the computer software implementing embodiments of the present disclosure, nor should the arrangement of components be interpreted as having a dependency or requirement regarding any one or combination of components illustrated in the exemplary embodiment of computer system 1000.

[0152] The computer system (1000) may include several human interface input devices. Such human interface input devices may respond to input by one or more users, such as tactile input (e.g., keystrokes, swipes, data glove movements), audio input (e.g., voice, clapping), visual input (e.g., gestures), and olfactory input (not shown). The human interface devices may also be used to capture certain media that do not necessarily involve direct conscious human input, such as sound (e.g., speech, music, environmental sounds), images (e.g., scanned images, photographic images obtained from a still image camera), and video (e.g., two-dimensional video, three-dimensional video, including stereoscopic video).

[0153] Input human interface devices may include a keyboard (1001), a mouse (1002), a trackpad (1003), a touch screen (1010), a data glove (not shown), a joystick (1005), a microphone (1006), a scanner (1007), a camera (1008), etc. (only one of each is shown).

[0154] The computer system (1000) may also include several human interface output devices. Such human interface output devices may stimulate one or more of the user's senses, for example, through tactile output, sound, light, and smell / taste. Such human interface output devices may include haptic output devices (e.g., haptic feedback via a touchscreen (1010), data gloves (not shown), or joystick (1005), although haptic feedback devices that do not function as input devices may also be present), audio output devices (such as speakers (1009), headphones (not shown)), visual output devices (such as screens (1010), including CRT screens, LCD screens, plasma screens, and OLED screens, each with or without touchscreen input capability, each with or without haptic feedback capability, some of which may provide two-dimensional visual output or output in more than three dimensions via means such as stereographic output, virtual reality glasses (not shown), holographic displays, and smoke tanks (not shown)), and printers (not shown). These visual output devices (such as a screen (1010)) may be connected to the system bus (1048) via a graphics adapter (1050).

[0155] The computer system (1000) may also include human-accessible storage devices and their associated media, such as optical media including CD / DVD ROM / RW (1020) along with media such as CD / DVD (1021), thumb drives (1022), removable hard drives or solid state drives (1023), legacy magnetic media such as tape and floppy disks (not shown), dedicated ROM / ASIC / PLD-based devices such as security dongles (not shown), etc.

[0156] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the subject matter disclosed herein does not encompass transmission media, carrier waves, or other transitory signals.

[0157] The computer system (1000) may also include a network interface (1054) to one or more communication networks (1055). The one or more communication networks (1055) may be, for example, wireless, wired, or optical. Furthermore, the one or more communication networks (1055) may be local, wide-area, metropolitan, vehicular, industrial, real-time, delay-tolerant, etc. Examples of the one or more networks (1055) include local area networks such as Ethernet, WLAN, cellular networks including GSM, 3G, 4G, 5G, LTE, etc.; TV wired or wireless wide-area digital networks including cable television, satellite television, and terrestrial broadcast television; vehicular, industrial, including CANBus, etc. Some networks typically require an external network interface adapter connected to some general data port or peripheral bus (1049) (e.g., a USB port on the computer system (1000)), while others are typically integrated into the core of the computer system (1000) by connecting to a system bus as described below (e.g., an Ethernet interface to a PC computer system or a cellular network interface to a smartphone computer system). Using any of these networks, the computer system (1000) can communicate with other entities. Such communication can be unidirectional, receive only (e.g., broadcast TV), unidirectional transmit only (e.g., a CANbus to some CANbus devices), or bidirectional, e.g., communication to other computer systems using local-area or wide-area digital networks. As noted above, several protocols and protocol stacks can be used with each of these networks and network interfaces.

[0158] The aforementioned human interface devices, human-accessible storage devices, and network interfaces may be connected to the core (1040) of the computer system (1000).

[0159] A core (1040) may include one or more central processing units (CPUs) (1041), graphics processing units (GPUs) (1042), specialized programmable processing units in the form of field programmable gate arrays (FPGAs) (1043), hardware accelerators for some tasks (1044), etc. These devices, along with read-only memory (ROM) (1045), random access memory (1046), and internal mass storage devices (1047) such as internal hard drives or SSDs that are not user-accessible, may be connected via a system bus (1048). In some computer systems, the system bus (1048) may be accessible in the form of one or more physical plugs, allowing expansion with additional CPUs, GPUs, etc. Peripheral devices may be connected directly to the core's system bus (1048) or via a peripheral bus (1049). Peripheral bus architectures include PCI, USB, etc.

[0160] The CPU (1041), GPU (1042), FPGA (1043), and accelerator (1044) can execute several instructions that, in combination, can constitute the aforementioned computer code. That computer code can be stored in ROM (1045) or RAM (1046). Transient data can also be stored in RAM (1046), while persistent data can be stored, for example, in internal mass storage (1047). The use of cache memory, which can be closely associated with one or more CPUs (1041), GPUs (1042), mass storage devices (1047), ROMs (1045), RAMs (1046), etc., allows for fast storage and retrieval from any memory device.

[0161] The computer-readable medium can have computer code thereon for performing various computer-implemented operations. The medium and computer code may be those specially designed and constructed for the purposes of the present disclosure, or they may be of the kind well known and available to those skilled in the computer software arts.

[0162] By way of example and not limitation, the architecture, particularly the computer system (1000) having the core (1040), can provide functionality as a result of a processor (including a CPU, GPU, FPGA, accelerator, etc.) executing software embodied in one or more tangible computer-readable media. Such computer-readable media can be the user-accessible mass storage devices introduced above, as well as media associated with some storage of the core (1040) that is non-transitory in nature, such as the core's internal mass storage device (1047) or ROM (1045). Software implementing various embodiments of the present disclosure can be stored in such devices and executed by the core (1040). The computer-readable media can include one or more memory devices or chips according to particular needs. The software can cause the core (1040) and particularly the processor (including a CPU, GPU, FPGA, etc.) therein to perform particular processes or particular portions of particular processes described herein, including defining data structures stored in RAM (1046) and modifying such data structures according to the software-defined processes. Additionally, or alternatively, a computer system may provide functionality as a result of logic hardwired or otherwise embedded in circuitry (e.g., accelerator (1044)) that can operate in place of or together with software to perform particular processes or portions of particular processes described herein. References to software can include logic, and vice versa, as appropriate. References to computer-readable media can encompass circuitry (such as an integrated circuit (IC)) that stores software for execution, circuitry embodying logic for execution, or both, as appropriate. The present disclosure encompasses any suitable combination of hardware and software.

[0163] While this disclosure has described several exemplary embodiments, there are alterations, permutations, and various substitute equivalents that fall within the scope of this disclosure. It will thus be appreciated that those skilled in the art will be able to devise numerous systems and methods that, although not explicitly shown or described herein, embody the principles of the present disclosure and are therefore within its spirit and scope.

[0164] Appendix A: Acronyms AMVP: Advanced Motion Vector Prediction ASIC: Application-Specific Integrated Circuit BMS:Benchmark Set BS:Boundary Strength BV: Block Vector CANBus: Controller Area Network Bus CD: Compact Disc CPR: Current Picture Referencing CPU: Central Processing Units CRT:Cathode Ray Tube CTB: Coding Tree Blocks CTU: Coding Tree Units CU: Coding Unit DPB: Decoder Picture Buffer DVD: Digital Video Disc FPGA: Field Programmable Gate Areas GOP:Groups of Pictures GPU: Graphics Processing Units GSM: Global System for Mobile communications (pan-European digital mobile communications system) HDR: High Dynamic Range HEVC: High Efficiency Video Coding HRD: Hypothetical Reference Decoder IBC: Intra Block Copy IC: Integrated Circuit JEM: Joint Exploration Model LAN: Local Area Network LCD: Liquid-Crystal Display LIC: Local Illumination Compensation LTE: Long-Term Evolution MR-SAD: Mean-Removed Sum of Absolute Difference MR-SATD: Mean-Removed Sum of Absolute (Hadamard-Transformed Difference) MV: Motion Vector OLED: Organic Light-Emitting Diode PB: Prediction Blocks PCI: Peripheral Component Interconnect PLD: Programmable Logic Device PPS: Picture Parameter Set PU: Prediction Units RAM: Random Access Memory ROM: Read-Only Memory SCC: Screen Content Coding SDR: Standard Dynamic Range SEI: Supplementary Enhancement Information SMVP: Spatial Motion Vector Predictor SNR: Signal Noise Ratio SPS: Sequence Parameter Set SSD: Solid-state drive TMVP: Temporal Motion Vector Predictor TU: Transform Units USB: Universal Serial Bus VUI: Video Usability Information VVC:Versatile Video Coding [Explanation of symbols]

[0165] 111 Target Block 121 Blocks in the same location 131 target blocks 141 Target Picture 142 target blocks 144 Sub-CU 149 Movement Shift 151 Co-located Pictures 152 Blocks in the same location 154 Sub-CU 200 Communication Systems 210 Terminal Devices 220 Terminal Devices 230 Terminal Devices 240 terminal devices 250 Communication Network 301 Video Sources 302 Video Picture Stream 303 Video Encoder 304 Encoded Video Data 305 Streaming Server 306 Client Subsystem 307 Copy 308 Client Subsystem 309 Incoming copy, video data 310 Video Decoder 311 Video Picture Output Stream 312 Display 313 Capture Subsystem 320 Electronic Devices 330 Electronic Devices 401 Channel 410 Video Decoder 412 Rendering Devices 415 Buffer Memory 420 Parser 421 Symbol 430 Electronic Devices 431 Receiver 451 Scaler / Descaler Unit 452 Intra-picture prediction unit, intra-prediction module 453 Motion Compensation Prediction Unit 455 Aggregator 456 Loop Filter Unit 457 Reference Picture Memory 458 Target Picture Buffer 501 Video Sources 503 Video Encoder 520 Electronic Devices 530 Source Coder 532 encoding engine 533 decoder 534 Reference Picture Memory 535 Predictor 540 Transmitter 543 coded video sequence 545 Entropy Coder 550 Controller 560 Communication Channels 603 Video Encoder 621 General Controller 622 Intra Encoder 623 Residual Calculator 624 Residual Encoder 625 Entropy Encoder 626 Switch 628 Residual Decoder 630 Interencoder 710 Video Decoder 771 Entropy Decoder 772 Intra Decoder 773 Residual Decoder 774 Reconstruction Module 780 Interdecoder 800 processes 900 processes 1000 Computer Systems and Architecture 1001 keyboard 1002 Mouse 1003 Trackpad 1005 Joystick 1006 Mike 1007 Scanner 1008 Camera 1009 Speaker 1010 Touch Screen 1020 CD / DVD ROM / RW 1021 CD / DVD 1022 thumb drive 1023 removable hard drive or solid state drive 1040 cores 1041 Central Processing Unit (CPU) 1042 Graphics Processing Unit (GPU) 1043 Field Programmable Gate Area (FPGA) 1044 Hardware Accelerator 1045 Read-Only Memory (ROM) 1046 Random Access Memory 1047 Core Internal Mass Storage 1048 System Bus 1049 Peripheral Bus 1050 graphics adapter 1054 network interface 1055 Communication Network

Claims

1. 1. A video decoding method performed by a decoder computer, comprising: decoding prediction information for a current block, a further division of a coding unit (CU), in a current picture that is part of a coded video sequence; determining whether the area of ​​the target block is greater than a threshold; if the area of ​​the current block is determined to be less than or equal to the threshold, constructing a first motion vector predictor list including a first number of motion vector predictors; if the area of ​​the current block is determined to be greater than the threshold, constructing a second motion vector predictor list including a second number of motion vector predictors; reconstructing the current block based on one of the first motion vector predictor list or the second motion vector predictor list; Including, the second motion vector predictor list includes a first type of motion vector predictor, or includes the first type of motion vector predictor and a second type of motion vector predictor, the first type of motion vector predictor being a spatial motion vector predictor or a temporal motion vector predictor, and the second type of motion vector predictor being a history-based motion vector predictor; the first motion vector predictor list includes only the second type of motion vector predictor; method.

2. If the prediction information indicates a partition size of the target block, the steps of constructing the first motion vector predictor list and constructing the second motion vector predictor list depend on whether the partition size is greater than or less than the threshold. The method of claim 1.

3. The method of claim 1 , wherein the threshold is a preset number of luminance samples, the preset number being 32 or 64.

4. The method described in claim 1, wherein the threshold corresponds to the picture resolution of the target block, and when the first picture resolution of a first target block is higher than the second picture resolution of a second target block, the first threshold of the first target block is greater than the second threshold of the second target block.

5. The method described in claim 1, wherein the threshold is signaled using a sequence parameter set (SPS), a picture parameter set (PPS), a slice or a tile header in the coded video sequence.

6. The method of claim 1 , wherein the first motion vector predictor list does not include spatial or temporal motion vector predictions.

7. 2. The method of claim 1, wherein a first number of redundancy checks are performed to construct the first motion vector predictor list including the first number of motion vector predictors, and a second number of redundancy checks are performed to construct the second motion vector predictor list including the second number of motion vector predictors, wherein the first number of redundancy checks is less than the second number of redundancy checks.

8. The method of claim 1 , wherein the motion vector predictor list including the first number of motion vector predictors is constructed without a redundancy check.

9. The method of any one of claims 1 to 8, wherein at least one of the motion vector predictors uses the current picture as a reference picture.

10. An apparatus for video decoding, configured to perform the method of any one of claims 1 to 9.

11. A computer program product for causing a processor of an apparatus for video decoding to perform the method of any one of claims 1 to 9.

12. 1. A video encoding method in an encoder for generating a video sequence, comprising: generating prediction information indicating a partition size of a coding unit (CU) in a current picture; Dividing the CU based on the prediction information to obtain a current block; determining whether the area of ​​the target block is greater than a threshold related to the area of ​​the target block; if the area of ​​the current block is determined to be less than or equal to the threshold, encoding the current block based on a first motion vector predictor list including a first number of motion vector predictors; if the area of ​​the current block is determined to be greater than the threshold, encoding the current block based on a second motion vector predictor list including a second number of motion vector predictors; obtaining the video sequence based on the encoded current block; including the threshold and the prediction information in the video sequence. Including, the second motion vector predictor list includes a first type of motion vector predictor, or includes the first type of motion vector predictor and a second type of motion vector predictor, the first type of motion vector predictor being a spatial motion vector predictor or a temporal motion vector predictor, and the second type of motion vector predictor being a history-based motion vector predictor; the first motion vector predictor list includes only the second type of motion vector predictor; method.

13. If the prediction information indicates a partition size of the target block, the step of encoding the current block based on the first motion vector predictor list and the step of encoding the current block based on the second motion vector predictor list depend on whether the partition size is greater than or less than the threshold. The method of claim 12.

14. The method of claim 12 , wherein the threshold is a preset number of luminance samples, the preset number being 32 or 64.

15. The method described in claim 12, wherein the threshold corresponds to the picture resolution of the target block, and if the first picture resolution of a first target block is higher than the second picture resolution of a second target block, the first threshold of the first target block is greater than the second threshold of the second target block.

16. The method of claim 12, wherein the threshold is signaled using a sequence parameter set (SPS), a picture parameter set (PPS), a slice or a tile header in the coded video sequence.

17. The method of claim 12 , wherein the first motion vector predictor list does not include a spatial or temporal motion vector predictor.

18. 13. The method of claim 12, wherein a first number of redundancy checks are performed to construct the first motion vector predictor list including the first number of motion vector predictors, and a second number of redundancy checks are performed to construct the second motion vector predictor list including the second number of motion vector predictors, the first number of redundancy checks being less than the second number of redundancy checks.

19. The method of claim 12 , wherein the motion vector predictor list including the first number of motion vector predictors is constructed without a redundancy check.

20. The method of any one of claims 12 to 19, wherein at least one of the motion vector predictors uses the current picture as a reference picture.

21. An apparatus for video coding, configured to perform the method according to any one of claims 12 to 20.

22. A computer program product for causing a processor of an apparatus for video encoding to perform the method of any one of claims 12 to 20.

23. A non-transitory computer-readable storage medium storing a video bitstream, wherein the bitstream is generated by a method according to any one of claims 12 to 20.

24. A method of video encoding in an encoder, comprising: generating a bitstream; storing the encoded bitstream; Equipped with The bitstream is generated by a method according to any one of claims 12 to 20. method.

Images