Method and apparatus for motion vector prediction

Abstract

The invention relates to encoding and decoding video modes. The motion vector is used to predict a current coding unit (CU) within a current video frame. The motion vector is from the generated motion vector candidate MVC list. The list includes MVs determined according to a first mode and / or a second mode, each mode specifying an MV candidate location. The first MV location is located within the current video frame and the second MV location is for a video frame that is different than the current frame. The MVC list is generated for a current coding unit (CU), wherein the MV position is relative to the position of the current CU. In particular, the MVC list is generated according to the size of the current CU and the size of the grid specifying the minimum distance between the two MV locations. One or more MVs specified by the MV location of the pattern are included in an MVC list.

Description

[0001] This application is a divisional application. The application number of the original application is 201980043454.8, and the original application date is July 1, 2019. The entire contents of the original application are incorporated into this application by reference. Technical Field

[0002] The present disclosure relates to coding and encoding of video images, and more particularly to motion prediction. Background Art

[0003] Digital video communication and storage applications are implemented by a wide variety of digital devices, such as digital cameras, cellular radiotelephones, laptops, broadcast systems, video teleconferencing systems, etc. One of the most important and challenging tasks in these applications is video compression. The task of video compression is complex and is constrained by two conflicting parameters: compression efficiency and computational complexity. Video coding standards such as ITU-T H.264 / AVC or ITU-TH.265 / HEVC provide a good compromise between these parameters. Therefore, almost all video compression applications must support video coding standards.

[0004] Video compression techniques (eg, inter-frame prediction, intra-frame prediction, and loop filters) have been proven to be effective and thus have been adopted by various video coding standards, such as H.264 / AVC and H.265 / HEVC.

[0005] The basic processing unit of video compression is usually called a macroblock, which is a part of a picture with a size of, for example, 16x16 or 64x64 pixels. In the latest video coding standard H.265 / HEVC, the so-called coding tree unit (CTU) is used as the basic processing unit. CTU is also called the largest coding unit (LCU). In H.265 / HEVC, CTUs are processed one by one in a scanning order from the upper left corner to the lower right corner of the picture. Each CTU can be further divided into smaller coding units (CU) by using, for example, quad-tree split. CU includes information related to inter-frame and intra-frame prediction information as well as residuals.

[0006] For example, in the H.265 / HEVC standard, an important part of inter-frame prediction is motion vector (MV) coding. Motion vectors are usually predicted and coded using the following two schemes:

[0007] 1. Construct a motion vector from a motion vector prediction value and the difference between the motion vector obtained through motion estimation processing and the prediction value.

[0008] This MV coding method in the HEVC standard is called advanced motion vector prediction.

[0009] 2. Derive motion vectors by selecting from a set of configurable candidates (predictors) without encoding the motion vector difference. This approach is called merge mode.

[0010] For both techniques, a large number of potential prediction candidates constructed from already encoded motion vectors can be considered. In the HEVC standard, there are four sets of motion vector predictors: spatial, temporal, combined bi-predictive, and zero candidates. During encoding, the best motion vector predictor is selected from a certain number of candidates and its index in the candidate list is written to the bitstream. An example of the location of the spatial MVP candidate (for merge mode) in the HEVC standard is shown in Figure 1. Figure 6 shown.

[0011] In the example given, the MVP candidates are denoted as A0, A1, B0, B1, and B2 respectively. i Indicates the predicted value on the left, position B j Indicates the prediction value above the current block. In this article, the current block is also referred to as the current coding unit (CU). It should be noted that in general, the candidate position may depend on the block coding order. Depending on the coding order, candidate positions can be selected from the top, left, right, and bottom of the neighboring blocks (i.e., blocks adjacent to the current block). In the HEVC standard, all spatial MVP candidates (for merge mode and for advanced motion vector prediction mode) belong to neighboring blocks. This means that these blocks share boundaries with the current block.

[0012] In the example given, the MVP candidates are denoted as A0, A1, B0, B1, and B2 respectively. i Indicates the predicted value on the left, position B j Indicates the prediction value above the current block. In this article, the current block is also referred to as the current coding unit (CU). It should be noted that in general, the candidate position may depend on the block coding order. Depending on the coding order, candidate positions can be selected from the top, left, right, and bottom of the neighboring blocks (i.e., blocks adjacent to the current block). In the HEVC standard, all spatial MVP candidates (for merge mode and for advanced motion vector prediction mode) belong to neighboring blocks. This means that these blocks share boundaries with the current block. Summary of the invention

[0013] Embodiments of the invention are defined by the features of the independent claims, further advantageous embodiments of the embodiments are defined by the features of the dependent claims.

[0014] The apparatus and method provided by the present disclosure are used to: (1) generate a motion vector candidate (MVCs) list according to a motion vector MV (LDMVP candidate) of a first mode and / or according to a MV (TMVP candidate) of a second mode, (2) add adjacent MV candidates to the LDMVP candidate, (3) generate a mode (i.e., a first mode and / or a second mode) relative to the position of the current CU according to the size of the current CU and the grid size. In the present disclosure, the terms "LDMVP / TMVP position", "first / second mode position", "LDMVP / TMVP mode position", "position of the first / second mode", and / or "position of the LDMVP / TMVP mode" are used synonymously.

[0015] According to one aspect of the present disclosure, a device for generating a motion vector candidate (MVCs) list for a current decoding unit (CU) is provided, wherein the device includes a processing circuit for including one or more motion vectors (MVs) determined according to a first mode and one or more MVs determined according to a second mode in the MVC list, wherein the first mode specifies the MV position within a current video frame where the current CU is located, and the second mode specifies the MV position in a video frame different from the current video frame.

[0016] The motion vector candidate is used for motion prediction of the current CU. This means that the motion information of the motion vector is used for prediction at the corresponding candidate position. Therefore, the motion vector position is the position of the corresponding vector candidate block. A block can be one pixel / sample and / or multiple samples corresponding to a sample block.

[0017] In other words, the location of a motion vector is the location within the video frame that specifies the pixel or pixels to which the motion is assigned.

[0018] The MV positions in the first mode are for the current video frame. This means that the first mode has candidate positions for which motion information is available and / or stored in a buffer. The mode that is the first mode is called the long-distance motion vector predictor (LDMVP) mode. Such a mode generates MV positions to capture the spatial aspects of the motion information around the current CU in the current frame that is available at the encoding / decoding time.

[0019] The MV positions in the second mode are for video frames that are different from the current frame. For example, the different frame may be a frame that is before the current frame (relative to the scanning order or encoding / decoding order). This means that the second mode has candidate positions in those picture areas for which there is no motion information in the current frame at the encoding / decoding moment of the current video frame, for which motion information is available and / or stored in the buffer. For example, in the case where the encoding / decoding order is from the upper left corner to the lower right corner of the current video frame, the picture area may be the lower right part of the picture. The mode as the second mode is called the temporal motion vector predictor (TMVP) mode. Such a mode generates MV positions to capture the temporal aspects of the motion information around the current CU in a frame that is different from the current frame.

[0020] Including MV candidate positions from the first mode and the second mode in the MVC list may provide the advantage that both spatial and temporal characteristics of motion information are considered. In addition, using the TMVP mode as the second mode may provide the advantage that motion information is obtained from certain areas of a previous video frame, which is not available in said areas of the current frame due to the scanning order of encoding / decoding.

[0021] Furthermore, since the first mode and the second mode have motion information from different picture areas (ie, areas where MV information is available and unavailable), motion prediction can be more accurate because motion information of the current frame and previous video frames are used.

[0022] The MV position of the first mode can refer to the position of the current CU. The current CU position can be the upper left corner, where the position is in pixels or pixel block units. For example, the block unit can be a 2x2 pixel block or any other unit. In particular, in VVC, a 4x4 grid is used to store motion information. In this case, the block unit is a 4x4 pixel block.

[0023] The MV position of the second mode may have the same reference position as the first MV mode position.Alternatively, the reference position of the TMVP mode may be determined based on some MV positions from one of the blocks neighboring the current CU.

[0024] In particular, the MV positions using a pattern (including the first pattern and / or the second pattern) provide the benefit that only a small number of MV positions from which motion information may be obtained and / or acquired are used. Therefore, the use of the pattern limits the number of MV position candidates, thereby requiring fewer read and write operations when accessing motion information. Therefore, the sparsity of the MV candidate positions reflected by the pattern makes prediction more efficient and fast.

[0025] The above-mentioned pattern can be any pattern suitable for determining motion information based on the motion vector at the pattern position. In other words, for example, the pattern can be adjusted according to its size and / or extension within the video frame relative to the current CU, and the pattern can be selected or designed to have a specific structure and / or shape.

[0026] For example, a pattern corresponding to the LDMVP pattern may be determined or generated such that the pattern position has the following characteristics: Figures 7 to 10 Another mode that is particularly suitable for motion information from the top of the CU is as follows: Fig.18 The first mode is not limited to Figures 7 to 10 or Fig.18 Patterns such as those shown in , may have any other shape / structure suitable for limiting the motion information obtained at pattern-based locations.

[0027] Therefore, using a mode (including a mode composed of a first mode and / or a second mode) can provide the advantage of making prediction more flexible, so that only motion information of a position limited by the mode position is used. This can include a mode position that can be adjacent to a CU and / or an adjusted MV position of the first mode.

[0028] According to one aspect of the present disclosure, the processing circuit of the above-mentioned device is used to include one or more MVs in the MVCs list, and the above-mentioned one or more MVs are determined according to a position adjacent to the current CU in the current video frame and different from the MV position of the first mode.

[0029] This means that the MVC list is extended by neighboring points that provide motion information and therefore help motion prediction. This can provide the advantage that the motion prediction is more accurate because the positions of neighboring points around the current CU of the current frame can be generated at distinct positions that are not covered by the first mode positions.

[0030] For example, Fig. 27 and Fig.28 As shown, adjacent points can be generated at positions near one or more edges of the current CU and included in the MVC list. In this case, the adjacent points are located at the left edge and the upper edge of the current CU, which are particularly suitable for encoding / decoding of the current video frame.

[0031] Therefore, including neighboring points in the MVC list allows for spatial refinement of the MVC list, which may improve the accuracy of motion prediction.

[0032] According to one aspect of the present disclosure, the processing circuit is used to determine the first mode including determining new MV positions for one or more MV positions that are unavailable and / or not stored in the buffer in the first mode, so that the new MV positions are available and / or stored in the buffer.

[0033] This means that some positions of the first pattern may be outside the current video frame, i.e., the position is larger than the size of the current frame. Therefore, motion information for this point does not exist. This may also limit the size / extension of the pattern when generating it.

[0034] Motion information may also not be available at the above-mentioned position in the first mode due to the encoding / decoding order performed for the CU of the current video frame. In other words, since the lower right region of the CTU is to be encoded / decoded later, at a particular moment, motion information for the lower right region does not yet exist. Therefore, even if a position in that region is generated (i.e., is part of the pattern), no motion information exists at that point at that moment.

[0035] Furthermore, because the motion information may be accessed, it may not be stored in a memory (i.e., a buffer). Such a buffer may be, for example, a row buffer, which has fast read and write access times and is typically used to store motion information for a restricted region of a CTU adjacent to a current CTU for which motion prediction is performed. A row buffer stores data that is quickly read and / or written in parallel for an entire cache line.

[0036] Positions that do not contain and / or store motion information in the first mode are discarded but otherwise used as a basis for determining new MV positions such that the new MV positions have motion information available and / or stored in a buffer.

[0037] This may provide the advantage that, since the new MV positions are determined based on the discarded MV positions, the new MV positions provide motion information and still contain the spatial information of the original first mode.

[0038] Furthermore, discarding positions from the first pattern may provide the advantage that increasing the size of the first pattern by inapplicable MV positions is avoided. This reduces the storage of the first pattern.

[0039] According to one aspect of the present disclosure, determining the new MV position includes assigning an x-position and / or a y-position of one or more MV positions to the new MV position and discarding the one or more MV positions from the first pattern.

[0040] Thus, the position assignment from the discarded position to the new MV position may be performed relative to any of the x and / or y position components of the discarded MV position. In other words, the position assignment may be performed in any direction.

[0041] This may provide the advantage that the position allocation is performed in a very flexible manner, allowing fine tuning of the MV candidate positions of the first mode. Thus, the prediction accuracy may be further improved due to the spatial fine tuning of the MV candidate positions.

[0042] Furthermore, discarding unavailable motion information MV positions in the first mode may provide the advantage of removing MV positions from the first mode that are unsuitable for motion prediction and / or motion vector prediction due to having no motion information available and / or stored in the buffer.

[0043] According to one aspect of the present disclosure, the determining includes assigning a new y position or a new x position to one or more new MV positions having the same x position or the same y position using a predetermined scheme.

[0044] The above scheme may be a function. This means, for example, that using the function, with the position of the allocation as input, a new y position and / or x position of the allocation can be calculated.

[0045] According to one aspect of the present disclosure, the predetermined scheme is to shift the assigned y position or x position of the new MV position by a predetermined offset value.

[0046] The offset value may be a number of pixels, such that the y position and / or x position (e.g., in pixels) are shifted by the pixel offset by adding and / or subtracting the offset value from the adjusted y position and / or x position. Alternatively, the predetermined scheme may be a factor that may be multiplied and / or divided by the y position and / or x position. Other schemes may also be provided to achieve reallocation of the allocated positions when the x position and / or y position of the allocated positions are the same.

[0047] Thus, the reallocation of positions (e.g., by shifting any of the x-components or y-components of the allocated positions) spreads out those allocated positions that may have the same x-position and / or y-position. In other words, after the position allocation, some of the discarded positions may be located at the same position. The component-wise reallocation of the allocated positions that have the same x-position or y-position is to spread out these positions to different positions that are still available and / or stored in the buffer.

[0048] This may provide the advantage that position redundancy is avoided, since some candidate positions may be located at the same position and thus provide the same motion information. Thus, dispersing the allocation positions increases the number of candidate positions, which may have different motion information since they are located at different positions. Thus, prediction may be performed more accurately using a sufficient number of MV candidate positions.

[0049] According to one aspect of the present disclosure, determining the second mode includes reflecting one or more MV positions of the first mode with reference to the reflected line.

[0050] A reflection line refers to a line passing through two (pixel) positions of the current video frame, which divides the frame (e.g., the current frame) into two regions, a region of available motion information (in scanning order or encoding / decoding order) and a region of unavailable motion information. This means, for example, that according to a common encoding / decoding order, the upper left portion of a picture / frame contains available motion information, while the lower right picture portion has no available motion information.

[0051] Determining the second pattern by mirroring one or more MV positions of the first pattern may provide the advantage that the second pattern is generated with low complexity.

[0052] According to one aspect of the present disclosure, the processing circuit is used to generate a pattern relative to the position of the current CU based on: the size of the current CU; a grid size that specifies the minimum distance between two MV positions belonging to the pattern; wherein the pattern specifies the position of the motion vector (MV), and the mode is the first mode and / or the second mode.

[0053] The size of the current CU may be the width or height of the CU. The grid size may be the minimum distance between MV positions of a mode (ie, the first mode or the second mode) along the x-axis or y-axis given by the corresponding grid intervals Gx and Gy.

[0054] According to one aspect of the present disclosure, the processing circuit is used to determine the second mode and / or the first mode based on at least one of the following: an iteration value specifying the size of the second mode and / or the first mode on the grid; the minimum size that can be configured for the CU; and the maximum size that can be configured for the CU.

[0055] Generating a mode (including a first mode and / or a second mode) based on CU size, grid parameters, and / or iteration values ​​can provide the advantage that a mode of arbitrary size and / or shape can be generated in a flexible manner for a current CU in a current frame (first mode) and / or for a previous frame (second mode). In other words, the mode can be adjusted to be optimal for motion prediction. This means, for example, that the size and / or shape of the first mode can be optimized for the spatial motion characteristics of the current frame (LDMVP mode), while the size and / or shape of the second mode can be optimized for the temporal characteristics of the previous frame (TMVP mode).

[0056] According to one aspect of the present disclosure, a device is provided for generating a list of motion vector candidates (MVCs) for a current coding unit (CU), the device comprising a processing circuit for determining a pattern specifying a position of a motion vector relative to a position of a current CU based at least on: a size of the current CU; a grid size specifying a minimum distance between two MV positions belonging to the pattern; and for including one or more MVs specified by the MV positions of the pattern in the MVC list.

[0057] According to one aspect of the present disclosure, the processing circuit is configured to determine a mode position according to a first iteration value and / or a minimum or maximum size that can be configured for a CU.

[0058] The first iteration value (or generally, the iteration value) determines the pattern size. The pattern size refers to the extension of the pattern position in terms of its maximum distance relative to the current CU position. In the present disclosure, the current CU uses the upper left position of the current CU as the reference position.

[0059] The above distances can be defined in pixel units. The above distances can be calculated using, for example, the Euler metric or any other metric suitable for distance. The size of the pattern can be determined by the grid size. The grid size refers to the minimum distance between the positions of the MVs belonging to the pattern.

[0060] The determined mode may be a spatial mode (ie, LDMVP mode) and / or a temporal mode (ie, TMVP mode). The mode defines one or more positions of MV candidates around the current CU and contains a limited number of candidate positions, so that the mode may be sparse.

[0061] Due to the sparsity of the pattern, using the pattern to include one or more MVs in the pattern in the MVC list based on the pattern candidate position can provide the advantage that fewer read and write operations are required when accessing motion information. Therefore, the sparsity of the MV candidate positions reflected by the pattern makes prediction more efficient and faster.

[0062] According to one aspect of the present disclosure, the processing circuit is configured to adapt the grid size according to the size of the current CU and / or the maximum and minimum values ​​of the grid size.

[0063] Generating a mode (including a first mode and / or a second mode) according to a CU size, a grid parameter, and / or a (first) iteration value may provide the advantage of generating a mode of arbitrary size and / or shape for the current CU in a flexible manner. In other words, the mode may be adjusted to be optimal for motion prediction. This means, for example, that the size and / or shape of the first mode may be optimized for the spatial motion characteristics of the current frame (LDMVP mode), while the size and / or shape of the second mode may be optimized for the temporal characteristics of the previous frame (TMVP mode).

[0064] According to one aspect of the present disclosure, the processing circuit is used to determine a neighboring position that specifies a MV position different from the first pattern within the current video frame, and to combine the neighboring position with the first pattern.

[0065] Determining neighboring MV positions different from the first mode and combining them with the first mode may provide the advantage of spatially fine-tuning the first mode, thereby improving the accuracy of motion prediction, because the positions of the neighboring points may be specifically selected, for example, in those areas around the current CU that are particularly suitable for motion prediction but not covered by the first mode. Fig. 27 and Fig.28 Examples of such neighboring points are shown in , which are located at the left and top edges of the current CU.

[0066] Once the motion vector candidate positions are determined using any of the above embodiments of the present disclosure, it is checked whether the motion vectors of these candidate positions have been included in the MVC list. If not, the motion vector is included in the list.

[0067] According to one aspect of the present disclosure, an encoder is provided for encoding a current decoding unit (CU) of a current video frame, the encoder comprising: a device for generating a list of one or more MVCs according to any of the above aspects of the present disclosure; an inter-frame prediction unit for determining a prediction of a current CU based on at least one motion vector (MV) in the MVC list; and a compression unit for encoding the current CU by using the prediction of the CU.

[0068] According to one aspect of the present disclosure, a decoder is provided for decoding a current coding unit (CU) of a video frame, the decoder comprising: a device for generating a list of one or more MVCs according to any of the above aspects of the present disclosure; an inter-frame prediction unit for determining a prediction of a current CU based on at least one motion vector (MV) in the MVC list; and a decompression unit for decoding the current CU by using the prediction of the CU.

[0069] Using the motion information in the motion vector obtained at the MVC position, MVCs can be used for motion prediction. The current CU can be encoded and / or decoded by calculating the difference between the CU and the prediction. Alternatively or additionally, the index of the list of MVCs can be encoded, for example, even if there is no residual because the prediction is already close to the CU (i.e. the residual is almost zero). In principle, the index of the list can also be determined implicitly on the encoder side and the decoder side.

[0070] According to one aspect of the present disclosure, a method for generating a motion vector candidate list for a current decoding unit (CU) is provided, the method comprising the following steps: including the following in an MVC list: one or more motion vectors (MVs) determined according to a first mode, the first mode specifying the MV position within a current video frame where the current CU is located; one or more MVs determined according to a second mode, the second mode specifying the MV position in a video frame different from the current video frame.

[0071] According to one aspect of the present disclosure, a method for generating a list of motion vector candidates (MVCs) for a current coding unit (CU) is provided, the method comprising the following steps: determining a pattern specifying a position of a motion vector (MV) relative to a position of a current CU based at least on: a size of the current CU; a grid size specifying a minimum distance between two MV positions belonging to the pattern; and including one or more motion vectors specified by the MV positions of the pattern in the list.

[0072] According to one aspect of the present disclosure, there is provided a computer-readable non-transitory medium for storing a program, the medium comprising instructions, which when executed on a processor, causes the processor to perform method steps according to any of the aforementioned methods.

[0073] The details of one or more embodiments are set forth in the drawings and the description below. Other features, objects, and advantages will be apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] In the following, embodiments of the present invention are described in more detail with reference to the accompanying drawings, in which:

[0075] Figure 1A A block diagram showing an example of a video encoding system for implementing an embodiment of the present invention;

[0076] Figure 1B A block diagram showing another example of a video encoding system for implementing an embodiment of the present invention;

[0077] Figure 2 A block diagram showing an example of a video encoder for implementing an embodiment of the present invention;

[0078] Figure 3 A block diagram showing an example structure of a video decoder for implementing an embodiment of the present invention;

[0079] Figure 4 A block diagram showing an example of an encoding device or a decoding device;

[0080] Figure 5 A block diagram showing another example of an encoding device or a decoding device;

[0081] Figure 6 is an example of the location of spatial MVP candidates (for merge mode) in the HEVC standard;

[0082] Figure 7 is an example of an LDMVP pattern called a "square" LDMVP pattern;

[0083] Figure 8 is an example of an LDMVP pattern called a "diagonal" LDMVP pattern;

[0084] Fig. 9 is an example of an LDMVP pattern called a "triangular" LDMVP pattern;

[0085] Fig.10 is an example of an LDMVP pattern called a "star" LDMVP pattern;

[0086] Fig.11 Flowchart of the algorithm for generating the LDMVP list;

[0087] Fig.12 This is a flow chart of the PatternGeneration program;

[0088] Fig.13 is an example of the LDMVP "sun" pattern generated by the PatternGeneration procedure with three iterations R=3;

[0089] Fig.14 is an example of the LDMVP “sun” pattern generated by the PatternGeneration procedure with four iterations R=4;

[0090] Fig.15 is an example of a candidate order at iteration 1 for the "sun" pattern position generated by the PatternGeneration procedure with R=3;

[0091] Fig.16 is an example of a candidate order at iteration 3 for the "sun" pattern position generated by the PatternGeneration procedure with R=3;

[0092] Fig.17 This is the flowchart of the GeneratePatternTMVP program;

[0093] Fig.18 is an example of a pattern composed of three iterations of LDMVP mode position R=3 and one iteration of TMVP mode position Rt=1;

[0094] Fig.19is an example of a pattern composed of three iterations of LDMVP mode position R=3 and two iterations of TMVP mode position Rt=2;

[0095] Fig. 20 is a second example of a mode composed of three iterations of LDMVP mode position R=3 and two iterations of TMVP mode position Rt=2;

[0096] Fig.21 is a third example of a mode composed of three iterations of LDMVP mode position R=3 and two iterations of TMVP mode position Rt=2;

[0097] Fig. 22 An example of candidate order for LDMVP+TMVP mode with R=3 and Rt=1;

[0098] Fig.23 is a first example of a combination mode including only TMVP candidates;

[0099] Fig.24 is a second example of a combination mode including only TMVP candidates;

[0100] Fig.25 is a TMVP pattern generated by mirroring the LDMVP pattern with R=3;

[0101] Fig.26 is a TMVP pattern generated by mirroring a portion of the LDMVP pattern of R=3;

[0102] Fig. 27 is an example of a modified LDMVP mode by adding six neighboring candidate points;

[0103] Fig.28 is an example of a modified LDMVP mode by adding four neighboring candidate points;

[0104] Fig.29 is an example of a combination mode, the combination mode includes an LDMVP mode, a TMVP mode, and adjacent candidate points;

[0105] Fig.30 is an example of an LDMVP mode called a "sun" LDMVP mode and allowed regions R0 and R1;

[0106] Fig.31 An example of dividing the "Sun" LDMVP pattern into different classes;

[0107] Fig.32 To show an example of the application of the adjustCandidatePositions procedure on the "sun" mode;

[0108] Fig.33 This is the flowchart of the program adjustCandidatePositions;

[0109] Fig.34 This is the flowchart of the program additionalChecks;

[0110] Fig.35 Flowchart of the algorithm for generating the LDMVP list modified by the program additionalChecks;

[0111] Fig.36 To illustrate a first example of application of the additionalChecks program on the "diagonal" mode;

[0112] Fig.37 To illustrate a second example of the application of the additionalChecks program on the "diagonal" mode;

[0113] Fig.38 To illustrate the third example of the application of the additionalChecks program on the "diagonal" mode;

[0114] Fig.39 To illustrate the fourth example of the application of the additionalChecks program on the "diagonal" mode;

[0115] Fig.40 This is the flowchart of the program replacePosition;

[0116] Fig.41 Flowchart of the algorithm for generating the LDMVP list modified by the procedure replacePosition;

[0117] Fig.42 A fifth example showing the application of the replacePosition procedure on the "diagonal" mode;

[0118] Fig.43 To illustrate the sixth example of the application of the replacePosition procedure on the "diagonal" mode;

[0119] Fig.44 A seventh example showing the application of the replacePosition procedure on the "diagonal" mode;

[0120] Fig.45 To illustrate the eighth example of the application of the replacePosition procedure on the "diagonal" mode;

[0121] In the following, identical reference numerals denote identical or at least functionally equivalent features, unless explicitly stated otherwise. DETAILED DESCRIPTION

[0122] In the following description, reference is made to the accompanying drawings, which form a part of the present disclosure and show by way of illustration specific aspects of embodiments of the invention or in which specific aspects of embodiments of the invention may be used. It should be understood that embodiments of the invention may be used in other aspects and include structural or logical changes not shown in the accompanying drawings. Therefore, the following specific embodiments should not be considered restrictive, and the scope of the invention is defined by the appended claims.

[0123] For example, it should be understood that the disclosure related to the described method may also be true for the corresponding device or system configured to perform the method, and vice versa. For example, if one or more specific method steps are described, the corresponding device may include one or more units (e.g., functional units) to perform the one or more method steps (e.g., one unit performs one or more steps, or multiple units each perform one or more of the multiple steps), even if such one or more units are not explicitly described or shown in the drawings. On the other hand, for example, if a specific device is described based on one or more units (e.g., functional units), the corresponding method may include a step that performs the function of one or more units (e.g., a step that performs the function of one or more units, or multiple steps that perform the functions of one or more units respectively), even if such one or more steps are not explicitly described or shown in the drawings. In addition, it should be understood that unless otherwise specifically noted, the features of the various exemplary embodiments and / or aspects described herein may be combined with each other.

[0124] Video coding and decoding generally refers to the processing of a sequence of pictures that form a video or a video sequence. Instead of the term "picture", the terms "frame" or "image" can be used as synonyms in the field of video coding and decoding. Video coding and decoding (or in general, coding and decoding) includes two parts, video encoding and video decoding. Video encoding is performed on the source side and generally includes processing (e.g., by compression) the original video picture to reduce the amount of data required to represent the video picture (for more efficient storage and / or transmission). Video decoding is performed on the destination side and generally includes inverse processing relative to the encoder to reconstruct the video picture. Embodiments of "coding and decoding" involving video pictures (or in general, pictures) should be understood as involving "encoding" or "decoding" of video pictures or corresponding video sequences. The combination of the encoding part and the decoding part is also called CODEC (coding and decoding).

[0125] In the case of lossless video codecs, the original video pictures can be reconstructed, i.e., the reconstructed video images have the same quality as the original video pictures (assuming no transmission losses or other data losses caused by storage or transmission). In the case of lossy video codecs, further compression (e.g., by quantization) is performed to reduce the amount of data representing video pictures that cannot be fully reconstructed at the decoder, i.e., the quality of the reconstructed video pictures is lower or inferior compared to the quality of the original video pictures.

[0126] Several video codec standards belong to the group of "lossy hybrid video codecs" (i.e., combining spatial and temporal prediction in the sample domain and 2D transform coding in the transform domain for applying quantization). Each picture of a video sequence is typically divided into a set of non-overlapping blocks, and the coding is typically performed at the block level. In other words, at the encoder, the video is typically processed (i.e., encoded) at the block (video block) level, for example, by using spatial (intra-picture) prediction and / or temporal (inter-picture) prediction to generate a prediction block, subtracting the prediction block from the current block (currently processed / to be processed block) to obtain a residual block, transforming the residual block and quantizing the residual block in the transform domain to reduce the amount of data to be sent (compression), while at the decoder, the inverse process relative to the encoder is applied to the coded block or compressed block to reconstruct the current block for presentation. In addition, the encoder replicates the decoder processing loop so that both will generate the same prediction (e.g., intra-frame prediction and inter-frame prediction) and / or reconstruction to process (i.e., codec) subsequent blocks.

[0127] In the following examples, based on Figures 1A to 3 A video coding system 10, a video encoder 20, and a video decoder 30 are described.

[0128] Figure 1A 1 is a schematic block diagram illustrating an example codec system 10 that can utilize the techniques of the present application, such as a video codec system 10 (or short codec system 10). A video encoder 20 (or short encoder 20) and a video decoder 30 (or short decoder 30) of the video codec system 10 represent example devices that can be used to perform techniques according to various examples described herein.

[0129] like Figure 1A As shown, the codec system 10 includes a source device 12 for providing encoded picture data 21 to a destination device 14 for decoding the encoded picture data 13, for example.

[0130] The source device 12 includes an encoder 20 , and may additionally (ie, optionally) include a picture source 16 , a pre-processor (or pre-processing unit) 18 (eg, picture pre-processor 18 ), and a communication interface or communication unit 22 .

[0131] The image source 16 may include or may be any type of image capture device (e.g., a camera for capturing real-world images) and / or any type of image generation device (e.g., a computer graphics processor for generating computer-animated images) or any type of other device for obtaining and / or providing real-world images, computer-generated images (e.g., screen content, virtual reality (VR) images), and / or combinations thereof (e.g., augmented reality (AR) images). The image source may be any type of memory or storage device that stores any of the above images.

[0132] In distinction from the pre-processor 18 and the processing performed by the pre-processing unit 18 , the picture or picture data 17 may also be referred to as a raw picture or raw picture data 17 .

[0133] The preprocessor 18 is used to receive (raw) picture data 17 and to perform preprocessing on the picture data 17 to obtain a preprocessed picture 19 or preprocessed picture data 19. The preprocessing performed by the preprocessor 18 may include, for example, cropping, color format conversion (e.g., from RGB to YCbCr), color correction, or denoising. It is understood that the preprocessing unit 18 may be an optional component.

[0134] The video encoder 20 is used to receive the pre-processed picture data 19 and provide the encoded picture data 21 (hereinafter, for example, based on Figure 2 Describes further details).

[0135] The communication interface 22 of the source device 12 may be used to receive the encoded picture data 21 and to send the encoded picture data 21 to another device (eg, the destination device 14 or any other device) via the communication channel 13 for storage or direct reconstruction.

[0136] The destination device 14 includes a decoder 30 (eg, a video decoder 30 ), and may additionally (ie, optionally) include a communication interface or communication unit 28 , a post-processor 32 (or a post-processing unit 32 ), and a display device 34 .

[0137] The communication interface 28 of the destination device 14 is used to receive the encoded picture data 21 (or any other processed version of the encoded picture data), for example directly from the source device 12 or from any other source (such as a storage device, such as an encoded picture data storage device), and provide the encoded picture data 21 to the decoder 30.

[0138] The communication interface 22 and the communication interface 28 can be used to send or receive the encoded picture data 21 or the encoded data 13 via a direct communication link (e.g., a direct wired connection or a wireless connection) between the source device 12 and the destination device 14 or any type of network (e.g., a wired network or a wireless network or any combination of the above, or any type of private network and public network, or any combination of the above).

[0139] The communication interface 22 may be used, for example, to package the encoded picture data 21 into a suitable format (eg, packets) and / or process the encoded picture data using any type of transmission encoding or processing for transmission over a communication link or communication network.

[0140] The communication interface 28 forms a corresponding part of the communication interface 22 and may, for example, be used to receive transmitted data and process the transmitted data using any type of corresponding transmission decoding or processing and / or de-packaging to obtain the encoded picture data 21 .

[0141] Both the communication interface 22 and the communication interface 28 may be configured as follows: Figure 1A The unidirectional communication interface shown by the arrow in the communication channel 13 pointing from the source device 12 to the destination device 14 may be configured as a bidirectional communication interface and may be used, for example, to send and receive messages, such as to establish a connection, to respond or to exchange any other information related to the communication link and / or data transmission (e.g., encoded picture data transmission).

[0142] The decoder 30 is used to receive the encoded picture data 21 and provide decoded picture data 31 or decoded picture 32 (hereinafter, for example, based on Figure 3 or Figure 5 Describes further details).

[0143] The post-processor 32 of the destination device 14 is used to post-process the decoded picture data 31 (also called reconstructed picture data) (e.g., decoded picture 31) to obtain post-processed picture data 33, such as post-processed picture 33. The post-processing performed by the post-processing unit 32 may include, for example, color format conversion (e.g., from YCbCr to RGB), color correction, cropping, or resampling, or any other processing to prepare the decoded picture data 31 for display, such as by a display device 34.

[0144] The display device 34 of the destination device 14 is used to receive the post-processed image data 33 to display the image to, for example, a user or viewer. The display device 34 can be or include any type of display for presenting the reconstructed image, such as an integrated or external display or display screen. The display can include, for example, a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro LED display, a liquid crystal on silicon (LCoS), a digital light processor (DLP), or any other type of display.

[0145] although Figure 1A The source device 12 and the destination device 14 are depicted as separate devices, but embodiments of the device may also include both devices or the functionality of both devices, the source device 12 or the corresponding functionality and the destination device 14 or the corresponding functionality. In such embodiments, the source device 12 or the corresponding functionality and the destination device 14 or the corresponding functionality may be implemented using the same hardware and / or software or by separate hardware and / or software or any combination thereof.

[0146] Based on the description, it is obvious to those skilled in the art that Figure 1A The presence and (precise) division of different units or functionalities within the source device 12 and / or destination device 14 shown may vary depending on the actual device and application.

[0147] The encoder 20 (eg, video encoder 20) or the decoder 30 (eg, video decoder 30) or both the encoder 20 and the decoder 30 may be configured to receive the video signal via a processor such as a decoder. Figure 1B The processing circuit shown is implemented, and the processing circuit can be one or more microprocessors, digital signal processors (digital signal processors, DSP), application-specific integrated circuits (application-specific integrated circuits, ASIC), field-programmable gate arrays (field-programmable gate arrays, FPGA), discrete logic, hardware, dedicated video encoding, or any combination thereof. Encoder 20 can be implemented via processing circuit 46 to implement reference Figure 2 The various modules discussed in the encoder 20 and / or any other encoder system or subsystem described herein. The decoder 30 may be implemented via the processing circuit 46 to implement the reference Figure 3The various modules discussed in the decoder 30 of the present invention and / or any other decoder system or subsystem described herein. The processing circuitry may be used to perform various operations as discussed below. Figure 5 As shown, if the technology is partially implemented in software, the device can store the instructions of the software in a suitable non-transitory computer-readable storage medium, and can use one or more processors to execute the instructions in hardware to perform the technology of the present disclosure. For example, one of the video encoder 20 and the video decoder 30 can be integrated as shown. Figure 1B Part of a combined encoder / decoder (CODEC) in a single device is shown.

[0148] Source device 12 and destination device 14 may include any of a variety of devices, including any kind of handheld or fixed device, such as a notebook or laptop computer, a mobile phone, a smart phone, a tablet or tablet computer, a camera, a desktop computer, a set-top box, a television, a display device, a digital media player, a video game console, a video streaming device (such as a content service server or a content delivery server), a broadcast receiver device, a broadcast transmitter device, etc., and may not use an operating system or use any type of operating system. In some cases, source device 12 and destination device 14 may be equipped for wireless communication. Therefore, source device 12 and destination device 14 may be wireless communication devices.

[0149] In some cases, Figure 1A The video codec system 10 shown is only an example, and the technology of the present application can be applied to video codec settings (e.g., video encoding or video decoding) that do not necessarily include any data communication between the encoding device and the decoding device. In other examples, data is extracted from a local memory, transmitted through a network stream, etc. The video encoding device can encode and store data to a memory, and / or the video decoding device can read and decode data from a memory. In some examples, encoding and decoding are performed by devices that do not communicate with each other but only encode data to a memory and / or read and decode data from a memory.

[0150] For ease of description, embodiments of the present invention are described herein, for example, by referring to reference software of high-efficiency video coding (HEVC) or versatile video coding (VVC), the next generation video coding standard developed by the video coding experts group (VCEG) of ITU-T and the video coding joint working group (JCT-VC) of ISO / IEC motion picture experts group (MPEG). It will be understood by those skilled in the art that embodiments of the present invention are not limited to HEVC or VVC.

[0151] Encoders and encoding methods

[0152] Figure 2 An example block diagram of an example video encoder 20 for implementing the techniques of the present application is shown. Figure 2 In the example of , the video encoder 20 includes an input 201 (or an input interface 201), a residual calculation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, and an inverse transform processing unit 212, a reconstruction unit 214, a loop filter unit 220, a decoded picture buffer (DPB) 230, a mode selection unit 260, an entropy coding unit 270, and an output 272 (or an output interface 272). The mode selection unit 260 may include an inter-frame prediction unit 244, an intra-frame prediction unit 254, and a segmentation unit 262. The inter-frame prediction unit 244 may include a motion estimation unit and a motion compensation unit (not shown). As shown in FIG. Figure 2 The illustrated video encoder 20 may also be referred to as a hybrid video encoder or a video encoder according to a hybrid video codec.

[0153] The residual calculation unit 204, the transform processing unit 206, the quantization unit 208, and the mode selection unit 260 may be referred to as forming a forward signal path of the encoder, and the inverse quantization unit 210, the inverse transform processing unit 212, the reconstruction unit 214, the buffer 216, the loop filter 220, the decoded picture buffer (DBP) 230, the inter-frame prediction unit 244, and the intra-frame prediction unit 254 may be referred to as forming a reverse signal path of the video encoder 20, wherein the reverse signal path of the video encoder 20 corresponds to the signal path of the decoder (see Figure 3The inverse quantization unit 210, the inverse transform processing unit 212, the reconstruction unit 214, the loop filter 220, the decoded picture buffer (DPB) 230, the inter prediction unit 244, and the intra prediction unit 254 are also referred to as a “built-in decoder” forming the video encoder 20.

[0154] Pictures and picture segmentation (pictures and blocks)

[0155] The encoder 20 may be configured to receive, for example, via an input 201, a picture 17 (or picture data 17), such as a picture in a sequence of pictures forming a video or a video sequence. The received picture or picture data may also be a pre-processed picture 19 (or pre-processed picture data 19). For simplicity, the following description refers to picture 17. Picture 17 may also be referred to as a current picture and a picture to be encoded (particularly in video encoding and decoding, to distinguish the current picture from other pictures, such as previously encoded and / or decoded pictures in the same video sequence, i.e., a video sequence including the current picture).

[0156] A (digital) picture is or can be considered to be a two-dimensional array or matrix of samples with intensity values. The samples in the array can also be called pixels (short form of picture elements) or pel. The number of samples in the horizontal and vertical directions (or axes) of the array or picture defines the size and / or resolution of the picture. In order to represent color, three color components are usually used, that is, the picture can be represented by or include three sample arrays. In the RGB format or color space, the picture includes corresponding red, green and blue sample arrays. However, in video coding, each pixel is usually represented by a brightness and chrominance format or color space, such as YCbCr, which includes a brightness component indicated by Y (sometimes L is also used instead) and two chrominance components indicated by Cb and Cr. The brightness (or luma for short) component Y represents the brightness or grayscale intensity (for example, as in a grayscale picture), while the two chrominance (or chroma for short) components Cb and Cr represent the chrominance or color information components. Accordingly, a picture in the YCbCr format includes a luma sample array of luma sample values ​​(Y) and two chroma sample arrays of chroma values ​​(Cb and Cr). A picture in the RGB format can be converted or transformed to the YCbCr format, and vice versa, a process also referred to as color conversion or conversion. If the picture is monochrome, the picture may include only luma sample values. Accordingly, for example, a picture may be a luma sample array in a monochrome format or a luma sample array and two corresponding chroma sample arrays in 4:2:0, 4:2:2, and 4:4:4 color formats.

[0157] An embodiment of the video encoder 20 may include a picture segmentation unit ( Figure 2), the picture segmentation unit is used to divide the picture 17 into multiple (usually non-overlapping) picture blocks 203. These blocks can also be called root blocks, macroblocks (H.264 / AVC), or coding tree blocks (CTB) or coding tree units (CTU) (H.265 / HEVC and VVC). The picture segmentation unit can be used to use the same block size and a corresponding grid defining the block size for all pictures in the video sequence, or to change the block size between pictures or groups of pictures or subsets of pictures, and divide each picture into corresponding blocks.

[0158] In other embodiments, the video encoder may be configured to directly receive the block 203 of the picture 17, for example, forming one, several, or all blocks of the picture 17. The picture block 203 may also be referred to as a current picture block or a picture block to be encoded.

[0159] As with picture 17, picture block 203 is or can be considered as a two-dimensional array or matrix of samples with intensity values ​​(sample values), but with smaller dimensions than picture 17. In other words, block 203 may, for example, include one sample array (e.g., a luminance array in the case of monochrome picture 17, or a luminance or chrominance array in the case of a color picture) or three sample arrays (e.g., one luminance array and two chrominance arrays in the case of a color picture 17) or any other number and / or kind of arrays, depending on the color format applied. The number of samples in the horizontal and vertical directions (or axes) of block 203 defines the size of block 203. Accordingly, the block may, for example, be an MxN (M columns by N rows) sample array or an MxN array of transform coefficients.

[0160] like Figure 2 The illustrated embodiment of the video encoder 20 may be used to encode the picture 17 on a block-by-block basis, for example, performing encoding and prediction by block 203 .

[0161] like Figure 2 The illustrated embodiment of the video encoder 20 may also be used to segment and encode a picture using slices (also referred to as video slices), wherein a picture may be divided into or encoded using one or more slices (typically non-overlapping), and each slice may include one or more blocks (e.g., CTUs).

[0162] like Figure 2The illustrated embodiment of the video encoder 20 may also be used to segment and / or encode a picture using tile groups (also referred to as video tile groups) and / or tiles (also referred to as video tiles), wherein a picture may be divided into or encoded using one or more tile groups (typically non-overlapping), and each tile group may, for example, include one or more blocks (e.g., CTUs) or one or more tiles, wherein each tile may, for example, be rectangular and may include one or more blocks (e.g., CTUs), such as complete or partial blocks.

[0163] Residual calculation

[0164] The residual calculation unit 204 can be used to calculate the residual block 205 (also referred to as residual 205) based on the picture block 203 and the prediction block 265 (other details about the prediction block 265 are provided later), for example, by subtracting the sample values ​​of the prediction block 265 from the sample values ​​of the picture block 203 sample by sample (pixel by pixel) to obtain the residual block 205 in the sample domain.

[0165] Transform

[0166] The transform processing unit 206 may be configured to apply a transform (e.g., discrete cosine transform (DCT) or discrete sine transform (DST)) to the sample values ​​of the residual block 205 to obtain transform coefficients 207 in the transform domain. The transform coefficients 207 may also be referred to as transform residual coefficients and represent the residual block 205 in the transform domain.

[0167] The transform processing unit 206 can be used to apply an integer approximation of the DCT / DST, such as the transform specified by H.265 / HEVC. Such integer approximations are typically scaled by a specific factor compared to an orthogonal DCT transform. In order to preserve the norm of the residual blocks processed by the forward and inverse transforms, additional scaling factors are used as part of the transform process. The scaling factors are typically selected based on certain constraints, such as whether the scaling factor is a power of 2 for the shift operation, the bit depth of the transform coefficients, a trade-off between accuracy and implementation cost, etc. For example, a specific scaling factor can be specified for the inverse transform performed by the inverse transform processing unit 212 (and, for example, the corresponding inverse transform performed by the inverse transform processing unit 312 at the video decoder 20), and a corresponding scaling factor for the forward transform performed by the transform processing unit 206 at the encoder 20 can be specified accordingly.

[0168] Embodiments of the video encoder 20 (respectively the transform processing unit 206) may be configured to output transform parameters, such as the type of transform, directly or via the entropy coding unit 270 after encoding or compression, so that, for example, the video decoder 30 may receive and use the transform parameters for decoding.

[0169] Quantification

[0170] The quantization unit 208 may be configured to quantize the transform coefficients 207, for example by applying scalar quantization or vector quantization, to obtain quantized coefficients 209. The quantized coefficients 209 may also be referred to as quantized transform coefficients 209 or quantized residual coefficients 209.

[0171] The quantization process may reduce the bit depth associated with some or all of the transform coefficients 207. For example, during quantization, an n-bit transform coefficient may be rounded down to an m-bit transform coefficient, where n is greater than m. The degree of quantization may be modified by adjusting a quantization parameter (QP). For example, for scalar quantization, different scaling ratios may be applied to achieve finer or coarser quantization. A smaller quantization step size corresponds to a finer quantization, while a larger quantization step size corresponds to a coarser quantization. A quantization parameter (QP) may indicate an applicable quantization step size. For example, a quantization parameter may be an index of a predefined set of applicable quantization step sizes. For example, a small quantization parameter may correspond to fine quantization (small quantization step size), while a large quantization parameter may correspond to coarse quantization (large quantization step size), or vice versa. Quantization may include dividing by a quantization step size, and corresponding and / or inverse dequantization (e.g., performed by the inverse quantization unit 210) may include multiplying by a quantization step size. Embodiments according to some standards (e.g., HEVC) may be used to use a quantization parameter to determine a quantization step size. In general, the quantization step size can be calculated based on the quantization parameter using a fixed point approximation of an equation involving division. Additional scaling factors can be introduced for quantization and dequantization to recover the norm of the residual block, which may be modified due to the use of scaling in the fixed point approximation of the equations for the quantization step size and the quantization parameter. In an example embodiment, the scaling and dequantization of the inverse transform can be combined. Alternatively, a custom quantization table can be used and sent from the encoder to the decoder, for example in a bitstream. Quantization is a lossy operation, where the loss increases with the increase of the quantization step size.

[0172] Embodiments of video encoder 20 (respectively quantization unit 208 ) may be configured to output a quantization parameter (QP) directly or after encoding via entropy encoding unit 270 , so that, for example, video decoder 30 may receive and apply the quantization parameter for decoding.

[0173] Inverse Quantization

[0174] The inverse quantization unit 210 is used to apply the inverse quantization scheme of the quantization scheme applied by the quantization unit 208 to the quantized coefficients to obtain dequantized coefficients 211 by applying the inverse scheme of the quantization scheme applied by the quantization unit 208 based on or using the same quantization step size as the quantization unit 208. The dequantized coefficients 211 may also be referred to as dequantized residual coefficients 211, and correspond to the transform coefficients 207 (although they are generally not equal to the transform coefficients due to the loss of quantization).

[0175] Inverse Transform

[0176] The inverse transform processing unit 212 is used to apply the inverse transform (e.g., inverse discrete cosine transform or inverse discrete sine transform or other inverse transform) of the transform applied by the transform processing unit 206 to obtain a reconstructed residual block 213 (or corresponding dequantized coefficients 213) in the sample domain. The reconstructed residual block 213 may also be referred to as a transform block 213.

[0177] Refactoring

[0178] The reconstruction unit 214 (e.g., an adder or summer 214) is used to add the transform block 213 (i.e., the reconstructed residual block 213) to the prediction block 265, for example by adding the sample values ​​of the reconstructed residual block 213 to the sample values ​​of the prediction block 265 (sample by sample) to obtain the reconstructed block 215 in the sample domain.

[0179] Filtering

[0180] The loop filter unit 220 (or simply "loop filter" 220) is used to filter the reconstructed block 215 to obtain a filter block 221, or generally, to filter the reconstructed samples to obtain filtered samples. The loop filter unit is used, for example, to smooth pixel transitions or otherwise improve video quality. The loop filter unit 220 may include one or more loop filters, such as a de-blocking filter, a sample-adaptive offset (SAO) filter, or one or more other filters, such as a bilateral filter, an adaptive loop filter (ALF), a sharpening, a smoothing filter or a collaborative filter, or any combination of the above. Although Figure 2 The loop filter unit 220 is shown as an in loop filter, but in other configurations, the loop filter unit 220 may be implemented as a post loop filter. The filter block 221 may also be referred to as a filter reconstruction block 221 .

[0181] Embodiments of the video encoder 20 (respectively the loop filter unit 220) may be configured to output loop filter parameters (e.g., sample adaptive offset information), e.g., directly or after encoding via the entropy encoding unit 270, so that, e.g., the decoder 30 may receive and apply the same loop filter parameters or corresponding loop filter for decoding. Decoded Picture Buffer

[0182] The decoded picture buffer (DPB) 230 may be a memory for storing reference pictures (or, in general, reference picture data) for encoding video data by the video encoder 20. The DPB 230 may be formed by any of a variety of storage devices, such as a dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magneto-resistive RAM (MRAM), resistive RAM (RRAM), or other types of storage devices. The decoded picture buffer (DPB) 230 may be used to store one or more filter blocks 221. The decoded picture buffer 230 may also be used to store other previous filter blocks of the same current picture or a different picture (e.g., a previously reconstructed picture), such as a previous reconstruction and filter block 221, and may provide a complete previously reconstructed (i.e., decoded) picture (and corresponding reference blocks and samples) and / or a partially reconstructed current picture (and corresponding reference blocks and samples), such as for inter-frame prediction. The decoded picture buffer (DPB) 230 may also be used to store one or more non-filtered reconstructed blocks 215 (or, in general, non-filtered reconstructed samples) or any other further processed versions of the reconstructed blocks or samples, for example when the reconstructed blocks 215 are not filtered by the loop filter unit 220.

[0183] Mode selection (segmentation and prediction)

[0184] The mode selection unit 260 includes a segmentation unit 262, an inter-frame prediction unit 244, and an intra-frame prediction unit 254, and is used to receive or obtain original picture data and reconstructed picture data, such as the original block 203 (current block 203 of the current picture 17), from the decoded picture buffer 230 or other buffers (such as line buffers, not shown), for example. The original picture data is, for example, the original block 203 (current block 203 of the current picture 17), and the reconstructed picture data is, for example, the same (current) picture and / or filtered and / or non-filtered reconstructed samples or blocks from one or more previously decoded pictures. The reconstructed picture data is used as reference picture data for prediction (such as inter-frame prediction or intra-frame prediction) to obtain a prediction block 265 or a prediction value 265.

[0185] The mode selection unit 260 can be used to determine or select partitions and prediction modes (such as intra-frame prediction mode or inter-frame prediction mode) for the current block prediction mode (including non-partitioning) and generate a corresponding prediction block 265, which is used to calculate the reference block 205 and to reconstruct the reconstruction block 215.

[0186] Embodiments of the mode selection unit 260 may be used to select a partition and a prediction mode (e.g., from modes supported by the mode selection unit 260 or available for the mode selection unit 260) that provide the best match, or in other words, provide the smallest residual (the smallest residual means better compression for transmission or storage), or the smallest signaling overhead (the smallest signaling overhead means better compression for transmission or storage), or both. The mode selection unit 260 may be used to determine the partition and prediction mode based on rate distortion optimization (RDO), i.e., select the prediction mode that provides the smallest rate distortion. In this document, the terms "best", "minimum", "optimal", etc. do not necessarily mean "best", "minimum", "optimal" as a whole, but may refer to achieving a termination or selection criterion, such as a value exceeding or falling below a threshold, or obtaining a "suboptimal selection" but reducing other constraints in complexity and processing time.

[0187] In other words, the partitioning unit 262 can be used to, for example, iteratively use quad-tree partitioning (QT), binary partitioning (BT), or triple-tree partitioning (TT) or any combination of the above to divide the block 203 into smaller block partitions or sub-blocks (which form a block), and to, for example, perform prediction for each block partition or sub-block, wherein the mode selection includes selecting a tree structure of the partitioned block 203, and the prediction mode is applied to each block partition or sub-block.

[0188]

[0046] Below, the segmentation (eg, performed by segmentation unit 260) and prediction processes (performed by inter-prediction unit 244 and intra-prediction unit 254) performed by example video encoder 20 will be explained in more detail.

[0189] segmentation

[0190] The segmentation unit 262 can segment (or divide) the current block 203 into smaller partitions, such as smaller blocks of squares or rectangles. These smaller blocks (which can also be called sub-blocks) can be further segmented into smaller partitions. This is also called tree-partitioning or hierarchical tree-partitioning, where, for example, a root block located at root tree level 0 (hierarchical level 0, depth 0) can be recursively segmented, for example, into two or more blocks of the next lower tree level, such as nodes located at tree level 1 (hierarchical level 1, depth 1), where these blocks can be segmented again into two or more blocks of the next lower level, such as tree level 2 (hierarchical level 2, depth 2), etc., until the segmentation is terminated, for example, due to meeting a termination criterion, such as reaching a maximum tree depth or a minimum block size. Blocks that are not further segmented are also called leaf-blocks or leaf nodes of the tree. A tree that is split into two partitions is called a binary tree (BT), a tree that is split into three partitions is called a ternary tree (TT), and a tree that is split into four partitions is called a quadtree (QT).

[0191] As described above, the term "block" used herein may be a portion of a picture, in particular a square or rectangular portion. For example, with reference to HEVC and VVC, a block may be or correspond to a coding tree unit (CTU), a coding unit (CU), a prediction unit (PU), and a transform unit (TU) and / or correspond to a corresponding block, such as a coding tree block (CTB), a coding block (CB), a transform block (TB), or a prediction block (PB).

[0192] For example, a coding tree unit (CTU) may be or include a CTB of luma samples, two corresponding CTBs of chroma samples, or a CTB of samples of a monochrome picture or a picture encoded using three independent color planes and a syntax structure for encoding samples. Accordingly, a coding tree block (CTB) may be an NxN block of samples for some value of N, such that the division of the constituent parts into CTBs is a single partition. A coding unit (CU) may be or include a coding block of luma samples, two corresponding coding blocks of chroma samples, or a coding block of samples of a monochrome picture or a picture encoded using three independent color planes and a syntax structure for encoding samples. Accordingly, a coding block (CB) may be an MxN block of samples for some values ​​of M and N, such that the division of the CTB into coding blocks is a single partition.

[0193] In an embodiment, for example, according to HEVC, a coding tree unit (CTU) may be divided into CUs by using a quadtree structure represented as a coding tree. It is decided at the CU level whether to encode a picture area using inter-picture (temporal) prediction or intra-picture (spatial) prediction. Depending on the PU partition type, each CU may be further divided into one, two, or four PUs. Within a PU, the same prediction process is applied, and relevant information is sent to the decoder based on the PU. After obtaining a residual block by applying a prediction process based on the PU partition type, the CU may be partitioned into transform units (TUs) according to another quadtree structure similar to the coding tree for the CU.

[0194] In an embodiment, for example, according to the latest video coding standard currently under development (called universal video coding (VVC)), a combined quad-tree and binarytree (quad-tree and binarytree, QTBT) partitioning is used to partition the coding block. In the QTBT block structure, the CU is either square or rectangular. For example, the coding tree unit (CTU) is first partitioned by a quadtree structure. The leaf nodes of the quadtree are further partitioned by a binary tree or a ternary tree structure. The leaf nodes of the partitioned tree are called decoding units (CUs), and this subdivision is used for prediction and transform processing, and no further partitioning is performed. This means that in the QTBT coding block structure, CU, PU, ​​and TU have the same block size. At the same time, multiple partitions (such as ternary tree partitions) can be used with the QTBT block structure.

[0195] In one example, mode select unit 260 of video encoder 20 may be used to perform any combination of the segmentation techniques described herein.

[0196] As described above, the video encoder 20 is configured to determine or select an optimal or best prediction mode from a (eg, predefined) prediction mode set, which may include, for example, an intra-frame prediction mode and / or an inter-frame prediction mode.

[0197] Intra prediction

[0198] The intra-frame prediction mode set may include, for example, 35 different intra-frame prediction modes defined in HEVC, such as non-directional modes (such as DC (or average) mode and planar mode) or directional modes, or may include, for example, 67 different intra-frame prediction modes defined in VVC, such as non-directional modes (such as DC (or average) mode and planar mode) or directional modes.

[0199] The intra prediction unit 254 is configured to generate an intra prediction block 265 according to an intra prediction mode in an intra prediction mode set, using reconstructed samples of neighboring blocks of the same current picture.

[0200] The intra-frame prediction unit 254 (or generally, the mode selection unit 260) is also used to output intra-frame prediction parameters (or generally, information indicating the selected intra-frame prediction mode for the block) to the entropy encoding unit 270, and the intra-frame prediction parameters are included in the encoded picture data 21 in the form of syntax elements 266, so that, for example, the video decoder 30 can receive and use the prediction parameters for decoding.

[0201] Inter prediction

[0202] The set of inter prediction modes (or possible inter prediction modes) depends on the available reference pictures (i.e., previously at least partially decoded pictures stored in the DBP 230, for example) and other inter prediction parameters, such as, for example, whether to use the entire reference picture or only a portion of the reference picture (e.g., a search window area around the area of ​​the current block) to search for the best matching reference block, and / or, for example, whether to apply pixel interpolation, such as half-pixel and / or quarter-pixel interpolation.

[0203] In addition to the above prediction modes, a skip mode and / or a direct mode may also be applied.

[0204] The inter-frame prediction unit 244 may include a motion estimation (ME) unit and a motion compensation (MC) unit ( Figure 2(not shown in the figure). The motion estimation unit may be configured to receive or obtain a picture block 203 (a current picture block 203 of a current picture 17) and a decoded picture 231 or at least one or more previously reconstructed blocks (e.g., reconstructed blocks of one or more other / different previously decoded pictures 231) for motion estimation. For example, a video sequence may include a current picture and a previously decoded picture 231, or in other words, the current picture and the previously decoded picture 231 may be (or form) part of a picture sequence forming a video sequence.

[0205] For example, the encoder 20 may be configured to select a reference block from a plurality of reference blocks in the same or different pictures in a plurality of other pictures, and provide the reference picture (or reference picture index) and / or the offset (spatial offset) between the position (x, y coordinates) of the reference block and the position of the current block as an inter-frame prediction parameter to the motion estimation unit. The offset is also referred to as a motion vector (MV).

[0206] The motion compensation unit is used to obtain (e.g., receive) inter-frame prediction parameters, and to perform inter-frame prediction based on or using the inter-frame prediction parameters to obtain an inter-frame prediction block 265. The motion compensation performed by the motion compensation unit may involve obtaining or generating a prediction block based on a motion / block vector determined by motion estimation, possibly performing interpolation with sub-pixel accuracy. Interpolation filtering may generate additional pixel samples from known pixel samples, thereby potentially increasing the number of candidate prediction blocks that can be used to encode the picture block. Upon receiving the motion vector of the PU of the current picture block, the motion compensation unit may place the prediction block pointed to by the motion vector in one of the reference picture lists.

[0207] The motion compensation unit may also generate syntax elements associated with blocks and video slices for use by video decoder 30 in decoding picture blocks of a video slice. As an additional option or alternative to slices and corresponding syntax elements, partition groups and / or partitions and corresponding syntax elements may be generated or used.

[0208] Entropy Coding

[0209] The entropy coding unit 270 is used to apply an entropy coding algorithm or method (e.g., variable length coding (VLC), context adaptive VLC scheme (CAVLC), arithmetic coding scheme, binarization, context adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding or another entropy coding method or technique) to the quantization coefficients 209, inter-frame prediction parameters, intra-frame prediction parameters, loop filter parameters, and / or other syntax elements, or bypass (not compress) these parameters to obtain coded picture data 21, which can be output via output 272 in the form of a coded bitstream 21, so that, for example, the video decoder 30 can receive and use these parameters for decoding. The coded bitstream 21 can be sent to the video decoder 30, or stored in a memory for subsequent transmission or extraction by the video decoder 30.

[0210] Other structural variations of the video decoder 20 may be used to encode the video stream. For example, for some blocks or frames, the non-transform based encoder 20 may directly quantize the residual signal without the transform processing unit 206. In another embodiment, the encoder 20 may combine the quantization unit 208 and the inverse quantization unit 210 into a single unit.

[0211] Decoder and decoding method

[0212] Figure 3 An example of a video decoder 30 for implementing the techniques of the present application is shown. The video decoder 30 is configured to receive coded picture data 21 (e.g., coded bitstream 21), for example, encoded by an encoder 20, to obtain a decoded picture 331. The coded picture data or bitstream includes information for decoding the coded picture data, for example, data representing picture blocks (and / or partition groups or partitions) of a coded video slice and associated syntax elements.

[0213] exist Figure 3In the example of FIG. 3 , decoder 30 includes an entropy decoding unit 304, an inverse quantization unit 310, an inverse transform processing unit 312, a reconstruction unit 314 (e.g., a summer 314), a loop filter 320, a decoded picture buffer (DBP) 330, a mode application unit 360, an inter-frame prediction unit 344, and an intra-frame prediction unit 354. Inter-frame prediction unit 344 may be or include a motion compensation unit. In some examples, video decoder 30 may perform the same as relative Figure 2 The encoding channel of the video encoder 100 is reciprocal to the decoding channel.

[0214] As described with respect to the encoder 20, the inverse quantization unit 210, the inverse transform processing unit 212, the reconstruction unit 214, the loop filter 220, the decoded picture buffer (DPB) 230, the inter-prediction unit 344, and the intra-prediction unit 354 are also referred to as forming a “built-in decoder” of the video encoder 20. Accordingly, the inverse quantization unit 310 may be functionally equivalent to the inverse quantization unit 110, the inverse transform processing unit 312 may be functionally equivalent to the inverse transform processing unit 212, the reconstruction unit 314 may be functionally equivalent to the reconstruction unit 214, the loop filter 320 may be functionally equivalent to the loop filter 220, and the decoded picture buffer 330 may be functionally equivalent to the decoded picture buffer 230. Therefore, the description provided for the various units and functions of the video encoder 20 applies accordingly to the various units and functions of the video decoder 30.

[0215] Entropy decoding

[0216] The entropy decoding unit 304 is used to parse the bitstream 21 (or in general, the encoded picture data 21) and, for example, perform entropy decoding on the encoded picture data to obtain, for example, quantized coefficients 309 and / or decoded encoding parameters ( Figure 3 The entropy decoding unit 304 may be used to apply a decoding algorithm or scheme corresponding to the encoding scheme described for the entropy encoding unit 270 of the encoder 20. The entropy decoding unit 304 may also be used to provide the inter-frame prediction parameters, intra-frame prediction parameters, and / or other syntax elements to the mode application unit 360, and to provide other parameters to other units of the decoder 30. The video decoder 30 may receive syntax elements at the video slice level and / or the video block level. In addition to (or as an alternative to) slices and corresponding syntax elements, block groups and / or blocks and corresponding syntax elements may also be received and / or used.

[0217] Inverse Quantization

[0218] The inverse quantization unit 310 may be configured to receive a quantization parameter (QP) (or, in general, information related to inverse quantization) and quantization coefficients from the encoded picture data 21 (e.g., parsed and / or decoded by the entropy decoding unit 304), and to apply inverse quantization to the decoded quantization coefficients 309 based on the quantization parameter to obtain dequantized coefficients 311, which may also be referred to as transform coefficients 311. The inverse quantization process may include using a quantization parameter determined by the video encoder 20 for each video block in a video slice (or partition or group of partitions) to determine a degree of quantization, and likewise a degree of inverse quantization that should be applied.

[0219] Inverse Transform

[0220] The inverse transform processing unit 312 can be used to receive the dequantized coefficients 311 (also referred to as transform coefficients 311) and to apply a transform to the dequantized coefficients 311 to obtain a reconstructed residual block 213 in the sample domain. The reconstructed residual block 213 can also be referred to as a transform block 313. The transform can be an inverse transform, such as an inverse DCT, an inverse DST, an inverse integer transform, or a conceptually similar inverse transform process. The inverse transform processing unit 312 can also be used to receive transform parameters or corresponding information from the encoded picture data 21 (e.g., parsed and / or decoded by the entropy decoding unit 304) to determine the transform to be applied to the dequantized coefficients 311.

[0221] Refactoring

[0222] The reconstruction unit 314 (eg, adder or summer 314) may be configured to add the reconstructed residual block 313 to the prediction block 365, eg, by adding sample values ​​of the reconstructed residual block 313 to sample values ​​of the prediction block 365, to obtain a reconstructed block in the sample domain.

[0223] Filtering

[0224] The loop filter unit 320 (in the encoding loop or after the encoding loop) is used to filter the reconstruction block 315 to obtain a filter block 321, for example to smooth pixel transitions or improve video quality. The loop filter unit 320 may include one or more loop filters, such as a deblocking filter, a sample adaptive offset (SAO) filter, or one or more other filters, such as a bilateral filter, an adaptive loop filter (ALF), a sharpening, a smoothing filter, or a collaborative filter, or any combination thereof. Although Figure 3 The loop filter unit 320 is shown as an in-loop filter, but in other configurations, the loop filter 320 may be implemented as a post-loop filter.

[0225] Decoded image buffer

[0226] The decoded video block 321 of the picture is then stored in a decoded picture buffer 330, which stores the decoded picture 331 as a reference picture for subsequent motion compensation and / or output of other pictures for display.

[0227] The decoder 30 is configured to output a decoded picture 311 via an output 312 for presentation to or viewing by a user.

[0228] predict

[0229] The inter-frame prediction unit 344 can be functionally equivalent to the inter-frame prediction unit 244 (especially equivalent to the motion compensation unit), and the intra-frame prediction unit 354 can be functionally equivalent to the inter-frame prediction unit 254, and performs partitioning or partitioning decisions and predictions based on corresponding information and partitioning and / or prediction parameters received from the encoded picture data 21 (e.g., parsed and / or decoded by the entropy decoding unit 304). The mode application unit 360 can be used to perform prediction (intra-frame prediction or inter-frame prediction) by block based on the reconstructed picture, block, or corresponding samples (filtered or unfiltered) to obtain a prediction block 365.

[0230] When the video slice is encoded as an intra-coded (I) slice, the intra-prediction unit 354 of the mode application unit 360 is used to generate a prediction block 365 for a picture block of the current video slice based on the indicated intra-prediction mode and data from a previously decoded block of the current picture. When the video picture is encoded as an inter-coded (i.e., B or P) slice, the inter-prediction unit 344 (e.g., a motion compensation unit) of the mode application unit 360 is used to generate a prediction block 365 for a video block of the current video slice based on a motion vector and other semantic elements received from the entropy decoding unit 304. For inter-prediction, a prediction block may be generated from one of the reference pictures in one of the reference picture lists. The video decoder 30 may construct the reference frame lists, List 0 and List 1, using a default construction technique based on the reference pictures stored in the DPB 330. In addition to or as an alternative to slices (e.g., video slices), the same or similar aspects can be applied to embodiments using block groups (e.g., video block groups) and / or blocks (e.g., video blocks), for example, video can be encoded and decoded using I, P, or B block groups and / or blocks.

[0231] The mode application unit 360 is used to determine the prediction information of the video block of the current video slice by parsing the motion vector or related information and other semantic elements, and use the prediction information to generate a prediction block for the current video block being decoded. For example, the mode application unit 360 uses some received semantic elements to determine the prediction mode (e.g., intra-frame prediction or inter-frame prediction) for encoding and decoding the video block of the video slice, the inter-frame prediction slice type (e.g., B slice, P slice, or GPB slice), the construction information of one or more reference picture lists of the slice, the motion vector of each inter-frame coded video block of the slice, the inter-frame prediction state of each inter-frame coded video block of the slice, and other information to decode the video block in the current video slice. In addition to slices (e.g., video slices) or as an alternative to slices, the same or similar aspects can be applied to embodiments using block groups (e.g., video block groups) and / or blocks (e.g., video blocks), for example, I, P, or B block groups and / or blocks can be used to encode and decode video.

[0232] like Figure 3 The illustrated embodiment of the video decoder 30 may be used to partition and / or decode a picture using slices (also referred to as video slices), wherein a picture may be divided into or decoded using one or more slices (typically non-overlapping), and each slice may include one or more blocks (e.g., CTUs).

[0233] like Figure 3 The illustrated embodiment of the video decoder 30 may be used to segment and / or decode a picture using partition groups (also referred to as video partition groups) and / or blocks (also referred to as video blocks), wherein a picture may be divided into one or more partition groups (typically non-overlapping) or decoded using one or more partition groups, and each partition group may, for example, include one or more blocks (e.g., CTUs) or one or more partitions, wherein each block may, for example, be rectangular and may include one or more blocks (e.g., CTUs), such as complete or partial blocks.

[0234] Other variations of the video decoder 30 may be used to decode the encoded picture data 21. For example, the decoder 30 may generate an output video stream without using the loop filter unit 320. For example, a non-transform based decoder 30 may directly inverse quantize the residual signal for certain blocks or frames without using the inverse transform processing unit 321. In another embodiment, the video decoder 30 may combine the inverse quantization unit 310 and the inverse transform processing unit 312 into a single unit.

[0235] It should be understood that the processing result of the current step can be further processed and then output to the next step in the encoder 20 and the decoder 30. For example, after interpolation filtering, motion vector derivation, or loop filtering, further operations such as cropping or shifting can be performed on the processing results of interpolation filtering, motion vector derivation, or loop filtering.

[0236] It should be noted that further operations may be applied to the motion vector derived from the current block (including but not limited to the control point motion vector in affine mode, the sub-block motion vector in affine, planar, ATMVP mode, the temporal motion vector, etc.). For example, the value of the motion vector is constrained to a predefined range based on its representative bit. If the representative bit of the motion vector is bit depth (bitDepth), the range is -2^(bitDepth-1) to 2^(bitDepth-1)-1, where "^" represents exponentiation. For example, if bitDepth is set equal to 16, the range is -32768 to 32767; if bitDepth is set equal to 18, the range is -131072 to 131071. For example, the value of the derived motion vector (e.g., the MV of four 4x4 sub-blocks within an 8x8 block) is constrained so that the maximum difference between the integer parts of the four 4x4 sub-block MVs is no more than N pixels, for example, no more than one pixel. Two methods for constraining motion vectors based on bitDepth are provided here.

[0237] Method 1: Remove the overflow MSB (most significant bit) by doing the following:

[0238] ux=(mvx+2 bitDepth ) % 2 bitDepth (1)

[0239] mvx=(ux>= 2 bitDepth-1 ) ? (ux - 2 bitDepth ) : ux (2)

[0240] uy=(mvy+2 bitDepth ) % 2 bitDepth (3)

[0241] mvy = (uy>= 2 bitDepth-1 ) ? (uy - 2 bitDepth ) : uy (4)

[0242] Wherein, mvx is the horizontal component of the motion vector of the image block or sub-block, mvy is the vertical component of the motion vector of the image block or sub-block, and ux and uy indicate intermediate values;

[0243] For example, if the value of mvx is -32769, after applying formulas (1) and (2), the value obtained is 32767. In computer systems, decimal numbers are stored as two's complement. The two's complement of -32769 is 1,0111,1111,1111,1111 (17 bits), and then the MSB is discarded, and the two's complement obtained is 0111,1111,1111,1111 (decimal number is 32767), which is the same as the output of applying formulas (1) and (2).

[0244] ux= (mvpx + mvdx +2 bitDepth ) % 2 bitDepth (5)

[0245] mvx = (ux>= 2 bitDepth-1 ) ? (ux - 2 bitDepth ) : ux (6)

[0246] uy= (mvpy + mvdy +2 bitDepth ) % 2 bitDepth (7)

[0247] mvy = (uy>= 2 bitDepth-1 ) ? (uy - 2 bitDepth ) : uy (8)

[0248] As shown in equations (5) to (8), the above operations can be applied during the summation of mvp and mvd.

[0249] Method 2: Remove the overflow MSB by shaving the following value

[0250] vx=Clip3(-2 bitDepth-1 ,2 bitDepth-1 -1,vx)

[0251] vy=Clip3(-2 bitDepth-1 ,2 bitDepth-1 -1,vy)

[0252] Where vx is the horizontal component of the motion vector of the image block or sub-block, vy is the vertical component of the motion vector of the image block or sub-block; x, y, and z correspond to the three input values ​​of the MV reduction process, respectively. The function Clip3 is defined as follows:

[0253]

[0254] Figure 44 is a schematic diagram of a video codec device 400 according to an embodiment of the present disclosure. The video codec device 400 is suitable for implementing the disclosed embodiments described herein. In an embodiment, the video codec device 400 may be a Figure 1A A decoder of the video decoder 30, or a decoder such as Figure 1A The video encoder 20 is an encoder.

[0255] The video codec device 400 includes an inlet port 410 (or input port 410) and a receiving unit (Rx) 420 for receiving data; a processor, a logic unit, or a central processing unit (CPU) 430 for processing data; a transmitter unit (Tx) and an outlet port 450 (output port 450) for sending data; and a memory 460 for storing data. The video codec device 400 may also include an optical-to-electrical (OE) component and an electrical-to-optical (EO) component coupled to the inlet port, the receiver unit 420, the transmitter unit 440, and the outlet port 450 for input and output of optical or electrical signals.

[0256] The processor 430 can be implemented by hardware and software. The processor 430 can be implemented as one or more CPU chips, cores (e.g., multi-core processors), FPGAs, ASICs, and DSPs. The processor 430 communicates with the inlet port 410, the receiver unit 420, the transmitter unit 440, the outlet port 450, and the memory 460. The processor 430 includes a codec module 470. The codec module 470 implements the disclosed embodiments described herein. For example, the codec module 470 implements, processes, prepares, or provides various codec operations. Therefore, including the codec module 470 provides substantial improvements to the functions of the video codec device 400, and realizes the conversion of the video codec device 400 to different states. Alternatively, the codec module 470 is implemented as an instruction stored in the memory 460 and executed by the processor 430.

[0257] The memory 460 may include one or more disks, tape drives, and solid-state drives, and may be used as an overflow data storage device to store programs when a program is selected for execution, and to store instructions and data read during program execution. The memory 460 may be, for example, volatile and / or nonvolatile, and may be a read-only memory (ROM), a random access memory (RAM), a ternary content-addressable memory (TCAM), and / or a static random-access memory (SRAM).

[0258] Figure 5 is a simplified block diagram of an apparatus 500 according to an exemplary embodiment, which may be used as Figure 1A and Figure 1B The source device 12 and / or destination device 14 in.

[0259] The processor 502 in the apparatus 500 may be a central processing unit. Alternatively, the processor 502 may be any other type of device or devices capable of operating or processing existing information or later developed information. Although the disclosed embodiments may be practiced using a single processor (e.g., processor 502) as shown, advantages in speed and efficiency may be achieved using more than one processor.

[0260] In an embodiment, the memory 504 in the apparatus 500 may be a read-only memory (ROM) device or a random access memory (RAM) device. Any other suitable type of storage device may be used as the memory 504. The memory 504 may include code and data 506 accessed by the processor 502 using a bus 512. The memory 504 may also include an operating system 508 and application programs 510. The application programs 510 include at least one program that allows the processor 502 to perform the methods described herein. For example, the application programs 510 may include application programs 1 to N, and application programs 1 to N also include a video codec application that performs the methods described herein.

[0261] The apparatus 500 may also include one or more output devices, such as a display 518. In one example, the display 518 may be a touch-sensitive display that combines a display with a touch-sensitive element operable to sense touch input. The display 518 may be connected to the processor 502 via the bus 512.

[0262] Although described here as a single bus, the bus 512 of the device 500 may be composed of multiple buses. In addition, the auxiliary memory 514 may be directly coupled to other components of the device 500, or may be accessed via a network, and may include a single integrated unit (e.g., a memory card) or multiple units (e.g., multiple memory cards). The device 500 may therefore be implemented in a variety of configurations.

[0263] In the proposals for the VVC standard, such as JVET-J0012, JVET-J0021, JVET-J0029, JVET-J0058, JVET-J0059, it has been proposed to use motion information from non-adjusted blocks. In the present disclosure, the spatial MVP candidates captured from such blocks are represented as long-distance motion vector prediction (LDMVP) candidates. The positions of LDMVP candidates are usually obtained using certain patterns.

[0264] Generating the LDMVP list requires access to a large amount of motion information of non-adjusted decoding units. This may be critical in cases where the MVP location is outside the current CTU. In practice, the information of the currently processed CTU may be stored in a cache, while the previously processed CTU may be stored in an external memory. Therefore, extracting information from another CTU other than the current CTU may require access to an external memory, which increases latency and power consumption. In addition, extracting motion information from, for example, the upper CTU may be more complex than capturing motion information from the left CTU, which is the previous CTU in processing order and may also be stored in a cache.

[0265] For these reasons, it may be advantageous to modify the LDMVP mode so as to restrict access to other CTUs.

[0266] Depending on the encoding / decoding order, at the moment of encoding / decoding the current video frame, some motion information may not be available in some parts of the current video frame.

[0267] For this reason, it may be advantageous to use motion information from a video frame different from the current frame that has been encoded / decoded before the current frame, so that motion information is also available at those positions where the current frame does not provide such motion information.

[0268] The corresponding candidate positions from which the current frame cannot obtain motion information belong to the temporal motion vector predictor (TMVP). These TMVP positions are to be generated. One or more TMVP candidate positions are called a TMVP mode.

[0269] Both modes (LDMVP and TMVP) can be combined in order to build a motion vector candidate list (MVC list). The mode can be modified, which means that some candidate positions can be changed and / or discarded. In addition, the LDMVP mode can be modified by neighboring positions that may not be part of the original LDMVP mode.

[0270] In the present disclosure, the LDMVP mode is defined as a set of offsets (e.g., in pixels or other basic units) relative to the position of the current block. The position of the current block represents the position of the upper left sample of the current block. In other words, the offset or offset value refers to the position relative to the reference position. The TMVP mode is also similarly defined based on the offset relative to the position of the current block.

[0271] In the present disclosure, the above position of the upper left sample of the current block or CU is used as the reference position. However, the reference position is not limited to the upper left sample position. Alternatively, any other sample position may be used for reference. For example, this includes sample positions at the upper right, lower left, and / or lower right of the current CU. Another option is to use sample positions along the upper, lower, left, and right edges of the current CU as reference positions.

[0272] Some positions defined by the mode and the corresponding motion information may not be available because they are outside the current picture or current slice, or may correspond to intra-frame decoding units, or may be rejected for some other reasons. Therefore, the set of candidate positions is a subset of the positions defined by the mode. The set of candidate positions is checked according to a predefined order to generate the final LDMVP candidate list. Fig.11 The block diagram of the LDMVP list generation algorithm is shown in FIG. The input parameters of this algorithm are: mode P, CU position (x CU ,y CU ), and CTU position (x CTU ,y CTU ).

[0273] In the following, it is assumed that CTU width and height are predefined parameters available in the LDMVP algorithm. The output of the LDMVP list generation algorithm is the LDMVP candidate list. The procedure generatePositions(P,x CU ,y CU )Use (x CU ,y CU) as the base point and using the offsets from pattern P in order according to the candidate priority, the position (x,y) is generated. The priority here refers to the order in which the candidates are inserted into the MVC list. Before adding a new candidate to the list, check that the new candidate does not coincide with a candidate already in the list. For some modifications to the LDMVP algorithm, the procedure LdmvpAlgorithmSpecificChecks(P,x CU ,y CU ,x CTU ,y CTU ) to perform additional checks.

[0274] In the following, reference Fig.11 The flowchart briefly summarizes the method steps of the LDMVP list generation algorithm. In step S1101, check whether the position (x, y) belongs to a pattern relative to the current CU position (xcu, ycu). In step S1102, check whether the position (x, y) is available. If not, use the next position (x, y). If available, check whether the block corresponding to the position (x, y) is inter-frame coded (step S1103). If not, perform the above two test steps S1101 and S1102 on the next position (x, y). If so, a specific check performed by the program LDMVPAlgorithmSpecificChecks can be performed on the position (x, y) (step S1104), which uses x, y, xcu, ycu, xctu, yctu as input parameters. In step S1105, motion information is obtained for the position (x, y). This means obtaining a motion vector for the position (x, y). Subsequently, it is checked (step S1106) whether the obtained motion vector MV (having components Vx and Vy) is consistent with any MV previously added to the LDMVP list. If the MV is identical to any previous MV in the list, the check starting from step S1101 is performed on the next position (x, y). If the MV is not in the list, the MV is added to the LDMVP list (S1107). When all positions (x, y) have been checked by steps S1101 to S1107, the algorithm ends and the LDMVP list is provided as output. The check of step S1106, i.e. whether the obtained MV is consistent with an MV already present in the MVC list, is also called full pruning. This means that the obtained MV is checked against all MVs already present in the MVC list.

[0275] Fig.11 The flowchart of the LDMVP list generation algorithm shown in FIG. 1 is an exemplary embodiment. Fig.35 and / or Fig.41In the flowchart of the LDMVP list generation algorithm, the procedures additionalChecks and replacePosition (discussed further below) are inserted into the flowchart of the LDMVP list generation algorithm. Optionally, step S1106 can be skipped completely. This means that all MVs corresponding to the MV candidate positions can be included in the MVC list. Alternatively, step S1106 can be performed partially. This means, for example, that the obtained MV is checked against a portion of the MVs already present in the MVC list. This partial check is called partial pruning. In one example, partial pruning can be performed for the MV that was most recently included in the MVC list.

[0276] The following embodiments of the present disclosure describe apparatus and methods for: (1) generating a mode, including generating a spatial LDMVP mode as a first mode and / or generating a temporal TMVP mode as a second mode, (2) generating an MVC candidate list including one or more MV candidates for the TMVP mode and / or the LDMVP mode, and (3) changing other LDMVP mode positions for those candidate positions for which motion information is not available or stored in a buffer.

[0277] First embodiment

[0278] According to a first embodiment, an apparatus is provided for generating a list of motion vector candidates (MVCs) for a current coding unit (CU) by determining a pattern that specifies the positions of motion vectors (MVs) relative to the positions of a current CU. The pattern is determined based on at least the size of the current CU and a grid size that specifies the minimum distance between two MV positions belonging to the pattern. One or more MVs specified by the MV positions of the pattern are included in the list of MVCs.

[0279] The size of the current CU may be the width and / or height of the current CU. The size of the grid may be determined by the grid spacing Gx and Gy, which specifies the minimum distance between two positions belonging to the pattern. The grid sizes Gx and Gy may be in pixels. For example, Gx=4 means that the grid size along the x direction of the current frame has four pixels.

[0280] The mode position can be the position of the MV candidate of the first mode or the second mode. In the present disclosure, the first mode refers to the spatial LDMVP mode, and the second mode refers to the temporal TMVP mode. However, the terms "first" and "second" are merely labels used to distinguish between the LDMVP mode and the TMVP mode. Therefore, the terms "first" and "second" are not restrictive in the present disclosure.

[0281] According to an embodiment, the mode position is determined according to the first iteration value and / or the minimum or maximum size configurable for the CU.

[0282] The first iteration value refers to the number of iterations, and the iteration value / number R may be predefined and may depend on the grid size or the maximum CU size. For example, in the case of certain hardware requirements regarding the pattern size in pixels, the number of iterations is determined by dividing the pixel size by the grid size or the maximum CU size. In general, the number R may be recalculated "on the fly" according to the CU size.

[0283] For VVC, the minimum size of a CU can be 4x4. For example, in the current VVC, 4x4 blocks are used for intra prediction, while for inter prediction, the minimum luma block size can be 8x4 or 4x8. Alternatively, a 4x4 luma block can be used only in Affine mode. In this mode, the 4 adjacent chroma blocks corresponding to one luma block have the same MV, so that the minimum chroma block in VVC (considering inter prediction) is also 4x4. For VVC, the grid size for motion information storage is still 4x4.

[0284] For example, the maximum size may be the maximum value of Gx and Gy, ie, Max(Gx, Gy), where the value of Gx and / or Gy depends on the CU size.

[0285] To construct the pattern, the following parameters may be used: CU width w, CU height h, horizontal grid size Gx, vertical grid size Gy, number of iterations R, minimum horizontal offset minw, minimum vertical offset minh.

[0286] The output of this procedure is a set of offsets S, which defines the pattern position relative to the CU position.In this disclosure, the position of the current block refers to the position of the top left sample of the current block.

[0287] Fig.12 The GeneratePattern(w,h,Gx,Gy,R,minw,minh) procedure is shown. For example, the grid sizes Gx and Gy can be 4, 8, 16, 32, 64, or larger values. Gx and Gy can depend on the block size, i.e., the size of the current block / CU. For example, Gx can be equal to w of the current CU, and Gy can be equal to h of the current CU.

[0288] For example, the number of iterations R may be 1, 2, 3, 4 or a greater value. The value of the number of iterations R may depend on the size of the CU. In one example, a larger R value is used for a larger CU and a smaller R value is used for a smaller CU.

[0289] The number of iterations or the iteration value R determines the size of the pattern. The size of the pattern represents the range of the pattern position in terms of its maximum distance relative to the current CU position. In the present disclosure, the current CU position as the reference position is the upper left position of the current CU. The distance can be defined in pixels. The above distance can be calculated using a metric such as the Euclidean metric. The size of the pattern can also be determined by the grid size. The grid size refers to the minimum distance between the MV positions belonging to a pattern.

[0290] Thus, the iteration value R and the grid sizes Gx and Gy along the x and y directions determine the size of the pattern. The iteration value determines how many pattern positions are created along a particular direction. The directions can be along the x-axis only, along the y-axis only, and / or along a combination of the x-axis and the y-axis.

[0291] For example, the minimum horizontal offset minw may be equal to the minimum CU width or the minimum inter-CU width, or equal to a value of 4, 8, or greater. For example, the minimum vertical offset minw may be equal to the minimum CU height or the minimum inter-CU height, or equal to a value of 4, 8, or greater. For some codecs, the minimum CU width / height and the minimum inter-CU width / height may be different.

[0292] Using the above input parameters, the LDMVP pattern may be generated as the first pattern by the program GeneratePattern to generate the LDMVP candidate position set S as follows.

[0293] Initially, the set S is empty (step S1201). In step S1202, the subsequent steps are performed R times. At each iteration step "i", the increments Dx and Dy are determined according to the grid sizes Gx and Gy (step S1203). This means that at each iteration step, the candidate position is determined relative to the reference position. Here, the reference position is the upper left position of the current CU. Alternatively, any other position can be used as a reference.

[0294] In an exemplary embodiment of the present disclosure, the generated pattern consists of five pattern "arms" j=1, ..., 5, which are Fig.12 In the flowchart of , the offset components OffsetX(1) to OffsetX(5) and OffsetY(1) to OffsetY(5) are represented. Therefore, at each iteration step "i", the offset pair {OffsetX(j), OffsetY(j)} defines the candidate positions of the j-th pattern "arm". The term "arm" refers to a subset of candidate positions of a pattern generated along a specific direction. The direction is specified based on the Dx, Dy increments and the input parameters. The candidate positions of an "arm" can be called a series, and the series defines the subset S of candidate positions of the j-th arm. j .

[0295] exist Fig.12 In the example of , a corresponding candidate position is determined for each "arm" j=1, ..., 5 in steps S1204 to S1208. In steps S1209 and S1210, the offset xy components are included in the set S as an offset pair {OffsetX(j), OffsetY(j)}. In step S1211, the final set S is provided as output.

[0296] Fig.13 An example of a pattern generated using the above procedure GeneratePattern(w,h,Gx,Gy,R,minw,minh) is shown, using the parameters w=128, h=128, Gx=32, Gy=32, R=3, minw=4, minh=4. Fig.14 Another example is shown where w=128, h=128, Gx=32, Gy=32, R=4, minw=4, minh=4.

[0297] Use the reference above Fig.12 The common feature of the modes generated by the direction of the mode "arm" is that the MV mode position is located at the upper left, upper, upper right, left, and lower left parts adjacent to the current CU. Fig.13 and Fig.14 It can be seen that each series contains three or four candidate positions, corresponding to the iteration value R = 3 or R = 4, respectively. This shows how the first iteration value R determines the size of the pattern.

[0298] In this disclosure, a pattern with an "arm" orientation generated by the GeneratePattern procedure described above is referred to as a "sun" pattern.

[0299] Using the candidates from set S, a final LDMVP candidate list is generated. The offset value pairs for each arm can be added to set S in various ways, depending on the arm index "j" and the number of iterations "i". For example, the candidates obtained in iteration 1 can be added to the list first, followed by the candidates obtained in iteration 2, and so on. Within one iteration, the order of adding candidates can be, for example, clockwise or counterclockwise. Fig.15 and Fig.16 Examples of candidate rankings based on the probabilities of suitable motion sources are shown for one iteration and three iterations, respectively.

[0300] Compared to existing search patterns, the "sun" pattern proposed in the present disclosure removes positions that are less likely to provide useful motion information and / or motion vector information.

[0301] This may provide the advantage of reducing the number of candidate positions while maintaining coding gain.

[0302] According to an embodiment, the processing circuit is configured to adapt the grid size according to the size of the current CU and / or the minimum value and the maximum value of the grid size.

[0303] This means that the horizontal grid size Gx and the vertical grid size Gy can depend on the CU size. For example, Gx can be equal to w and Gy can be equal to h. If the number of iterations R is fixed, the pattern size will be small for small blocks and large for large blocks. Too small a pattern size will result in coding losses due to the rather small search area. Too large a pattern size requires access to a large amount of motion information that may not be in the cache, which can cause problems for software and hardware implementations. This problem can be solved by imposing further restrictions on the grid size.

[0304] The minimum and maximum values ​​for the grid size are called parameters minGridSize and maxGridSize.

[0305] In an exemplary embodiment of the present disclosure, the grid size depends on the block size and two parameters: minGridSize and maxGridSize. This means that if the CU width w is less than minGridSize, the grid size Gx is minGridSize. If the grid width w is greater than maxGridSize, Gx is maxGridSize, in other cases Gx is w. The same scheme can be applied to Gy. For example, the value of minGridSize can be 4, 8, or 16. For example, the value of maxGridSize can be 32 or 64.

[0306] To prevent the pattern from being too small or too large, another scheme for adaptive grid size can be used: a fixed search range instead of a fixed number of iterations R. The search range can be defined as the maximum distance from the CU to the candidate position in the horizontal and vertical directions. For example, for Fig.18 In the mode shown in , the search range is limited to 3*max(Gx,Gy). Fig.19 In the mode shown, the search range is limited to 4*max(Gx,Gy). The number of iterations R can be calculated as searchRange / max(Gx,Gy).

[0307] The disadvantage of this method is that the number of iterations R is too large for small blocks. For example, for a search range of 128 and a 4x4 block, R is equal to 32. Therefore, if the pattern is Fig.18 As in the illustrated mode, each iteration contains 5 positions, so the number of candidate positions in this case is equal to 32*5=160.

[0308] This problem can be solved by limiting the number of iterations as proposed above. The maximum number of iterations can be 4, 5, 6, 7, 8 or greater. Therefore, the number of iterations R can be calculated as min(Rmax, searchRange / max(Gx, Gy)). For most cases, the coding performance using the limit of Rmax = 8 is close to that of the unrestricted version. However, in this case, the final number of positions to be examined in the pattern can be reduced, for example, from 160 to 40.

[0309] For example, the above limitations can be as follows: minGridSize = 8, maxGridSize = 64, Rmax = 6. In this case, for example, for a CU of size 4x128 and searchRange = 128, Gx is equal to 8, Gy is equal to 64, and R = 2. For a CU of size 4x4 and searchRange = 128, Gx is equal to 8, Gy is equal to 8, and R = 6.

[0310] In the case of using the above adaptive grid size technique, if Gx is not equal to Gy, the grid can be rectangular. By using a square grid, the software and hardware implementation of the LDMVP algorithm can be further simplified. The side length G of the square can be calculated based on the CU width w and CU height h. For example, the side length G of the square can be calculated using the following equations:

[0311] 1. G = max(w, h).

[0312] 2. G = min(w, h).

[0313] 3. G = (w + h + 1) >> 1 (>>x represents a right shift of an integer value, which is equivalent to an integer division by 2x).

[0314] 4. G = (w + h) >> 1.

[0315] 5. G is set to the value of the power of two closest to (w + h + 1) >> 1.

[0316] 6. G is set to the value of the power of two closest to (w + h) >> 1

[0317] 7. 1 << ((k + r + 1) >> 1), where w = 2k and h = 2r (<<x represents a left shift of an integer value, which is equivalent to multiplying by 2x)

[0318] 8. 1 << ((k + r) >> 1), where w = 2k and h = 2r.

[0319] The pattern generated according to the first embodiment can be a spatial LDMVP pattern. In the present disclosure, the LDMVP pattern is referred to as the first pattern.

[0320] According to an embodiment, the processing circuit of the above-mentioned device is used to determine neighboring positions that specify the MV position within the current video frame and are different from the first pattern, and to combine these neighboring points with the first pattern.

[0321] This means that, in addition to Figure 6 In addition to the traditional neighboring merge candidates shown in , other neighboring candidates can be added to the LDMVP candidate. Therefore, because these neighboring candidate positions are located at those positions around the CU where motion information is available at the encoding / decoding time, these neighboring candidate positions are also spatial candidate points.

[0322] Therefore, a pattern (eg, LDMVP pattern) generated by the procedure GeneratePattern and modified by neighboring MV candidate points can be considered as a new LDMVP pattern.

[0323] Fig. 27 and Fig.28 An example of a mode containing LDMVP and additional neighboring candidates is shown, the neighboring candidates being located to the left and above the current CU, where motion information is available at the encoding / decoding moment of the current video frame.

[0324] The mode generated according to the first embodiment may also be a temporal motion vector predictor (TMVP) mode. In the present disclosure, the TMVP mode is referred to as the second mode.

[0325] The motivation for using the TMVP pattern (possibly in combination with the LDMVP pattern) comes from the fact that in most cases the LDMVP pattern (e.g. Figures 8 to 11 The LDMVP pattern shown or the "sun" pattern of the present disclosure) does not include MV candidates from the right or below the current CU. This is because the CU is usually traversed in raster order.

[0326] Therefore, at the encoding / decoding moment of the current CU, motion information from the right and from the bottom is not available. Therefore, since the candidate positions of the LDMVP mode are located at the upper left, upper, upper right, left, and / or lower left portion relative to the position of the current CU, the LDMVP mode can provide motion information for the CU of the current video frame. In order to also use motion information that is not available in the encoding / decoding instance, additional TMVP candidates (described in the H.265 / HEVC standard) are added to the motion vector candidate (MVC) list.

[0327] These candidates use motion information from a previous frame different from the current video frame, which is typically available for the entire frame. Therefore, the position of the TMVP candidate can be to the right and / or below the current CU. The term TMVP refers to the temporal motion vector prediction value, which reflects that the above position and motion information can be obtained from a video frame different from the current video frame (e.g., a previous video frame).

[0328] Since the TMVP candidate positions are expected to be from below and to the right of the current CU, the TMVP mode (ie, the second mode) is different from the LDMVP mode (ie, the first mode).

[0329] In an exemplary embodiment, the TMVP pattern can be generated by the program GeneratePatternTmvp(w,h,Gx,Gy,Rt,minw,minh), where the parameters are CU width w, CU height h, horizontal grid size Gx, vertical grid size Gy, number of iterations R, number of iterations Rt (for TMVP candidates), minimum horizontal offset minw, and minimum vertical offset minh.

[0330] Therefore, the TMVP mode is also generated based on the iteration value and / or the minimum size or maximum size that can be configured for the CU.

[0331] The iteration value Rt of the TMVP mode may be equal to or different from the iteration value R for the LDMVP mode. This means that the size of the TMVP mode may be different from the size of the LDMVP mode.

[0332] This may provide the advantage of adapting the size of each mode, and therefore the range in which motion information relative to the current CU position may be used.

[0333] Fig.17 An example of an implementation of the GeneratePatternTmvp(w,h,Gx,Gy,Rt,minw,minh) procedure is shown. The TMVP candidate positions are generated in a manner similar to the procedure GeneratePattern for the LDMVP mode. The procedure provides a set St of TMVP candidate points.

[0334] Using the above input parameters, a TMVP pattern may be generated as a second pattern by the program GeneratePatternTmvp to generate a set St of TMVP candidate positions as follows.

[0335] Initially, the set St is empty (step S1701). In step S1702, the subsequent steps are performed Rt times. At each iteration step "i", the increments Dx and Dy are determined according to the grid sizes Gx and Gy (step S1703). This means that at each iteration step, the TMVP candidate position is determined relative to the reference position, where the reference position is the upper left position of the current CU.

[0336] In an exemplary embodiment of the present disclosure, the generated TMVP pattern consists of two pattern "arms" j=1,...,2, which are Fig.17 In the flowchart of , the offset components OffsetX(1) to OffsetX(2) and OffsetY(1) to OffsetY(2) are used to represent them. Therefore, at each iteration step "i", the offset pair {OffsetX(j), OffsetY(j)} defines the TMVP candidate positions for the j-th pattern "arm". Similar to the LDMVP patterns generated by the above-mentioned program GeneratePattern, the term "arm" represents a subset of candidate positions for the pattern generated along a specific direction. The direction is specified based on the Dx, Dy increments and the input parameters of the GeneratePatternTmvp program. The TMVP candidate positions of an "arm" can be called a series, which defines a subset of the candidate positions St of the j-th arm. j .

[0337] exist Fig.17 In the example of , in steps S1704 to S1705, a corresponding candidate position is determined for each "arm" j=1, ..., 2. In steps S1706 to S1707, the offset xy components are included in the set St as offset pairs {OffsetX(j), OffsetY(j)}. In step S1708, the final set St is provided as output.

[0338] Figures 18 to 21 An example of the TMVP mode is shown along with the LDMVP mode.

[0339] Second embodiment

[0340] In the first embodiment of the present disclosure, a first mode (i.e., LDMVP mode) and / or a second mode (i.e., TMVP mode) may be generated. These modes may now be used alone and / or by combining the first mode and the second mode, and motion vector candidates from the modes may be included in the motion vector candidate MVC list.

[0341] According to a second embodiment of the present disclosure, a device is provided for generating a motion vector candidate (MVCs) list for a current coding unit (CU). The device includes a processing circuit, and the processing circuit is used to include the following in the MVC list: one or more motion vectors (MVs) determined according to a first mode, the first mode specifies the MV position in the current video frame where the current CU is located; and one or more MVs determined according to a second mode, the second mode specifies the MV position in a video frame different from the current video frame.

[0342] The generated MVC list may have one or more MV candidates. The MVC list may not include all MV positions from the first mode and / or the second mode. This means that a subset of MV candidates from the first mode and / or the second mode may be included in the MVC list. This may also include not including MV candidates from the first mode and / or the second mode. This means that the MVC list may be empty and / or include MV candidates only from the second mode or only from the first mode.

[0343] The first mode of MV positions within the current video frame may be a spatial LDMVP mode. The second mode of MV positions within a video frame different from the current frame may be a temporal TMVP mode.

[0344] The MVC list can be a common list of spatial MV candidates and temporal MV candidates. The common list can also be called a merge list or an AMVP (Advanced Motion Vector Prediction) list. AMVP is a predictive motion vector coding and decoding technology. The process is as follows: the encoder / decoder builds a list of motion vector prediction values ​​and sends out an index in the list through a signal. Together with the above index, the difference between the motion vector prediction value and the encoded motion vector (also called motion vector difference (MVD)) is sent out through a signal.

[0345] In an exemplary embodiment, the AMVP list may also include other candidates, such as so-called bi-combined candidates, pairwise candidates, etc. The merged list is similar and may include other candidates, such as bi-combined candidates, pairwise candidates, etc.

[0346] The first mode and the second mode may be sub-modes of a single joint mode.

[0347] The MV position in the current frame may refer to the position of the current CU in the current frame. In the present disclosure, the reference position of the CU may be the upper left corner of the current CU.

[0348] The MV positions in a frame different from the current frame may refer to the current CU. This means that the MV positions of the first mode and the second mode have the same reference position ("base position").

[0349] Alternatively, the MV position of the second mode (i.e., TMVP mode) can have a "basic position" determined according to some MV positions (for example, from one of the blocks adjacent to the current CU). In other words, the reference position of the TMVP mode and the reference position of the LDMVP mode may be different. In the ATMVP technology (advanced temporal motion vector prediction) (also known as SbTMVP-sub-block based temporal motion vector prediction), the same method is used in VVC. For example, the offset between the basic positions of the first mode and the second mode can be set according to the first MV in the MVC list (in most cases, this MV is a candidate corresponding to one of the adjacent blocks).

[0350] A different video frame than the current video frame may be a frame that has been encoded / decoded before the current frame. This means that the different frame was previously processed (i.e., encoded / decoded) and may be stored in a buffer. The different frame before the current frame may be the previous frame of the current frame, or the previous frame in display order.

[0351] Figures 18 to 21 A combined pattern consisting of LDMVP and TMVP candidates (i.e., candidate positions of the first mode (LDMVP) and the second mode (TMVP)) is shown in FIG. The combined / joint pattern is constructed by consistent application of the corresponding procedures GeneratePattern(w,h,Gx,Gy,R,minw,minh) and GeneratePatternTmvp(w,h,Gx,Gy,Rt,minw,minh), where the parameters are CU width w, CU height h, horizontal grid size Gx, vertical grid size Gy, number of iterations R, number of iterations Rt (for TMVP candidates), minimum horizontal offset minw, minimum vertical offset minh.

[0352] These are programs corresponding to the apparatus and method for generating a pattern according to the first embodiment, and their corresponding implementations are described in Fig.12 and Fig.17 The flowchart is shown in FIG.

[0353] The iteration value Rt of the TMVP mode may be equal to or different from the iteration value R of the LDMVP mode. This means that the size of the TMVP mode may be different from the size of the LDMVP mode.

[0354] This may provide the advantage of adapting the size of each mode, and therefore the range in which motion information relative to the current CU position may be used.

[0355] The output of these procedures are two sets: set S contains the offsets of spatial LDMVP candidates for the current video frame (first mode), and set St contains the offsets of additional temporal TMVP candidates for different video frames (second mode).

[0356] The term "offset" indicates a position offset relative to the CU position (the position of the current block indicates the position of the upper left sample of the current block).

[0357] Fig.18 The result of applying GeneratePattern(w,h,Gx,Gy,R,minw,minh) and GeneratePatternTmvp(w,h,Gx,Gy,Rt,minw,minh) procedures to obtain a combined pattern is shown, where the parameters are Gx=32, Gy=32, R=3, Rt=1, minw=4, minh=4, and the combined pattern includes candidate positions of the LDMVP mode (first mode) and the TMVP mode (second mode). Figures 19 to 21 Other examples of combining LDMVP mode and TMVP mode are shown.

[0358] The final candidate list is generated using candidates from set S and St. Candidate positions from set S in LDMVP mode and set St in TMVP mode can be combined in any order.

[0359] For example, the candidate positions can be combined according to the following order:

[0360] 1. The candidate obtained in iteration 1 of the procedure GeneratePatternTmvp

[0361] 2. The candidate obtained in iteration 1 of the procedure GeneratePattern

[0362] 3. The candidate obtained in iteration 2 of the procedure GeneratePatternTmvp

[0363] 4. The candidate obtained in iteration 2 of the procedure GeneratePattern

[0364] And so on.

[0365] In one iteration, the order of LDMVP candidates can be, for example, Fig.15 As shown, or alternatively, it can be clockwise or counterclockwise. Fig. 22 An example of the candidate order of LDMVP+TMVP candidates is shown in , where R=3, Rt=1.

[0366] like Figures 19 to 21 As shown in the example in, in the present disclosure, the number of TMVP candidate positions is less than the number of LDMVP positions.

[0367] In an exemplary embodiment, only those TMVP candidates whose positions are located in the same CTU row in a collocated reference picture are added to the MVC candidate list.

[0368] In another exemplary embodiment, the combined pattern may contain only temporal TMVP candidates. Fig.23 and Fig.24 An example of such a mode is shown in . This means that in this case, the MVC list contains only one or more MV candidates from the TMVP mode.

[0369] According to an embodiment of the present disclosure, the processing circuit is configured to include one or more MVs determined based on a position adjacent to a current CU in a current video frame in an MVC list, where the position is different from the MV position in the first mode.

[0370] This means that, in addition to Figure 6 In addition to the traditional neighboring merged candidates shown in , other neighboring candidates can be added to the LDMVP candidate (first mode). This means that the neighboring candidate positions are also spatial candidate points, which are located at those positions around the CU where motion information is available at the encoding / decoding time.

[0371] Therefore, the LDMVP pattern generated by the program GeneratePattern and modified by the neighboring candidates can be regarded as a new LDMVP pattern, which is represented by an expanded set S, which includes the original set S (ie, the output of the program GeneratePattern) and the above-mentioned neighboring points.

[0372] Fig. 27 and Fig.28 Examples of patterns containing LDMVP and additional neighboring candidates are shown. As can be seen from these figures, neighboring points can be placed specifically, such as along the left edge or top edge of the current CU. These points may not be included in the GeneratePattern procedure unless neighboring points are specifically considered.

[0373] According to an embodiment of the present disclosure, determining the second mode includes reflecting one or more MV positions of the first mode with reference to the reflected line.

[0374] In an exemplary embodiment, the reflection line passes through the lower left corner and the upper right corner of the LDMVP pattern. Generally, the reflection / mirror line should follow the separation between available motion information (in scanning order or encoding / decoding order) and unavailable motion information. Therefore, the reflection line divides the frame and / or picture into two areas.

[0375] This means that, for example, according to a common encoding / decoding order, the upper left part of a picture / frame comprises available motion information, whereas the lower right picture has no available motion information.

[0376] Any other reflection line may be suitable for mirroring the LDMVP mode.

[0377] In one exemplary embodiment, the TMVP pattern is generated by mirroring the entire LDMVP pattern across the reflection line.

[0378] According to another example, the TMVP pattern is generated by reflecting only a portion of the LDMVP pattern.

[0379] Fig.25 and Fig.26 Examples of mode mirroring are shown, including full mirroring and partial mirroring.

[0380] Fig.29 Examples of combined modes are shown, including LDMVP mode and TMVP mode modified by neighboring points.

[0381] According to an embodiment of the present disclosure, the processing circuit of the above-mentioned device determines the first mode including determining new MV positions for one or more MV positions that are unavailable and / or not stored in the buffer in the first mode, so that the new MV positions are available and / or stored in the buffer.

[0382] This means that MV positions for which motion information is not available and / or not stored in the first mode are not included in the MVC list. These excluded MV positions in the first mode (i.e., LDMVP mode) are referred to as MV positions discarded / excluded from the first mode. The discarded / excluded MV positions of the first mode are used to determine new MV positions. For example, the new MV positions can be determined by adjusting the position of the discard point.

[0383] In addition, in this embodiment, as shown below Figure 30 to Figure 33 As discussed, it is not necessary that all excluded positions are not position adjusted. The result of the position adjustment is that the adjusted position is available and stored in the buffer. This means that the motion information of the new MV position is available and / or stored in the buffer.

[0384] For example, such a buffer may be a row buffer, which has fast read and write access times and is typically used to store motion information of a restricted region of CTUs adjacent to a current CTU for which motion prediction is performed.

[0385] A line buffer is a buffer used to store data from a CTU. In Versatile Video Coding (VVC), CTUs are decoded in raster order (in one tile / brick). For some methods, for example, for merge mode, data from the upper CTU is needed. For this purpose, all data from the lower CTU that may be needed during the decoding of the current CTU is stored in a specific buffer. This buffer is called a "line buffer".

[0386] Fig.30 An exemplary embodiment using a "sun" pattern generated by the program GeneratePattern as described in one of the previous embodiments is shown. Figure 8 The difference of the "diagonal" mode shown is that, for example, the initial mode positions are arranged diagonally relative to the current CU at its upper left corner, upper right corner, lower left corner, and in the middle of the left edge and the upper edge in the left-right direction. Thus, the "sun" mode covers the upper "triangle" area of ​​the CU in an isotropic-like manner.

[0387] Therefore, the "sun" mode can provide the advantage of using motion information limited to the top of the current CU, and the corresponding motion information is more evenly "distributed" in the upper and left areas around the current CU.

[0388] Fig.30 Also shown are regions R0 and R1 (horizontal line pattern) around the current CU, which refer to allowed regions defining the locations of motion vectors where motion information is available. Fig.30 In the example of , R0 covers the entire CTU to the left of the current CTU, while R1 contains motion information on the bottom small area of ​​the CTU to the upper left, above, and upper right of the current CTU. Some MV positions in the "sun" mode (i.e., the first mode) are already located in the current CTU, or in either region R0 or R1, so motion information for these positions is available and / or stored in the buffer. Fig.32 As shown, these MV positions remain unchanged (white squares). The rejected positions in the first pattern are marked with bold black and white hashed patterns, and these positions are to be adjusted.

[0389] In order to achieve that you do not have to adjust all excluded pattern positions, you can Fig.31 As shown in the example, the pattern positions are first grouped into several categories.

[0390] One option for grouping the pattern positions into several categories may be to group the LDMVP positions (i.e., the first pattern positions corresponding to the set S) into subsets Si. The subsets Si may also be referred to as series. This means that the pattern positions in the series Si are located in a particular direction. The direction may be, for example, in any of the directions of top, top right (diagonal), top left (diagonal), left, and / or bottom left (diagonal). Fig.31 In the example of , for the "sun" mode, there are five series S1 to S5, each series comprising three positions of the sun mode. The series S1 can also be called an "arm".

[0391] Assume that series S i There are n i positions, then each position can be represented by the position (x 0i ,y 0i )、(x 0i +dx i ,y 0i +dy i )、(x 0i +2*dx i ,y 0i +2*dy i ),…,(x 0i +n i *dx i ,y 0i +n i *dy i ) means, where (x 0i ,y 0i ) refers to the position of the first position in the series. For example, the first position may be the position closest to the current CU. For example, in a series where the pattern position is in the upper direction, the first position is the bottom position close to the top edge of the current CU. i anddy i Refers to the offset between adjacent positions in the horizontal and vertical directions respectively.

[0392] For example, based on the corresponding series of dx i and / or i value, it is now possible to group pattern elements (i.e. positions) into classes. This means that by using the value dx i and / or i (including whether it is positive or negative). Thus, each class may include one or more series of pattern positions.

[0393] For example, for Fig.31 Using the above method to classify the mode positions, the mode positions of the sun mode can be divided into their respective classes (here, class 1 to class 3) according to the following constraints on dx and dy:

[0394] 1. Class 1: dx=0, dy<0

[0395] 2. Class 2: dx!=0,dy<0

[0396] 3. Class 3: dx<0,dy>=0

[0397] The operation "!=" means "not equal to".

[0398] Applying dx-dy based classification to the solar pattern results in three classes, class 1 including one series, and classes 2 and 3 including two series each.

[0399] For example, the pattern positions may be divided into categories according to whether one or more pattern positions are located to the left and / or to the lower left of the current CTU. Fig.31 In the example of , this applies to pattern positions of class 3. Additionally or alternatively, pattern positions located above the current CTU (i.e., top left, top, and / or top right) may also be grouped based on whether the x position and / or y position of the position are the same. Fig.31 In the example of , the upper pattern positions along the left and right diagonals of the CU have different x and y positions. Therefore, these positions are classified into common class 2. While the top pattern positions above the middle of the CU have the same x position but different y positions. Therefore, these positions are classified into class 1.

[0400] Therefore, in general, Fig.31 As shown, the pattern position in the upward direction from the top of the CU is class 1, the pattern position in the upper left and upper right diagonal directions is class 2, and the remaining pattern positions are class 3. Fig.32 As shown, the pattern position within the current CTU and / or within any regions R0 and R1 remains unchanged.

[0401] According to an embodiment, determining the new MV position comprises assigning an x-position and / or a y-position of one or more MV positions to the new MV position and discarding the one or more MV positions from the first mode.

[0402] This means that the discarded / rejected MV positions of the first mode (LDMVP mode) are assigned (new) MV positions in the x-direction along the width of the current CTU and / or in the y-direction along the height of the current CTU.

[0403] The x and y directions, respectively, the x and y positions are in pixel coordinates, where the x pixel axis is along the width of the CTU and the y pixel axis is along the height of the CTU.

[0404] This is Fig.32This is shown in an example where the excluded pattern positions in class 1 and class 2 are assigned new positions, i.e., they are adjusted. Here, their y coordinates are adjusted along the CTU height so that they are located in region R1. Since the pattern positions in class 2 have different x positions, their adjusted positions are located in different positions in region R1. In contrast, since the x positions of the pattern positions of class 1 were originally the same, after the position adjustment, the pattern positions of class 1 are located in the same positions in region R1. After being assigned to the new MV positions, the original MV positions in the first pattern that are outside regions R0 and R1 and in regions R0 and R1 are discarded / rejected.

[0405] According to an embodiment, the above determination comprises assigning a new y position or a new x position to one or more new MV positions having the same x position or the same y position using a predetermined scheme.

[0406] This means that in the case where some new MV positions are located at the same point, one or more of the new MV positions are reassigned a new y position and / or a new x position.

[0407] The above scheme may be a function. This means, for example, that the reallocated position may be calculated using the function with the allocated position as input. The above calculation may be performed based on the CU position and / or the mode position.

[0408] In one exemplary embodiment, the predetermined scheme is to shift the assigned y-position and / or x-position of the new MV position by a predetermined offset value.

[0409] The offset value may be a number of pixels, thereby shifting the y position and / or x position (e.g., in pixels) by the pixel offset by adding and / or subtracting the offset value from the adjusted y position and / or x position value. Alternatively, the predetermined scheme may be a factor that is multiplied and / or divided by the y position and / or x position. Other schemes may also be provided to achieve reallocation of the adjusted position when the x position and / or y position of the adjusted position are the same.

[0410] The offset value may depend on the CU position and / or the mode position. For example, the y coordinate may be modified to be less than 4 points (the MV storage grid size) from the top CTU boundary. This means that the offset value may be limited depending on the CU position and distance. Alternatively and / or optionally, the offset value may be predetermined. In any case, the new MV position should be shifted to a new y position and / or x position such that these positions are not outside the allowed region after the shift.

[0411] exist Fig.32In the example of , the new MV positions of the pattern positions in class 1 have the same y position. Since the x position is the same for both the adjusted pattern position and the original pattern position, these positions are located at the same position in region R1. Therefore, one of the two new MV positions is assigned a new x position by shifting by an offset. In other words, the adjusted positions with the same position can be located in region R1. i The internal distribution is spread out so that the assigned locations have different positions. This means that the locations are located at different points. Fig.32 This is the case for the pattern positions of class 1, which have different positions after the position assignment and subsequent position reallocation.

[0412] In this embodiment where the mode positions can be divided into classes, the number of motion vector candidate positions is the same as the number of positions for the first / LDMVP mode. In addition, the mode and region are selected in this example to use motion information limited to the CTU on top of the current CTU.

[0413] exist Figure 30 to Figure 33 In the example shown, the pattern positions are located at the upper left corner, upper right corner, and lower left corner of the current CU along the diagonal. In addition, the positions are placed horizontally to the left from the middle of the left edge and vertically upward from the middle of the upper edge. In this disclosure, we call this mode the "sun" mode, which is composed of Fig.12 The scheme shown in the flowchart is generated by the GeneratePattern procedure and has been discussed in one of the above examples.

[0414] In this example, it is assumed that the CTU decoding order is from left to right and from top to bottom. Accordingly, the image parts above and to the left of the current CTU and the motion vector information are available in the current frame. The remaining parts may depend on the current CTU and can therefore only be decoded after the current CTU is decoded. However, the present invention is not limited to this. Another decoding order may generally be used, so that the availability of the parts of the current frame is different.

[0415] For the above modes (e.g. "Sun" mode), Figures 7 to 10 The mode shown, or any other suitable mode, position adjustment of the mode position can be achieved by the following adjustCandidatePositions procedure:

[0416] 1. All candidates are divided into Fig.31 Three categories shown.

[0417] 2. For white square candidates, no position adjustment is used.

[0418] 3. For black and white hash candidates (class 1 and class 2), the y position is changed to the highest y position of the current CTU minus one.

[0419] 4. For black-white hashed left and right candidates (class 2), the x position remains unchanged.

[0420] 5. For the black-and-white hashed intermediate candidates (class 1), the new x position can be calculated, for example, using the AdjustCandidatePositions(x,y,topYposOfCurCtu,xOffsetForOneGrid,log2Gy) procedure.

[0421] As discussed above, the results of applying adjustCandidatePositions to the "sun" mode are Fig.32 Shown in.

[0422] Fig.33 The flowchart of the procedure AdjustCandidatePositions(x,y,topYposOfCurCtu,xOffsetForOneGrid,log2Gy) is shown by way of an example of implementation. Here, topYposOfCurCtu is the highest y position of the current CTU, and xOffsetForOneGrid can be, for example, w>>2, Gx or Gy; 2log2Gy, where w is the width of the current CU, and Gx and Gy are the width and height of the pattern grid, respectively.

[0423] The purpose is to calculate an offset value, in this example, by subtracting (step S3307) or adding (step S3306) the offset, the x position is shifted to the left or right. In step 3301, the left shift is performed (in this example, bitwise by the operator "<<"), which corresponds to multiplying the offset value. In step S3302, the number of skipped grids is determined with reference to the highest y position of the current CTU, the y position of the point (x, y), and the offset. The value is shifted to the right (bitwise), corresponding to division. In step S3303, it is determined whether the number of skipped grids is even or odd (parity check). If the number is odd, the corresponding value "turnRight" is "1". In step 3304, the offset value is determined using the number of skipped grids and the x offset value of a grid. If turnRight is logical 1 (step S3305), the x position is shifted to the right (step S3306) by adding the offset value to the x value. If not, the x position is shifted to the left by subtracting the offset (step S3307). The adjusted candidate position is returned (step S3308).

[0424] The procedure AdjustCandidatePositions may be included as step S3300 between steps S1104 and S1105 in the flowchart of the LDMVP list generation algorithm (not shown).

[0425] According to an embodiment of the present disclosure, the above-mentioned processing circuit is used to generate a pattern relative to the position of the current CU based on: the size of the current CU; the size of the grid that specifies the minimum distance between two MV positions belonging to the above-mentioned pattern; wherein the above-mentioned pattern specifies the position of the motion vector (MV), and the mode is the first mode and / or the second mode.

[0426] The size of the current CU may be the width and / or height of the current CU. The size of the grid may be determined by the grid spacing Gx and / or Gy, which specifies the minimum distance between two positions belonging to the mode. The grid sizes Gx and Gy may be in pixels. For example, Gx=4 means that the grid size in the x direction of the current frame has four pixels.

[0427] The grid size may be the minimum distance between MV positions belonging to the same mode. This means that if the first mode and the second mode are not identical, the minimum distance of the first mode and the minimum distance of the second mode may be different. Furthermore, the minimum distance in the x-direction and in the y-direction may be different (rectangular grid).

[0428] According to an embodiment of the present disclosure, the above-mentioned processing circuit is used to determine the second mode and / or the first mode based on at least one of the following: an iteration value specifying the size of the second mode and / or the first mode on the grid; the minimum size that can be configured for the CU; and the maximum size that can be configured for the CU.

[0429] The iteration value refers to the number of iterations. The iteration value / number R may be predefined and may depend on the grid size or the maximum CU size. For example, in case of certain hardware requirements regarding the pattern size in pixels, the number of iterations is determined by dividing the pixel size by the grid size or the maximum CU size. In general, the number R may be recalculated "on the fly" depending on the CU size.

[0430] The iteration value relates to how many times the MV positions of the first or second pattern are generated along a particular direction for a given grid spacing Gx and Gy. The MV positions generated along a particular direction of a pattern are also referred to as a series or "arm" of the pattern.

[0431] Since the first pattern and the second pattern may have different grid spacings Gx and / or Gy, the size of the first pattern and the size of the second pattern may be different for the same iteration value. When the first pattern and the second pattern use the same grid spacing, if different iteration values ​​are used for the first pattern and the second pattern, the sizes of the two patterns may be different. In other words, the apparatus and method for generating the first pattern and / or the second pattern may use different iteration values.

[0432] For example, Fig.18The pattern shown is a pattern formed by combining the LDMVP (first pattern) and the TMVP (second pattern), the spatial LDMVP pattern uses an iteration value R = 3, and the temporal TMVP pattern uses an iteration value Rt = 1. For this case, the corresponding series (i.e., the pattern "arm") contains three points (for the LDMVP pattern) and one point (for the TMVP pattern), respectively.

[0433] For VVC, the minimum size of a CU can be 4x4. For example, in the current VVC, 4x4 blocks are used for intra prediction, while for inter prediction, the minimum luma block size can be 8x4 or 4x8. Alternatively, a 4x4 luma block can be used only in Affine mode. In this mode, the 4 adjacent chroma blocks corresponding to one luma block have the same MV, so that the minimum chroma block in VVC (considering inter prediction) is also 4x4. The grid size for motion information storage is still 4x4 for VVC.

[0434] For example, the maximum size may be the maximum value of Gx and Gy, ie, Max(Gx, Gy), where the value of Gx and / or Gy depends on the CU size.

[0435] According to an embodiment of the present disclosure, the generation mode (including the first mode and / or the second mode) includes adapting the grid size according to the size of the current CU and / or the minimum value or maximum value of the grid size.

[0436] This means that the horizontal grid size Gx and the vertical grid size Gy can depend on the CU size. For example, Gx can be equal to w and Gy can be equal to h. If the number of iterations R is fixed, the pattern size will be small for small blocks and large for large blocks. Too small a pattern size will result in coding losses due to the rather small search area. Too large a pattern size requires access to a large amount of motion information that may not be in the cache, which can cause problems for software and hardware implementations. This problem can be solved by imposing further restrictions on the grid size.

[0437] The minimum and maximum values ​​for the grid size are called parameters minGridSize and maxGridSize.

[0438] In an exemplary embodiment of the present disclosure, the grid size depends on the block size and two parameters: minGridSize and maxGridSize. This means that if the CU width w is less than minGridSize, the grid size Gx is minGridSize. If the grid width w is greater than maxGridSize, Gx is maxGridSize, in other cases Gx is w. The same scheme can be applied to Gy. For example, the value of minGridSize can be 4, 8, or 16. For example, the value of maxGridSize can be 32 or 64.

[0439] To prevent the pattern from being too small or too large, another scheme for adaptive grid size can be used: a fixed search range instead of a fixed number of iterations R. The search range can be defined as the maximum distance from the CU to the candidate position in the horizontal and vertical directions. For example, for Fig.18 In the mode shown in , the search range is limited to 3*max(Gx,Gy). Fig.19 In the mode shown, the search range is limited to 4*max(Gx,Gy). The number of iterations R can be calculated as searchRange / max(Gx,Gy).

[0440] The disadvantage of this method is that the number of iterations R is too large for small blocks. For example, for a search range of 128 and a 4x4 block, R is equal to 32. Therefore, if the pattern is Fig.18 As in the illustrated mode, each iteration contains 5 positions, so the number of candidate positions in this case is equal to 32*5=160.

[0441] This problem can be solved by limiting the number of iterations. For example, the maximum number of iterations Rmax can be 4, 5, 6, 7, 8 or more. Therefore, the number of iterations R is calculated as min(Rmax,searchRange / max(Gx,Gy)). For most cases, the encoding performance using the restriction of Rmax=8 is close to the performance of the unrestricted version. However, in this case, the final number of checked positions in the pattern can be reduced, for example from 160 to 40.

[0442] For example, the above restrictions may be as follows: minGridSize = 8, maxGridSize = 64, Rmax = 6. In this case, for example, for a CU of size 4x128 and searchRange = 128, Gx is equal to 8, Gy is equal to 64, and R = 2. For a CU of size 4x4 and searchRange = 128, Gx is equal to 8, Gy is equal to 8, and R = 6.

[0443] In the case of using the above-mentioned adaptive mesh size technique, if Gx is not equal to Gy, the mesh can be rectangular. By using a square mesh, the software and hardware implementation of the LDMVP algorithm can be further simplified. The side G of the square, where G = Gx = Gy, can be calculated based on the CU width w and the CU height h. For example, the side G of the square can be calculated using the following equations:

[0444] 1. G = max(w, h).

[0445] 2. G = min(w, h).

[0446] 3. G = (w + h + 1) >> 1 (>>x represents a right shift of an integer value, which is equivalent to an integer division by 2x).

[0447] 4. G = (w + h) >> 1.

[0448] 5. G is set to the value of the power of two closest to (w + h + 1) >> 1.

[0449] 6. G is set to the value of the power of two closest to (w + h) >> 1.

[0450] 7. 1 << ((k + r + 1) >> 1), where w = 2k and h = 2r (<<x represents a left shift of an integer value, which is equivalent to multiplication by 2x)

[0451] 8. 1 << ((k + r) >> 1), where w = 2k and h = 2r.

[0452] The apparatus and method according to any of the embodiments discussed above can be applied to any pattern, including the LDMVP pattern as the first pattern. In addition, the apparatus and method can be applied to a pattern including the first pattern and / or the second pattern, either individually and / or in any combination thereof. This means that after generating the LDMVP pattern, these methods can be applied in any order. Furthermore, according to the present disclosure, these methods can be implemented as separate processes. Alternatively, these processes can be implemented in combination with any method.

[0453] According to an embodiment, a method for generating a motion vector candidate (MVC) list for a current coding unit (CU) is provided. The steps of the method include including the following in the MVC list: one or more motion vectors (MVs) determined according to a first pattern that specifies the MV positions within the current video frame where the current CU is located; one or more MV positions determined according to a second pattern that specifies the MV positions in a video frame different from the current video frame.

[0454] According to an embodiment, a method for generating a motion vector candidate (MVC) list for a current coding unit (CU) is provided, the steps of the method comprising: determining a pattern specifying a position of a motion vector (MV) relative to a position of the current CU according to at least: the size of the current CU; a grid size specifying a minimum distance between two MV positions belonging to the pattern. The method also includes including one or more motion vectors specified by the MV positions of the pattern in the list.

[0455] The apparatus and method described in the above embodiments can be used with different LDMVP modes, for example, Figures 7 to 10 The above apparatus and method can be used with different LdmvpAlgorithmSpecificChecks(P,x CU ,y CU ,x CTU ,y CTU ) program. The above apparatus and method can be used for merging techniques and advanced motion vector prediction. The above apparatus and method can be used with different schemes for partitioning CTU into CUs, such as quadtree, ternary tree, binary tree and combinations thereof.

[0456] According to an embodiment, an encoder for encoding a current decoding unit (CU) of a current video frame is provided, the encoder comprising: a device for generating a motion vector candidate (MVC) list according to any one of the above embodiments; an inter-frame prediction unit for determining a prediction of a current CU based on at least one motion vector (MV) in the MVC list; and a compression unit for encoding the current CU by using the prediction of the CU.

[0457] According to an embodiment, a decoder for decoding a current coding unit (CU) of a video frame is provided, the decoder comprising: a device for generating a motion vector candidate (MVC) list according to any one of the above embodiments; an inter-frame prediction unit for determining a prediction of a current CU based on at least one motion vector (MV) in the MVC list; and a decompression unit for decoding the current CU by using the prediction of the CU.

[0458] The MVC list may be used for motion prediction using motion information of motion vectors at the locations of one or more MVCs.

[0459] For example, encoding can be performed by calculating the difference between the current CU and the previous prediction and by encoding the residual (difference) (including quantization and / or entropy coding). The prediction is given by the block for which the best motion vector is targeted. The best motion vector is determined from the MV candidate list by calculating a similarity measure, such as the sum of absolute difference (SAD). The best motion vector can be indicated from the encoder to the decoder in the bitstream of the encoded video, for example, as an index to the MVC list. The MVC corresponding to the index (assigned to the index) is the best MV, which will also be used for decoding. Here, the term "best" refers to the MV selected by the encoder and to be used by the decoder to obtain the prediction of the current block (CU). The above steps can be performed even if there is no residual because the prediction is already close to the CU (corresponding to zero (after quantization) residual). In principle, the index of the list can also be determined implicitly on the encoder side and the decoder side. For example, this may be the case for merge mode, where the number of MV candidates is only one candidate, so that there is no need to signal the index.

[0460] According to an embodiment, a computer-readable non-transitory medium for storing a program is provided, which includes instructions, which, when executed in a processor, cause the processor to perform the above method steps.

[0461] In the disclosed embodiment currently discussed, the procedure GeneratePattern (whose flowchart is in Fig.12 ) corresponding apparatus and method are shown in the figure, thereby generating an LDMVP mode, such as a "sun" mode. As discussed, the "sun" mode of the present disclosure has the property that motion vector candidate positions are generated in the top area of ​​the current CTU, and the corresponding points are fairly evenly located at the bottom left, left, top left, top, and top right. The LDMVP "sun" mode is generated for the current video frame.

[0462] Using the apparatus and methods associated with the program GeneratePatternTmvp, the LDMVP "sun" pattern can be modified by a temporal motion vector predictor pattern, the pattern position of which is in the lower right area surrounding the current CU and / or current CTU. These "lower" areas include the lower, lower right, and right parts. Since the motion information of these areas is not available and / or not stored in the buffer, the motion vector candidate positions are used for video frames different from the current video frame. In practice, the different frame is a previous video frame. This means that the frame is temporally prior to the current video frame. Therefore, at the encoding / decoding moment of the current frame, motion information is available and / or stored for the "lower" frame area.

[0463] Using the apparatus and methods of the present disclosure, the "sun" LDMVP mode can be extended by adding neighboring candidate positions to the LDMVP mode.

[0464] Once the LDMVP "sun" pattern is generated (and possibly extended by neighboring locations), the LDMVP pattern located outside the current CTU and one or more regions {R i}, so that its new adjusted coordinates are located in any allowed area {R i}Inside.

[0465] The apparatus and method of the present disclosure are not limited to generating only the "sun" mode as the LDMVP mode. Any other mode suitable for motion prediction and / or motion vector prediction may be generated, such as Figures 7 to 10 In addition, the LDMVP mode may be modified by: (1) adding one or more neighboring points to the LDMVVP mode, and / or (2) adjusting the LDMVP candidate positions, and / or (3) adding the TMVP candidate positions from the TMVP mode to the final list of motion vector candidates (MVC list).

[0466] Third embodiment

[0467] According to the third embodiment of the present disclosure, the first mode (ie, the LDMVP candidate mode) may be further modified to modify one or more LDMVP candidate positions. As described in the following embodiments, the first mode (ie, the LDMVP candidate mode) may be further modified by any method of the third embodiment. Fig.11 The method steps of the LDMVP list generation algorithm will not be repeated, and the method steps related to the second embodiment of the present disclosure are discussed below.

[0468] As disclosed in the following exemplary embodiments of the present invention, the present disclosure utilizes a generated motion vector candidate list of motion vector positions whose corresponding motion information is used for motion prediction and / or motion vector prediction.

[0469] The prediction may be for a current block, ie, a current decoding unit CU, the current CU being located in a current coding tree unit CTU. The current CTU may be in a current frame (ie, a current video frame), the current video frame being composed of a plurality of CTUs.

[0470] According to an embodiment, an apparatus for generating a list of one or more motion vector candidates (MVCs) is provided. The MVC list is generated for a current coding unit (CU) according to a mode position relative to the position of the current coding unit (CU) and according to the position of the current CU within a coding tree unit (CTU). The mode defines the position of the motion vector (MV).

[0471] The motion vector candidate is used for motion prediction of the current CU. This means that the motion information of the motion vector at the corresponding candidate position is used for prediction. Therefore, the motion vector position is the position of the corresponding motion candidate block. The block can be one pixel / sample and / or multiple samples corresponding to the sample block.

[0472] The position of the current CU may be the upper left corner of the current CU, where the position is in pixels or in pixel block units. The block unit may be, for example, a 2x2 pixel block or any other unit. The position of the current CU (the CU is located within the current CTU) may be the distance (pixel / sample) from the upper left corner of the current CU to the upper left corner of the current CTU.

[0473] In other words, the term "motion vector position" may refer to "position of a motion vector candidate block" or "position of a motion candidate block". Furthermore, more than one pattern position may correspond to one CU, which is the case, for example, when the grid size is 16x16 and the neighboring CU size is 128x128 or 64x64. This may happen if the grid size is (much) smaller than the size of the neighboring CU, so that several pattern positions may be located within the same neighboring CU. In most cases, there is only one MV-MV pair corresponding to one CU, so that several pattern positions may refer to the same motion information.

[0474] Typically, modes are generated based on CUs and / or grids. In practice, CU-based modes can be complex, difficult to process by the decoder, and do not necessarily provide significant gains. Therefore, grid-based modes may be preferred over CU-based modes. For this case, more than one position in the mode can correspond to a CU.

[0475] LDMVP technology can be used in two processes:

[0476] 1. Merge Mode

[0477] 2. Motion vector prediction in AMVP (Advanced Motion Vector Prediction) mode.

[0478] For the first case, two things are borrowed from the corresponding MV candidate position: a motion vector and a reference index. In the case of B frames, this can be 2 MVs and 2 reference indices.

[0479] For the second case, only the motion vector is borrowed, and the reference index is signaled in the bitstream. LDMVP can also be optionally used for the second case (AMVP). Therefore, in general, the LDMVP of the present disclosure can be used for motion prediction and / or motion vector prediction. In addition, for prediction, not only the motion vector can be used, but also the reference index in the "reference" block can be used, which can be the MV candidate position.

[0480] According to an embodiment, the processing circuit of the above-mentioned device determines the MVC for the current CU including selecting a subset of MV positions from the pattern positions. In other words, the initial mode (i.e., the initial position) undergoes additional checks, based on which certain pattern positions may be retained, discarded, and / or changed.

[0481] In the following, reference Figure 36 to Figure 39 explains the selection of a subset from the (initial) pattern, Figure 36 to Figure 39 An example of location selection is shown. Figure 36 to Figure 39 In the examples, for illustration purposes, we have been using Figure 8 The LDMVP pattern "diagonal" can be used. Any other pattern can be used, such as Figure 7 , Fig. 9 , and / or Fig.10 , including the "sun" pattern generated by the GeneratePattern procedure. An example of the "sun" pattern is in Figures 13 to 16 Shown in.

[0482] According to one embodiment, the above selection includes selecting those MV positions located within the current CTU from the pattern positions.

[0483] This can be Fig.36 As can be seen in the figure by the marked white square pattern positions, these white squares are inside the current CTU (rough dotted box). For these points, motion information is available. The pattern position remains unchanged (white square).

[0484] According to an embodiment, the selection of the subset of MV positions is performed by excluding from the pattern positions those MV positions which are not available and / or not stored in the buffer.

[0485] This means that the position may be outside the current video frame, i.e., the position is larger than the size of the current frame. Therefore, the information at that point does not exist. Motion information may also be unavailable at a certain position due to the encoding / decoding order performed on the CUs of the frame. In other words, since the lower right area of ​​the CTU is to be encoded / decoded later, at a specific moment, the motion information of the lower right area does not exist yet. Therefore, even if a position in this area is generated (i.e., is part of the pattern), no motion information exists at that point at that moment. In addition, because the corresponding information can be accessed, the motion information may not be stored in the memory (i.e., the buffer). Such a buffer can, for example, be a row buffer, which has fast read and write access times and is typically used to store motion information of a restricted area of ​​a CTU adjacent to the current CTU for which motion prediction is performed.

[0486] The line buffer is a buffer for storing data from a CTU. In Versatile Video Coding (VVC), CTUs are decoded in raster order (within one tile / brick). For some methods, for example, for merge mode, data from the CTU above is required. To this end, during the decoding of the current CTU, all data that may need to be obtained from the CTU below is stored in a specific buffer. This buffer is called a "line buffer". For motion information, including MV and reference index, a prediction of a line of motion information below is stored in the line buffer. When LDMVP is not used, in the worst case, the number of CTUs in a line * CTU size / motion information storage grid size (number_of_CTUs_in_a_line*CTU_size / motion_information_storage_grid_size) elements should be stored in the line buffer (for regular merges, it can use the neighboring CU from the CTU above).

[0487] When the LDMVP mode is unrestricted, much more MVs need to be stored than one row. However, this is unacceptable for hardware implementation.

[0488] Therefore, according to the present disclosure, using the LDMVP mode to limit the use of motion information requires storing a smaller amount of motion information in the line buffer. Therefore, prediction can be performed more efficiently and quickly.

[0489] For these situations, Fig.36 As shown, some mode positions are excluded. The coarse dotted box is the current CTU, which contains the current CU (dense dotted box). Regions R0 and R1 (horizontal line mode) refer to allowed regions, which define the motion vector positions for which motion information is available.

[0490] like Fig.36As can be seen in FIG, some pattern positions outside the current CTU are discarded. These pattern positions are outside the CTU and are not within any of the allowed regions R0 and R1 (black and white hash boxes). This means that in this embodiment, the discarded positions are completely rejected from the pattern and are no longer considered for motion vector candidate positions.

[0491] As for other pattern positions, since these positions are located in any of regions R0 and R1, their motion information is available, so these positions are retained from the pattern and their positions are unchanged (white boxes).

[0492] Figure 37 to Figure 39 An example of selecting a pattern position as described above with respect to the same "diagonal" pattern is shown, but the regions R0, R1, or R3 allowing motion vectors are arranged differently within a CTU adjacent to the current CTU. Figure 36 to Figure 39 It can be seen that in each case, because the position of the pattern relative to the region R0, R1, or R3 varies with the different arrangements of the regions within the adjacent CTU, the final list of candidate positions after the additional checks is different. It is clear to those skilled in the art that the results of the final list will also vary with the patterns having different positions (such as Figures 7 to 10 The mode shown varies.

[0493] In an exemplary embodiment of the present disclosure, Fig.35 As shown, the program additionalChecks(x,y,x CTU ,y CTU ,R) is added to the LDMVP list generation algorithm. This routine is added before getting the actual motion information based on the motion vector candidate list position.

[0494] The input parameters of the program are: test position (x, y), current CTU position (x CTU ,y CTU ), and a set of rectangles R defining areas in CTUs around the current CTU. Any of these areas corresponds to an area defining the positions of allowed motion vector candidates that can be used for prediction.

[0495] Fig.34 shows that additionalChecks(x,y,x CTU ,y CTU ,R) procedure, which checks whether the given position (x, y) belongs to the current CTU or rectangle R i, One of (steps S3401 and S3403). Check S3403 for all rectangular areas R iExecute (step S3402). If not, the position (x, y) is rejected (step S3404). This means that the position is discarded. If yes, the position (x, y) is accepted (step S3405).

[0496] pass Fig.35 The flowchart of FIG. 1 shows an exemplary embodiment of the LDMVP list generation algorithm including additional checks. Regarding the method steps S1101 to 1107 of the LDMVP list generation algorithm, the method steps of the additionalChecks program are included as step S3400 in Fig.35 The set R can be empty, which means that the above process only checks whether a given position belongs to the current CTU.

[0497] Figure 36 to Figure 39 Shows the Figure 8 Some examples of applying the additionalChecks procedure in the "diagonal" mode shown in . Specifically, for Figure 36 to Figure 39 For example, the CU position is relative to the CTU position (x CTU ,y CTU ) of pixels (16,16). In these examples, the rectangle R i Set to [x 0 i ,y 0 i ,w,h], where (x 0 i ,y 0 i ) is the rectangle relative to (x CTU ,y CTU ), w and h are the width and height of the rectangle respectively.

[0498] Table 1 provides the data for each of Examples 1 to 4 (i.e. Figure 36 to Figure 39 )Rectangle R i The corresponding setting of each example is the rectangle set {R i} and the accompanying figure, showing the application of the additionalChecks program to the "diagonal" mode. Positions rejected by the additionalChecks program are marked with black and white hash marks. Rectangle R i Marked with a horizontal line pattern.

[0499] Table 1 AdditionalChecks procedures applied to Examples 1 to 4

[0500]

[0501] In many cases, Example 4 is advantageous in practice because the CTU to the left of the current CTU and the bottom row of the CTU above the current CTU are used as the allowed area for motion prediction of the current CU. In other words, for the bottom row of these CTUs, only a limited amount of motion information is used for motion prediction. Since only a limited number of motion vectors are used, this makes motion prediction more efficient and faster. This choice is based on the assumption that, in particular, motion information from the upper CTU adjacent to the current CTU mainly contributes to motion prediction.

[0502] In conventional merging, only the right column of motion information from the left CTU is used.An additional criterion to consider when selecting the region of neighboring CTUs may be that it is simpler to store the left CTU than the upper CTU since the additional buffer size does not increase with the width of the frame.

[0503] So far, the data outside the current CU is not available and / or not stored in the buffer (e.g., not in any region R i However, the following reference Figures 40 to 45 As mentioned above, the discarded positions can still be used for motion prediction.

[0504] According to an embodiment, the processing circuit of the above apparatus determines the MVC for the current CU including replacing one or more excluded MV positions with substitute MV positions determined based on the excluded MV positions, thereby satisfying one or more predetermined conditions.

[0505] This means that, for example, the coordinate position of one or more excluded / discarded pattern positions may be changed by replacing or calculating new positions subject to predetermined conditions.

[0506] According to an embodiment, the one or more predefined conditions include that an alternative MV position is available and / or stored in a buffer.

[0507] This means that, starting from the excluded pattern position, alternative positions for these pattern positions are available and / or stored in the buffer. Fig.42 , again, a "diagonal" pattern is shown, and the same box marks or area markings are used to refer to the rejected pattern positions and the unchanged positions. Fig.42 As can be seen from the example, in the allowed region R1, there are now two positions added to this region (marked by the vertical line pattern). Because these points were not part of the original pattern, they would not be part of region R1 if only the pattern positions were used. In other words, as before, pattern positions that are not within either region R0 or R1 are still rejected, but these positions are still used to determine whether and where to add one or more new positions so that these new positions are within any allowed motion vector region.

[0508] According to an embodiment, the alternative MV position is determined to satisfy the following condition: the distance between the alternative MV position and the excluded MV position is the shortest. The term "distance" refers to a measurement of the magnitude of the spatial interval between two points along a specific direction. The spatial interval can be measured in units of the number of pixels along the pixel x and / or pixel y direction, using pixels as the unit of spatial measurement. The above distance can be calculated based on a certain metric, for example, the Euclidean norm and / or the p-norm, where p is equal to or greater than 1.

[0509] for Fig.42 In the example shown in , the shortest distance condition is applied to the discarded pattern positions, then for the second discarding position in the upper left part of the upper left CTU, a new position is added in the lower right corner of the upper left CTU, so that the added position is located in the region R1. Because the distance from the first discarding position to the region R1 is larger than that of the second position, the new position is not added for the first discarding position. A similar situation also occurs for the corresponding second discarding pattern position and the first discarding pattern position in the CTU above the current CTU.

[0510] The "distance" condition is an example of a predetermined condition and is not limited to this condition. For example, the distance condition may be combined with a distance cutoff corresponding to a threshold value, in which case the new position may be added to the region R if the distance is equal to or less than the cutoff value. i Other options and / or combinations of different conditions may be applied, thereby increasing one or more new locations available and / or stored in the buffer.

[0511] Figure 43 to Figure 45 It is shown that by Figure 42 to Figure 45 The shortest distance of the example) replaces one of the pattern positions to add other examples of the new position. Figure 43 to Figure 45 It can be seen that because the area R i The arrangement of the patterns also determines the pattern position relative to the region R i Therefore, adding a new position depends on the region R in the CTU adjacent to the current CTU. i The arrangement.

[0512] This means that, as an optimal scenario, the number of pattern positions can be maintained, since all discarded positions of the original pattern are replaced by new positions. Due to the specific choice of regions R0 to R3, Fig.21 This is the case in the example.

[0513] In an exemplary embodiment of the present disclosure, the LDMVP list generation algorithm includes Fig.41 The flowchart of the program replacePosition(x,y,x CTU,y CTU ,R), the program performs replacement of one or more pattern positions. Regarding the method steps S1101 to 1107 of the LDMVP list generation algorithm, the method steps of the replacePosition program are included as step S4000 in Fig.41 The input parameters of the program are: test position (x, y), current CTU position (x CTU ,y CTU ), and a set R of rectangles defining an area in a CTU surrounding the current CTU, from which allowed motion vector candidates are selected.

[0514] Fig.40 A flowchart of an exemplary implementation of the program replacePosition is shown. The program performs a check to see whether a given position (x, y) belongs to the current CTU or to a rectangle R i One of (steps S4001 and S4003). i Check S4003 is performed (step S4002). If the position (x, y) does not belong to the current CTU or one of the rectangles, then in step 4004 the position (x, y) is modified to the position (x mod ,y mod ) to meet the above requirements, that is, the position (x, y) and (x mod ,y mod ) is the shortest distance between them. The modified position (x mod ,y mod ) can be calculated as the closest to the position (x, y) belonging to the current CTU or rectangle R i If the position (x, y) belongs to the current CTU or any rectangular region R i , then in step S4005, the modified position (x mod ,y mod ) assigns the value (x,y), and (x mod ,y mod )=(x,y). This procedure returns one or more modified positions (x mod ,y mod ) as output (step S4006).

[0515] The term "closest" refers to the position (x, y) and the modified position (x mod ,y mod ). The distance can be calculated using a metric, such as the Euclidean metric or the p-norm, where p is equal to or greater than 1. The position (x) can be determined or calculated by any other method that satisfies the above requirements. mod ,y mod ).

[0516] Fig.41 The flowchart in FIG. 1 shows an exemplary implementation of the LDMVP list generation algorithm including the replacePosition procedure. The set R may be empty, which means that the procedure modifies the input position (x, y) to the position (x mod ,y mod ), thus belonging to the current CTU. Additionally or alternatively, the replacePosition procedure can replace two different positions (x0, y0), (x1, y1) with the same position (x mod ,y mod ). This means that one of the positions (x0, y0) or (x1, y1) is deleted. Therefore, in this case, the number of MV candidate positions is reduced by 1.

[0517] Figure 42 to Figure 45 Shows the Figure 8 Some examples of applying the replacePosition procedure in the "diagonal" mode shown in , where the CU position is relative to the CTU position (x CTU ,y CTU ) of pixels (8,8). In these examples, rectangle R i Set to [x 0 i ,y 0 i ,w,h], where (x 0 i ,y 0 i ) is the rectangle relative to position (x CTU ,y CTU ), w and h are the width and height of the rectangle respectively.

[0518] For these exemplary settings, Table 2 provides the values ​​for each of the 5 to 8 rectangles R i The corresponding setting of each example is the rectangle set {R i} and the accompanying figure, showing the application of the replacePosition procedure to the "diagonal" mode. Positions removed / rejected by the replacePosition procedure are marked with a black and white hash pattern, and positions added by the replacePosition procedure are marked with a vertical line pattern. Rectangle R i Marked with a horizontal line pattern.

[0519] Table 2 replacePosition procedure applied on examples 5 to 8

[0520]

[0521] Although several embodiments have been provided in the present disclosure, it should be understood that the disclosed systems and methods may be implemented in many other specific forms without departing from the spirit or scope of the present disclosure. The present examples are considered to be illustrative rather than restrictive, and the invention is not limited to the details given herein. For example, various elements or components may be combined or integrated in another system, or certain features may be omitted or not implemented.

[0522] In addition, without departing from the scope of the present disclosure, the techniques, systems, subsystems, and methods described and shown as discrete or separate in various embodiments may be combined or integrated with other systems, modules, techniques, or methods. Other items shown or discussed as coupled or directly coupled or communicating with each other may be indirectly coupled or communicated electrically, mechanically, or otherwise through some interface, device, or intermediate component. Other examples of changes, substitutions, and alterations may be determined by those skilled in the art and may be made without departing from the spirit and scope disclosed herein.

[0523] In general, the present disclosure relates to encoding and decoding of video images. A motion vector is used as a prediction value to predict an image of a current coding unit (CU) within a current video frame. The motion vector comes from a list of motion vector candidates (MVCs), which is generated. The list includes MVs determined according to a first mode and / or a second mode, each mode specifying a MV candidate position. The first MV position is within the current video frame, and the second MV position is for a video frame different from the current frame. An MVC list is generated for the current coding unit (CU), wherein the MV position is relative to the position of the current CU. In particular, the MVC list is generated based on the size of the current CU and the size of a grid specifying the minimum distance between two MV positions. One or more MVs specified by the MV position of the mode are included in the MVC list.

[0524] According to one aspect of the present disclosure, a device is provided for generating a list of one or more motion vector candidates (MVCs), the device comprising a processing circuit for determining the MVC for a current coding unit (CU) based on a mode position relative to a position of a current CU, the mode defining a position of a motion vector (MV) and a position of the current CU within a coding tree unit (CTU).

[0525] The motion vector candidate is used for motion prediction of the current CU. This means that the motion information of the motion vector at the corresponding candidate position is used for prediction. Therefore, the motion vector position is the position of the corresponding vector candidate block. The block can be one pixel / sample and / or multiple samples corresponding to the sample block.

[0526] The position of the current CU may be the upper left corner of the current CU, where the position is in pixels or in pixel block units. For example, the block unit may be a 2x2 pixel block or any other unit. In VVC, a grid size of 4x4 is more common for motion information storage. In addition, the sparsity of motion information storage may be different for the current frame and for other frames. For example, in VVC, the motion information of the current frame is stored in a 4x4 grid, while an 8x8 grid is used for other frames. For example, other frames include frames from which motion information is used based on the TMVP / ATMVP mode. The position of the current CU (the CU is within the current CTU) may be the distance from the upper left corner of the current CU to the (pixel / sample) position of the upper left corner of the current CTU. The pattern position relative to the position of the current CU may be predetermined. This means that the relative position may be set to a predefined value.

[0527] Determining the MVC using the relative position information between the current CU and the pattern and the current CU position within the CTU may have the following advantages: the prediction of the CU of the current frame is improved using the relative position information. In particular, using the MV position of the pattern provides the advantageous effect that only a few positions from which possible motion information is obtained and / or acquired are used. Thus, using a limited number of positions as defined by the pattern results in fewer read and write operations when accessing motion information, for example. Thus, a small number of pattern positions makes prediction more efficient and faster.

[0528] Furthermore, the above-mentioned pattern may be any pattern suitable for determining motion information based on the motion vector at the pattern position. In other words, the pattern may be adjusted according to its size and / or extension relative to the current CU within the video frame, and the pattern may be selected or designed to have a specific structure and / or shape. For example, a pattern (i.e., a pattern position) may be determined or generated so that the pattern position has the following structure: Figures 7 to 10 The pattern examples shown are "square", "diagonal", "triangle", or "star" structures. The above patterns are not limited to Figures 7 to 10 Patterns other than those shown in , may have any other shape / structure suitable for limiting the motion information obtained at the pattern-based positions. Thus, the advantage of using patterns is that the prediction can be made more flexible, so that only the motion information of the positions limited by the pattern positions is used. This may include pattern positions that can be added and / or replaced, etc.

[0529] According to one aspect of the present disclosure, the processing circuit of the above apparatus is used to determine the MVC for the current CU, including selecting a subset of MV positions from the pattern positions.

[0530] This means that the initial pattern (ie, initial position) may be subject to additional inspection, based on which certain pattern positions may be retained, discarded, and / or changed. In other words, one or more pattern positions may be selected.

[0531] An advantage of selecting a subset of MV positions is reduced complexity. In particular, this means that unnecessary data read / write operations and / or unnecessary access to data of some neighboring blocks are avoided. Therefore, a further technical advantage may be that storing additional data is avoided. Therefore, memory requirements may be reduced. Furthermore, the selection of a subset may provide the advantage of further adapting some pattern positions, which further improves the prediction accuracy.

[0532] In an exemplary embodiment of the present disclosure, the selection includes selecting an MV position located in the current CTU from the pattern positions.

[0533] This may provide the advantage that unnecessary reading and writing of data in the buffer is avoided since the information is already available for the position within the current CTU.

[0534] In an exemplary embodiment of the present disclosure, selecting the subset of MV positions is performed by excluding from the pattern positions those MV positions that are not available and / or not stored in the buffer.

[0535] This means that some pattern positions may be outside the current video frame, i.e., the position is larger than the current frame size. Therefore, the information at that point does not exist. This may also limit the size / extension of the pattern when generating it.

[0536] Motion information may also not be available at a certain location because of the order in which encoding / decoding is performed for the CUs of the frame. In other words, since the lower right region of the CTU is encoded / decoded at a later time, motion information for that region does not yet exist at a particular time. Therefore, even if a location in that region is generated (i.e., as part of a pattern), motion information for that point does not exist at that moment in time.

[0537] Furthermore, the motion information may not be stored in a memory (i.e., a buffer) because the corresponding information can be accessed. Such a buffer may be, for example, a row buffer, which has fast read and write access times and is typically used to store motion information for a restricted region of a CTU adjacent to the current CTU for which motion prediction is performed. A row buffer is a fast data storage that is read and / or written to an entire buffer row in parallel.

[0538] Excluding pattern positions that are not available and / or not stored in the buffer may have the following advantages: pattern positions for which motion information does not exist, and thus it is not possible to obtain motion information from those positions, are not used. Thus, excluding these points may further reduce the number of pattern positions to those for which motion information is actually available. Thus, the prediction may become faster since fewer pattern positions may be used for motion prediction. Furthermore, unnecessary read and write accesses to motion information at those motion vector positions for which motion information is not available may be avoided. This makes the prediction more efficient and faster.

[0539] According to one aspect of the present disclosure, the processing circuit is used to determine the MVC for the current CU, including replacing one or more excluded MV positions with alternative MV positions, wherein the alternative MV positions are determined based on the excluded MV positions to achieve one or more predetermined conditions.

[0540] This means that, for example, one or more excluded / discarded pattern positions may have their coordinate positions changed by replacing or calculating new positions under the constraints of predetermined conditions.

[0541] The above replacement is for and based on discarded positions, which correspond to those pattern positions for which motion information is not available and / or not stored in the buffer. The discarded positions are also referred to as rejected positions and / or excluded positions, which terms may be used interchangeably.

[0542] The advantage of position replacement of discarded positions is that the number of positions of possible motion vector candidates is not reduced compared to the number of positions of the original pattern. In other words, a sufficient number of MV positions may be needed to perform prediction with high accuracy and quality. This can be achieved by replacing some of the excluded pattern positions with positions for which motion information is available. This can maintain the number of MV positions close to the number of pattern positions.

[0543] The replacement position is determined based on the discarded position so that the replacement position satisfies some predetermined conditions. In other words, the replacement position is determined using a combination of the position information of the discarded position and one or more constraints. This may provide the advantage that parts of the position information of the original pattern (e.g., the shape / structure of the original pattern has been selected specifically for the content of the video frame) are fully and / or partially transferred to the replacement position.

[0544] According to an exemplary embodiment of the present disclosure, the one or more predetermined conditions include a condition that a replacement MV position is available and / or stored in a buffer.

[0545] According to an exemplary embodiment of the present disclosure, the substitute MV position is determined to satisfy a condition that a distance between the substitute MV position and the excluded MV position is the shortest.

[0546] The term "distance" refers to a measure of the magnitude of the spatial separation between two points along a particular direction. The spatial separation may be measured in units of the number of pixels along the pixel x and / or pixel y directions, using pixels as the unit of spatial measurement. The distance may be calculated based on a metric, such as the Euclidean norm and / or the p-norm, where p is equal to or greater than 1.

[0547] The "distance" condition is an example of a predetermined condition and is not limited to this condition. For example, the distance condition can be combined with a distance cutoff corresponding to a threshold. The discarded position can be replaced with a position that makes the distance equal to or less than the cutoff value. In other words, the distance may not be the minimum. Other options and / or combinations of different conditions can be applied so that replacement positions are available and / or stored in the buffer.

[0548] In the best case, the position replacement can make it possible to have the same number of positions as the number of discarded positions, providing the beneficial effect of preserving the number of positions. In addition, determining the replacement position for the shortest distance between the replacement position and the discarded position can provide the advantage that the spatial position of the motion vector candidate position is close to the shape / structure of the original pattern. Therefore, the shortest distance makes it possible to preserve the shape / structure of the original pattern. Therefore, the prediction can be performed more accurately and with high precision.

[0549] According to one aspect of the present disclosure, the processing circuit of the above apparatus is further configured to determine an MVC for a current CU, including adjusting the excluded MV position to an adjusted MV position, wherein the adjusted MV position is determined such that it is available and / or stored in a buffer.

[0550] This means that the discarded pattern positions are adjusted, and thus the adjusted MV positions are determined so that these positions are available and / or stored in the buffer. Since in this embodiment, all discarded positions are adjusted, the number of MV candidate positions is the same as the number of pattern positions. In other words, the number of MV positions is maintained.

[0551] This may provide the advantage that predictions are performed using a sufficient number of candidate locations, improving prediction accuracy.

[0552] Additionally or alternatively, two different mode positions may be replaced by only one position. In this case, the number of MVC positions is reduced by 1.

[0553] According to an exemplary embodiment of the present disclosure, determining the adjusted MV position includes assigning the excluded MV position to the assigned MV position in the x direction along the width of the current CTU and / or in the y direction along the height of the current CTU.

[0554] The x-direction and y-direction, respectively, the x-position and y-position are in pixel coordinates, where the x-pixel axis is along the width of the CTU and the y-pixel axis is along the height of the CTU.

[0555] Thus, the discarded position can be assigned to any position for which motion information is available for its x position component and / or y position component. In other words, the position assignment can be performed in either direction.

[0556] This may provide the advantage that the allocation is performed in a very flexible manner, allowing fine tuning of the MV candidate positions.Thus, due to the spatial fine tuning of the MV candidate positions, their prediction accuracy may be further improved.

[0557] According to another exemplary embodiment of the present disclosure, the determining includes reallocating the x position or y position of one or more allocated MV positions having the same x position or the same y position to the reallocated y position or x position using a predetermined scheme.

[0558] The solution may be a function. This means, for example, that the reallocated positions may be calculated using said function, with the allocated positions as input.

[0559] According to an exemplary embodiment of the present disclosure, the predetermined scheme is to shift the y component or the x component of the assigned MV position by a predetermined offset value.

[0560] The offset value may be a number of pixels, thereby shifting the y and / or x position (e.g., in pixels) by a pixel offset by adding and / or subtracting the offset value from the adjusted y and / or x position value. Alternatively, the predetermined scheme may be a factor by which the y and / or x position may be multiplied and / or divided. Other schemes are also possible for reallocating the allocated positions when their x and / or y positions are the same.

[0561] Thus, the reallocation of positions (e.g., by shifting any of the x-components or y-components of the allocated positions) spreads out those allocated positions that may have the same x-position and / or y-position. In other words, after the position allocation, some of the discarded positions may be located at the same position. The component-wise reallocation of the allocated positions that have the same x-position or y-position is to spread out these positions to different positions that are still available and / or stored in the buffer.

[0562] This may provide the advantage that position redundancy is avoided, since some candidate positions may be located at the same position and thus provide the same motion information. Thus, dispersing the allocation positions increases the number of candidate positions, which may have different motion information since they are located at different positions. Thus, prediction may be performed more accurately using a sufficient number of MV candidate positions.

[0563] Once the motion vector candidate positions are determined using any of the above embodiments of the present disclosure, it is checked whether the motion vectors of these candidate positions are already included in the MVC list. If not, the motion vector is included in the list.

[0564] According to one aspect of the present disclosure, an encoder is provided for encoding a current decoding unit (CU) of a video frame, the encoder comprising: a device for generating a list of one or more MVCs according to any of the above aspects of the present disclosure; an inter-frame prediction unit, for determining a prediction of a current CU based on at least one motion vector (MV) in the MVC in the list; and a compression unit, for encoding the current CU by using the prediction of the CU.

[0565] According to one aspect of the present disclosure, a decoder is provided for decoding a current coding unit (CU) of a video frame, the decoder comprising: a device for generating a list of one or more MVCs according to any of the above aspects of the present disclosure; an inter-frame prediction unit for determining a prediction of a current CU based on at least one motion vector (MV) in the MVC in the list; and a decompression unit for decoding the current CU by using the prediction of the CU.

[0566] MVC can be used for motion prediction using the motion information in the motion vector obtained at the MVC position. The current CU can be encoded and / or decoded by calculating the difference between the CU and the prediction. Alternatively or additionally, the index of the list of MVC can be encoded, for example, even if there is no residual because the prediction is already close to the CU (i.e. the residual is almost zero). In principle, the index of the list can also be determined implicitly on the encoder side and the decoder side.

[0567] According to one aspect of the present disclosure, a method for generating a list of one or more motion vector candidates (MVCs) is provided, the method comprising the following steps: determining the MVC for a current coding unit (CU) according to a mode position relative to a position of a current CU, the mode defining a position of a motion vector (MV) and a position of the current CU within a coding tree unit (CTU).

[0568] According to one aspect of the present disclosure, there is provided a computer-readable non-transitory medium including instructions for storing a program, which, when executed on a processor, causes the processor to perform the above method steps.

[0569] Reference Numbers List

[0570] Figure 1A

[0571] 10 Video Codec System

[0572] 12 Source Device

[0573] 13 communication channels

[0574] 14 Destination Device

[0575] 16 Image Source

[0576] 17 Image Data

[0577] 18 Preprocessor

[0578] 19 Preprocessing image data

[0579] 20 Video Encoder

[0580] 21 Encoded Image Data

[0581] 22 Communication Interface

[0582] 28 Communication Interface

[0583] 30 video decoders

[0584] 31 Decoding image data

[0585] 32 Postprocessor

[0586] 33 Post-processing image data 34 Display device

[0587] Figure 1B40 Video encoding and decoding system

[0588] 41 Imaging equipment

[0589] 42 Antenna

[0590] 43 Processor

[0591] 44 Memory

[0592] 45 Display Devices

[0593] 46 Processing Circuit

[0594] 20 Video Encoder 30 Video Decoder

[0595] Figure 2

[0596] 17 pictures (data)

[0597] 19 Preprocessing pictures (data) 20 Encoder

[0598] 21 Encoded picture data 201 input (interface)

[0599] 204 residual calculation [unit or step] 206 transformation processing unit

[0600] 208 Quantization Unit

[0601] 210 Inverse Quantization Unit

[0602] 212 Inverse transform processing unit

[0603] 214 Reconstruction unit 220 Loop filter unit 230 Decoded picture buffer (DPB) 260 Mode selection unit 270 Entropy coding unit 272 Output (interface)

[0604] 244 Inter prediction unit

[0605] 254 Intra prediction units

[0606] 262 Segmentation Unit

[0607] 203 Image Blocks

[0608] 205 Residual Block

[0609] 213 Reconstruction residual block

[0610] 215 Reconstruction Block

[0611] 221 filter block

[0612] 231 Decoded Image

[0613] 265 prediction block

[0614] 266 Syntax elements

[0615] 207 Transformation coefficients

[0616] 209 Quantization coefficient 211 Dequantization coefficient

[0617] Figure 3

[0618] 21 Encoded Image Data

[0619] 30 Video Decoder

[0620] 304 Entropy decoding unit

[0621] 309 Quantization coefficients

[0622] 310 Inverse Quantization Unit

[0623] 311 Dequantization coefficient

[0624] 312 Inverse transform processing unit

[0625] 313 Reconstruction residual block

[0626] 314 Reconstruction Unit

[0627] 315 Reconstruction Block

[0628] 320 loop filter 321 filter block 330 decoded picture buffer DBP 331 decoded picture

[0629] 360 Mode Application Unit

[0630] 365 prediction block

[0631] 366 Syntax elements

[0632] 344 inter-frame prediction unit 354 intra-frame prediction unit

[0633] Figure 4

[0634] 400 video codec device 410 ingress port / input port 420 receiver unit Rx 430 processor 440 transmitter unit Tx 450 egress port / output port 460 memory

[0635] 470 Codec Module

[0636] Figure 5

[0637] 500 source device or destination device 502 processor

[0638] 504 Memory

[0639] 506 Code and Data

[0640] 508 Operating System

[0641] 510 Applications

[0642] 512 bus

[0643] 518 Display

[0644] Figure 6

[0645] 600 MVP Candidates

[0646] Figure 7

[0647] 700 square LDMVP mode

[0648] Figure 8

[0649] 800 diagonal LDMVP mode

[0650] Fig. 9

[0651] 900 triangle LDMVP mode

[0652] Fig.10

[0653] 1000 Star LDMVP Mode

[0654] Fig.11

[0655] 1100 LDMVP List Generation Flowchart

[0656] Fig.12

[0657] 1200 GeneratePattern Flowchart

[0658] Fig.13

[0659] 1300 "Sun" LDMVP Mode with R = 3

[0660] Fig.14

[0661] 1400 "Sun" LDMVP Mode with R = 4

[0662] Fig.15

[0663] 1500 Candidate Order of "Sun" LDMVP Mode in Iteration 1 (R = 3) Fig.16

[0664] 1600 Candidate Order of "Sun" LDMVP Mode in Iteration 3 (R = 4) Fig.17

[0665] 1700 GeneratePatternTMVP Flowchart

[0666] Fig.18

[0667] 1800 Combined LDMVP + TMVP Mode with R = 3 and Rt = 1 Fig.19

[0668] 1900 Combined LDMVP + TMVP Mode with R = 3 and Rt = 2 Fig. 20

[0669] 2000 Combined LDMVP + TMVP Mode with R = 3 and Rt = 2 Fig.21

[0670] 2100 Combined LDMVP + TMVP Mode with R = 3 and Rt = 2 Fig. 22

[0671] 2200 Candidate Order of LDMVP + TMVP Mode with R = 3 and Rt = 1 Fig.23

[0672] 2300 Combined Mode with Only TMVP - Example 1

[0673] Fig.24

[0674] 2400 TMVP-only Combo Mode – Example 2

[0675] Fig.25

[0676] 2500 TMVP mode mirrored by LDMVP mode with R=3

[0677] Fig.26

[0678] 2600 TMVP mode based on partial mirroring of LDMVP mode with R=3

[0679] Fig. 27

[0680] 2700 Modified LDMVP mode by adding six adjacent candidate points

[0681] Fig.28

[0682] 2800 Modified LDMVP mode by adding four adjacent candidate points

[0683] Fig.29

[0684] 2900 combination modes, including LDMVP mode, TMVP mode, and adjacent candidate points Fig.30

[0685] 3000 Unadjusted "Sun" mode

[0686] Fig.31

[0687] 3100 Grouping of unadjusted "sun" mode positions

[0688] Fig.32

[0689] 3200 Adjust the "sun" mode position according to the group

[0690] Fig.33

[0691] 3300adjustCandidatePositions Flowchart

[0692] Fig.34

[0693] 3400additionalChecks Flowchart

[0694] Fig.35

[0695] 3500 LDMVP list generation flow chart with additionalChecks

[0696] Fig.36

[0697] 3600 Apply additionalChecks for “Diagonal” mode – Result Example 1

[0698] Fig.37

[0699] 3700 Apply additionalChecks for “Diagonal” mode – Result Example 2

[0700] Fig.38

[0701] 3800 Apply additionalChecks for “Diagonal” mode – Result Example 3

[0702] Fig.39

[0703] 3900 Apply additionalChecks for “diagonal” mode – Result Example 4

[0704] Fig.40

[0705] 4000replacePosition Flowchart

[0706] Fig.41

[0707] 4100LDMVP list generation flow chart with replacePosition

[0708] Fig.42

[0709] 4200 Apply replacePosition to the "Diagonal" mode – Result Example 5

[0710] Fig.43

[0711] 4300 Apply replacePosition to the "Diagonal" mode – Result Example 6

[0712] Fig.44

[0713] 4400 Apply replacePosition to the "Diagonal" mode – Result Example 7

[0714] Fig.45

[0715] 4500 Apply replacePosition to the “diagonal” mode – Result example 8

[0716] Definition of abbreviations and terms

[0717] MV Motion vector, motion vector

[0718] MVP Motion vector prediction, motion vector prediction

[0719] LDMVP Long-distance motion vector predictor, long-distance motion vector prediction value TMVP Temporal motion vector prediction, temporal motion vector prediction CTU Coding tree unit, coding tree unit

[0720] LCU Largest coding unit, maximum coding unit

[0721] CU Coding unit, decoding unit.

Claims

1. A device for generating a motion vector candidate MVCs list for a current decoding unit CU, wherein: The device comprises: Processing circuitry for: Include the following in the list of MVCs: - one or more motion vectors MVs determined according to a first mode, wherein the first mode specifies the MV position within the current video frame where the current CU is located; and - one or more MVs determined according to a second mode, said second mode specifying MV positions in a video frame different from said current video frame.

2. The device according to claim 1, wherein: The processing circuit is used to include the following in the MVC list: one or more MVs determined based on a position adjacent to the current CU in the current video frame, the position being different from the MV position in the first mode.

3. According to the device according to any one of claims 1 or 2, the processing circuit is used to determine the first mode, including determining a new MV position for one or more MV positions that are not available and / or not stored in the buffer in the first mode, so that the new MV position is available and / or stored in the buffer.

4. The device according to claim 3, wherein: The determining the new MV position includes assigning an x ​​position and / or a y position of the one or more MV positions to the new MV position and discarding the one or more MV positions from the first pattern.

5. The device according to claim 4, wherein: The determining includes assigning a new y position or a new x position to one or more of the new MV positions having the same x position or the same y position using a predetermined scheme.

6. The device according to claim 5, wherein: The predetermined solution is to shift the assigned y position or x position of the new MV position by a predetermined offset value.

7. The device according to any one of claims 1 to 6, wherein: Determining the second pattern includes reflecting one or more of the MV positions of the first pattern with reference to a reflected line.

8. The device according to any one of claims 1 to 7, wherein: The processing circuit is configured to generate a pattern relative to the position of the current CU according to: -The size of the current CU; - the size of the grid that specifies the minimum distance between two MV positions belonging to the pattern; Therein, the mode specifies the position of the motion vector MVs and the mode is the first mode and / or the second mode.

9. The device according to claim 8, wherein: The processing circuit is configured to determine the second mode and / or the first mode according to at least one of the following: - an iteration value specifying the size of the second pattern and / or the first pattern on the grid; -The minimum size that can be configured for a CU; as well as - Maximum size that can be configured for a CU.

10. A device for generating a motion vector candidate MVCs list for a current decoding unit CU, the device comprising: Processing circuitry for: The mode specifying the position of the motion vector MVs relative to the position of the current CU is determined at least according to: -The size of the current CU; and - the size of the grid that specifies the minimum distance between two MV positions belonging to the pattern; as well as One or more MVs specified by the MV position of the pattern are included in the MVC list.

11. The device according to any one of claims 8 to 10, wherein: The processing circuit is configured to determine the mode position according to a first iteration value and / or a minimum size or a maximum size that can be configured for the CU.

12. The device according to any one of claims 8 to 11, wherein: The processing circuit is configured to adapt the grid size according to the size of the current CU and / or a minimum value and a maximum value of the grid size.

13. The device according to claims 8 to 12, wherein: The processing circuit is used to determine a neighboring position that specifies a MV position different from the first pattern within the current video frame, and to combine the neighboring position and the first pattern.

14. An encoder for encoding a current decoding unit CU of a current video frame, comprising: The device for generating a list of motion vector candidate MVCs according to any one of claims 1 to 13; An inter-frame prediction unit, configured to determine a prediction of the current CU according to at least one motion vector MV in the MVC list; as well as A compression unit is configured to encode the current CU by using the prediction of the CU.

15. A decoder for decoding a current decoding unit CU of a video frame, comprising: The device for generating a list of motion vector candidate MVCs according to any one of claims 1 to 13; An inter-frame prediction unit, configured to determine a prediction of the current CU according to at least one motion vector MV in the MVC list; as well as A decompression unit is configured to decode the current CU by using the prediction of the CU.

16. A method for generating a motion vector candidate MVCs list for a current decoding unit CU, wherein: The method comprises the following steps: Include the following in the list of MVCs: - one or more motion vectors MVs determined according to a first mode, wherein the first mode specifies the MV position within the current video frame where the current CU is located; and - one or more MVs determined according to a second pattern, said second pattern specifying MV positions in a video frame different from said current video frame.

17. A method for generating a motion vector candidate MVCs list for a current decoding unit CU, wherein: The method comprises the following steps: The mode specifying the position of the motion vector MVs relative to the position of the current CU is determined at least according to: -The size of the current CU; and - the size of the grid specifying the minimum distance between two MV positions belonging to the pattern; and One or more motion vectors specified by the MV positions of the mode are included in the list.

18. A computer-readable non-transitory medium storing a program, the program comprising instructions, which, when executed on a processor, cause the processor to perform the method according to claim 16 and / or claim 17.

19. A bitstream comprising an index to a motion vector candidate MVC list, wherein the index is used to indicate a motion vector used for inter-frame prediction of a current CU.

20. A video streaming device, comprising a communication interface, the communication interface being used to receive and / or send a video stream, the video stream comprising an index of a motion vector candidate MVC list, the index being used to indicate a motion vector used for inter-frame prediction of a current CU.

21. A content server, comprising a communication interface, wherein the communication interface is used to receive and / or send a video stream, wherein the video stream is generated by the encoder according to claim 14.

Images