Method and related device for video encoding and / or decoding with bidirectional optical flow simplification with shift operation

Abstract

A method for performing bidirectional optical flow (BDOF) processing on a video sequence of images, wherein each image includes a plurality of blocks having a bidirectionally predicted inter-frame coding block (BPICB). The method comprises: obtaining a shifted pair of refinement parameters for each sub-block of the bidirectionally predicted inter-frame coding block, wherein the shifted pair of refinement parameters comprises a shifted first refinement parameter and a shifted second refinement parameter. The method comprises: determining a BDOF offset without performing any right shift using at least the shifted pair of refinement parameters by: for each sample in each sub-block within each bidirectionally predicted inter-frame coding block, using the shifted first refinement parameter, the shifted second refinement parameter, the first horizontal gradient block and the first vertical gradient block of the first prediction block, and the second horizontal gradient block and the second vertical gradient block of the second prediction block to determine the BDOF offset.

Description

Technical Field

[0001] The present disclosure relates generally to video processing, and more particularly, to video encoding and / or decoding and related methods and devices. Background Art

[0002] A video sequence contains a series of pictures. The color space commonly used in video sequences is YCbCr, where Y is the luminance (brightness) component and Cb and Cr are the chrominance components. Pictures are placed in the order in which they are displayed. Each picture is assigned a picture order count (POC) value to indicate its display order.

[0003] Video coding is used to compress a video sequence into a series of coded pictures. Typically, the pictures are divided into blocks of size 4x4 to 128x128. A block is a two-dimensional array of samples. Blocks serve as the basis for coding. The video decoder then decodes the coded pictures into pictures containing sample values.

[0004] The Moving Picture Experts Group (MPEG) and the ITU Telecommunication Standardization Sector (ITU-T) are developing a new video coding standard called Versatile Video Coding (VVC). Currently, the current version of the VVC draft specification is the output document approved by JVET: B. Bross, J. Chen, S. Liu, "Versatile Video Coding (Draft 6)", document number JVET-O2001 (also referred to as "VVC Draft 6" in this article).

[0005] The VVC Specification Draft 6 video coding standard uses a block structure called a quadtree plus binary tree plus ternary tree block structure (QTBT+TT), in which each picture is first divided into square blocks called coding tree units (CTUs). The size of all CTUs is the same, and the division is completed without any syntax control. Each CTU is further divided into coding units (CUs), which can have a square shape or a rectangular shape. The CTU is first divided by a quadtree structure, and then can be further divided using partitions of equal size in the vertical or horizontal direction in a binary structure to form coding units (CUs). Therefore, the block can have a square shape or a rectangular shape. The depth of the quadtree and the binary tree can be set by the encoder in the bitstream. Figure 1 An example of partitioning a CTU using QTBT is illustrated. The Ternary Tree (TT) part increases the possibility of partitioning a CU into three partitions instead of two equally sized partitions; this increases the possibility of using a block structure that is better suited to the content structure in the picture.

[0006] In order to achieve efficient compression in the time domain, inter-frame prediction techniques aim to exploit the similarities between pictures. Inter-frame prediction uses previously decoded pictures to predict blocks in the current picture. The previously decoded pictures are called reference pictures for the current picture.

[0007] In video encoders, a method called motion estimation can be used to find the most similar block in a reference picture. The displacement between the current block and its reference block is a motion vector (MV). MV has two components, MV.x and MV.y, i.e., the x direction and the y direction. Figure 2 An example of a MV between a current block and its reference block is illustrated. The MV may be signaled in the video bitstream.

[0008] A video decoder can decode the MV from the video bitstream. The decoder can then apply a method called motion compensation, which can use the MV to find a corresponding reference block in a reference picture.

[0009] If a block is predicted from at least one reference block in a reference picture, the block may be referred to as an inter block.

[0010] The number of reference blocks is not limited to one. In bidirectional motion compensation, two reference blocks can be used to further explore temporal redundancy, for example, the current block can be predicted based on two previously decoded blocks. Pictures using bidirectional motion compensation can be called bi-predicted pictures (B pictures). Figure 3 An example of a block with bidirectional motion compensation is illustrated.

[0011] The set of motion information can include MV (MV.x and MV.y) and reference pictures with POC numbers. Figure 3 As shown, if bidirectional motion compensation is used, there may be two sets of motion information, for example, set 0 has MV0, POC1 and associated block 0, and set 1 has MV1, POC2 and associated block 1.

[0012] The following explanation of potential problems is a current implementation as part of the present disclosure and should not be construed as previously known to others. Although calculation of certain parameters may be supported in VVC Draft 6 specification, the accuracy of the calculation may be reduced and may be inefficient depending on factors in the process. Summary of the invention

[0013] According to various embodiments of the present invention, there is provided a method for performing bidirectional optical flow (BDOF) processing on a video sequence including a plurality of images, wherein each of the plurality of images includes a plurality of blocks having a bidirectionally predicted inter-frame coding block. The method comprises: for each sub-block of the bidirectionally predicted inter-frame coding block, obtaining a shifted pair of refinement parameters, wherein the shifted pair of refinement parameters comprises a shifted first refinement parameter and a shifted second refinement parameter. The method comprises: determining the BDOF offset by the following operation without performing any right shift using at least the shifted pair of refinement parameters: for each sample in each sub-block within each bidirectionally predicted inter-frame coding block, using the shifted first refinement parameter, the shifted second refinement parameter, the first horizontal gradient block and the first vertical gradient block of the first prediction block, and the second horizontal gradient block and the second vertical gradient block of the second prediction block to determine the BDOF offset.

[0014] Compared to the existing method of VVC specification draft 6, one advantage that can be achieved is that two right shift operations for each position in the 4x4 sub-block can be avoided. Another advantage includes: in various embodiments, a method for saving the number of shift operations can be provided. Various embodiments include two shifts of a pair of refinement parameters (e.g., refinement parameters Vx and Vy) for each sub-block. Therefore, for a CU of size 128x128, only 1024x2=2048 shifts are required, which is only 6.25% of the number of shifts in VVC specification draft 6.

[0015] According to other embodiments of the present invention, there is provided an electronic device for performing bidirectional optical flow BDOF processing on a video sequence including a plurality of images, wherein each of the plurality of images includes a plurality of blocks having a bidirectionally predicted inter-frame coding block. The electronic device may include at least one processor and at least one memory, wherein the at least one memory is connected to the at least one processor and stores a program code, and the program code is executed by the at least one processor to perform an operation. The operation includes: for each sub-block of the bidirectionally predicted inter-frame coding block, a shifted pair of refinement parameters is obtained, wherein the shifted pair of refinement parameters includes a shifted first refinement parameter and a shifted second refinement parameter. The operation includes: determining the BDOF offset by the following operation without performing any right shift using at least a shifted pair of refinement parameters: for each sample in each sub-block within each bidirectionally predicted inter-frame coding block, the BDOF offset is determined using the shifted first refinement parameter, the shifted second refinement parameter, the first horizontal gradient block and the first vertical gradient block of the first prediction block, and the second horizontal gradient block and the second vertical gradient block of the second prediction block.

[0016] According to other embodiments of the present invention, there is provided a computer program including a program code to be executed by at least one processor of an electronic device, the execution of which causes the electronic device to perform an operation. The operation includes: for each sub-block of a bidirectionally predicted inter-frame coding block, obtaining a shifted pair of refinement parameters, wherein the shifted pair of refinement parameters includes a shifted first refinement parameter and a shifted second refinement parameter. The operation includes: determining a BDOF offset by the following operation without performing any right shift using at least a shifted pair of refinement parameters: for each sample in each sub-block within each bidirectionally predicted inter-frame coding block, using the shifted first refinement parameter, the shifted second refinement parameter, the first horizontal gradient block and the first vertical gradient block of the first prediction block, and the second horizontal gradient block and the second vertical gradient block of the second prediction block to determine the BDOF offset.

[0017] According to some further embodiments of the inventive concept, a computer program product includes a non-transitory storage medium including a program code to be executed by at least one processor (402, 420) of an electronic device (410), whereby execution of the program code causes the electronic device to perform operations. The operations include: for each subblock of a bidirectionally predicted inter-frame coding block, obtaining a shifted pair of refinement parameters, wherein the shifted pair of refinement parameters includes a shifted first refinement parameter and a shifted second refinement parameter. The operations include: determining a BDOF offset without performing any right shift using at least the shifted pair of refinement parameters by the following operations: for each sample in each subblock within each bidirectionally predicted inter-frame coding block, determining the BDOF offset using the shifted first refinement parameter, the shifted second refinement parameter, a first horizontal gradient block and a first vertical gradient block of a first prediction block, and a second horizontal gradient block and a second vertical gradient block of a second prediction block. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings illustrate certain non-limiting embodiments of the inventive concept, and are included to provide a further understanding of the present disclosure, and are incorporated in and constitute a part of this application. In the drawings:

[0019] Figure 1 An example of dividing a coding tree unit into coding units using a quadtree plus binary tree plus ternary tree block structure is shown;

[0020] Figure 2 An example of a motion vector between a current block and its reference block is shown;

[0021] Figure 3 An example of a block with bidirectional motion compensation is shown;

[0022] Figure 4is a block diagram illustrating an electronic device having picture processing circuitry configured according to some embodiments;

[0023] Figure 5 shows an example of the performance of a video encoder / decoder according to some embodiments;

[0024] Figure 6 is a flowchart of operations that can be performed by picture encoding circuitry and / or picture decoding circuitry for determining bidirectional optical flow related parameters to perform prediction of luma sample values ​​during encoding and / or decoding of a video according to some embodiments of the present disclosure; and

[0025] Figure 7 is a flowchart of operations that can be performed by a picture encoding circuit and / or a picture decoding circuit for determining bidirectional optical flow related parameters according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0026] Hereinafter, the inventive concept will be described more fully with reference to the accompanying drawings, in which examples of embodiments of the inventive concept are shown. However, the inventive concept can be embodied in a variety of different forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided so that the disclosure will be comprehensive and complete, and the scope of the inventive concept will be fully conveyed to those skilled in the art. It should also be noted that these embodiments are not mutually exclusive. A component from one embodiment can be assumed by default to be present in / used in another embodiment.

[0027] The following description presents various embodiments of the disclosed subject matter. These embodiments are presented as teaching examples and are not to be construed as limiting the scope of the disclosed subject matter. For example, some details of the embodiments may be modified, omitted or expanded without departing from the scope of the subject matter.

[0028] The bidirectional optical flow (BDOF) method can use the concept of optical flow and can be combined with bidirectional inter prediction to predict the luma sample value in the current block. When the BDOF method is enabled for a bidirectionally predicted inter block, it can derive a pair of refinement parameters Vx and Vy for each 4x4 sub-block within the inter block. Then, the method can further derive the final prediction sample for each 4x4 sub-block using the corresponding refinement parameters.

[0029] A general description of the BDOF method can be as follows. It should be noted that several steps in the method include using the most significant bit (MSB) of the variable while discarding the least significant bit (LSB) of the variable, where the MSB is achieved by a right shift of the variable (denoted as >>). Similarly, a left shift is denoted as <<.

[0030] Step 1: For a bi-predicted inter CU, a first prediction block predSampleL0 is generated using a first motion vector mvL0, and a second prediction block predSampleL1 is generated using a second motion vector mvL1.

[0031] Step 2: For each prediction block (predSampleL0 and predSampleL1), generate two gradient blocks gradientH and gradientV. Each entry in gradientH is the MSB of the horizontal gradient of the corresponding prediction sample. Each entry in gradientV is the MSB of the vertical gradient of the corresponding prediction sample. The gradient is calculated by calculating the difference between two adjacent samples in the horizontal or vertical direction. The two gradient blocks generated from predSampleL0 are called gradientHL0 and gradientVL0. The two gradient blocks generated from predSampleL1 are called gradientHL1 and gradientVL1.

[0032] Step 3: Generate a prediction sample difference block diff using the two prediction blocks predSampleL0 and predSampleL1. For each entry in the diff block, it is the MSB of the corresponding prediction sample difference in predSampleL0 and predSampleL1.

[0033] Step 4: Generate two gradient accumulation blocks tempH and tempV. Each entry in block tempH is calculated as the sum of the corresponding entries in the two gradient blocks gradientHL0 and gradientHL1. Each entry in block tempV is calculated as the sum of the corresponding entries in the two gradient blocks gradientVL0 and gradientVL1.

[0034] Step 5: For the 4x4 sub-block SB within the inter CU, the MSB of tempH, the MSB of tempV, the MSB and LSB of diff are used to calculate the BDOF related parameters (sGx2, sGy2, sGxGy, sGxdI and sGydI).

[0035] Step 6: Calculate a pair of refinement parameters Vx and Vy using the BDOF related parameters (sGx2, sGy2, sGxGy, sGxdI and sGydI).

[0036] Step 7: Clip the refinement parameters Vx and Vy to a predetermined range of values.

[0037] Step 8: For each sample in the 4x4 sub-block SB, calculate the BDOF offset (bdofOffset) using the values ​​of Vx, Vy and the corresponding entries in gradientHL0, gradientHL1, gradientVL0 and gradientVL1.

[0038] Step 9: For each sample in the 4x4 sub-block, calculate its final predicted sample value using bdofOffset and the corresponding entries in predSampleL0 and gradientHL0.

[0039] In VVC Draft 6, BDOF follows the above process. The text of the BDOF process of VVC Draft 6 is as follows:

[0040] 8.5.6.5 Bidirectional Optical Flow Prediction Process

[0041] The inputs to this process are:

[0042] – Two variables nCbW and nCbH, specifying the width and height of the current coding block,

[0043] – Two (nCbW+2)x(nCbH+2) arrays of luma prediction samples predSamplesL0 and predSamplesL1,

[0044] – The prediction list uses the flags predFlagL0 and predFlagL1,

[0045] – reference indices refIdxL0 and refIdxL1,

[0046] – Bidirectional optical flow utilization flag bdofUtilizationFlag[xIdx][yIdx], where xIdx=0..(nCbW>>2)-1, yIdx=0..(nCbH>>2)-1.

[0047] The output of this process is an array pbSamples of luma prediction sample values ​​of (nCbW) x (nCbH).

[0048] The variables bitDepth, shift1, shift2, shift3, shift4, offset4, and mvRefineThres are derived as follows:

[0049] – Set the variable bitDepth equal to BitDepth Y .

[0050] – Set the variable shift1 equal to Max(6,bitDepth-6).

[0051] – Set the variable shift2 equal to Max(4,bitDepth-8).

[0052] – Set the variable shift3 equal to Max(1,bitDepth-11).

[0053] – Set the variable shift4 equal to Max(3,15-bitDepth) and the variable offset4 equal to 1<<(shift4-1).

[0054] – Set the variable mvRefineThres equal to 1< <Max(5,bitDepth-7)。

[0055] For xIdx = 0..(nCbW>>2)-1 and yIdx = 0..(nCbH>>2)–1, the following applies:

[0056] – Set the variable xSb to (xIdx<<2)+1 and the variable ySb to (yIdx<<2)+1.

[0057] – If bdofUtilizationFlag[xIdx][yIdx] is equal to FALSE, then for x=xSb-1..xSb+2, y=ySb-1..ySb+2, the predicted sample values ​​of the current sub-block are derived as follows:

[0058]

[0059] – Otherwise (bdofUtilizationFlag[xIdx][yIdx] is equal to TRUE), the predicted sample value of the current sub-block is derived as follows:

[0060] – For x = xSb-1..xSb+4, y = ySb-1..ySb+4, the following sequential steps apply:

[0061] 1. Predict the position (h) of each corresponding sample position (x, y) in the sample array x ,v y ) is exported as follows:

[0062] h x =Clip3(1,nCbW,x) (8-801)

[0063] v y =Clip3(1,nCbH,y) (8-802)

[0064] 2. The variables gradientHL0[x][y], gradientVL0[x][y], gradientHL1[x][y], and gradientVL1[x][y] are derived as follows:

[0065]

[0066]

[0067]

[0068]

[0069] 3. The variables diff[x][y], tempH[x][y], and tempV[x][y] are derived as follows:

[0070] diff[x][y]=(predSamplesL0[h x ][v y ]>>shift2)-(predSamplesL1[h x ][v y ]>>shift2) (8-807)

[0071] tempH[x][y]=(gradientHL0[x][y]+gradientHL1[x][y])>>shift3 (8-808)

[0072] tempV[x][y]=(gradientVL0[x][y]+gradientVL1[x][y])>>shift3 (8-809)

[0073] – The variables sGx2, sGy2, sGxGy, sGxdI and sGydI are derived as follows:

[0074] sGx2=Σ i Σ j Abs(tempH[xSb+i][ySb+j])

[0075] Where i, j = -1..4 (8-810)

[0076] sGy2=Σ i Σ j Abs(tempV[xSb+i][ySb+j])

[0077] Where i, j = -1..4 (8-811)

[0078] sGxGy=Σi Σ j (Sign(tempV[xSb+i][ySb+j])*tempH[xSb+i][ySb+j])

[0079] Where i, j = -1..4 (8-812)

[0080] SXWh m =sGxGy>>12 (8-813)

[0081] SXWh s =sGxGy&((1<<12)-1) (8-814)

[0082] sGxdI=Σ i Σ j (-Sign(tempH[xSb+i][ySb+j])*diff[xSb+i][ySb+j])

[0083] Where i, j = -1..4 (8-815)

[0084] sGydI=Σ i Σ j (-Sign(tempV[xSb+i][ySb+j])*diff[xSb+i][ySb+j])

[0085] Where i, j = -1..4 (8-816)

[0086] – The horizontal motion offset and vertical motion offset of the current sub-block are derived as:

[0087]

[0088]

[0089] – For x=xSb-1..xSb+2, y=ySb-1..ySb+2, the predicted sample values ​​of the current sub-block are derived as follows:

[0090]

[0091]

[0092] Referring to the above-mentioned VVC specification draft 6, the gradient calculation of the above-mentioned step 2 is specified as equations (8-803), (8-804), (8-805) and (8-806). The sample difference calculation in the above-mentioned step 3 is specified as equation (8-807).

[0093] The gradient accumulation calculation in step 4 above is specified as equations (8-808) and (8-809).

[0094] The BDOF related calculations in step 5 above are specified as equations (8-810), (8-811), (8-812), (8-813) and (8-814), where Abs() is the absolute value function and Sign() is the sign function.

[0095] The calculation of Vx and Vy in the above step 6 and the clipping operation in step 7 are specified as equations (8-817) and (8-818).

[0096] The calculation of bdofOffset in step 8 above is specified as equation (8-819).

[0097] The generation of the final prediction sample in step 9 above is specified as equation (8-820).

[0098] Although VVC specification draft 6 includes a BDOF process, the calculation of the BDOF offset includes a right shift. Possible disadvantages of this approach include: the shift is applied to each position [x][y] in a sub-block of size 4x4. That is, each sub-block requires 4x4x2=32 shifts to calculate the offset. In the worst possible case, if the CU is of size 128x128, it has 128x128 / (4x4)=1024 sub-blocks. Therefore, it requires 1024x32=32764 shifts, which is a large number of shift operations.

[0099] In various embodiments, a method may be provided for determining a BDOF offset that includes two shift operations on a pair of refinement parameters, only one shift operation per 4x4 sub-block.

[0100] The currently disclosed embodiments may provide potential advantages. One advantage over the existing method of VVC Draft Specification 6 is that two right shift operations for each position in a 4x4 sub-block may be avoided. Another advantage includes that, in various embodiments, a method for saving the number of shift operations may be provided. Various embodiments include two shifts of a pair of refinement parameters (e.g., refinement parameters Vx and Vy) for each sub-block. Therefore, for a CU of size 128x128, only 1024x2=2048 shifts are required, which is only 6.25% of the number of shifts in VVC Draft Specification 6.

[0101] Figure 44 is a block diagram illustrating an electronic device 410 having a picture processing circuit 400 configured according to some embodiments. The picture processing circuit 400 may be a picture decoder circuit and / or a picture encoder circuit. The electronic device 410 may include, but is not limited to, a wireless terminal, a wireless communication device, a wireless communication terminal, a UE node / terminal / device, a game console, a video media player, etc. The electronic device 410 may be configured to provide wired and / or wireless communication. When configured for wireless communication, the electronic device 410 includes an antenna and a transceiver circuit 424 (also referred to as a transceiver, including a transmitter and a receiver), and the transceiver circuit is configured to provide uplink and downlink radio communications with other electronic devices and / or radio access network RAN ​​nodes (e.g., base stations, eNBs, gNBs, network nodes, etc.) of a wireless communication network. Alternatively or additionally, the electronic device 410 may include a wired interface 424, which is configured to communicate with other electronic devices and / or network nodes via a wired line. The electronic device 410 also includes a processor circuit 420 (also referred to as a processor) coupled to the interface 424, and a memory circuit 422 (also referred to as a memory) coupled to the processor circuit 420. The processor 420 is also coupled to the picture processing circuit 400, which includes a processor 402 (e.g., a graphics processing unit) coupled to a memory 404, and the memory 404 can store a current picture in a series of pictures of a video sequence for encoding and / or decoding according to any one or more embodiments disclosed herein. The memory 422 stores computer-readable program code, which, when executed by the processor 420, causes the processor 420 to perform operations according to any one or more embodiments disclosed herein. According to other embodiments, the picture processing circuit 400 may be implemented as an application-specific integrated circuit or other picture decoding circuit, and its operations may be performed at least partially or entirely by the processor 420 that performs operations from the memory 422. The electronic device 410 may receive a series of pictures of a video sequence for decoding from another UE, a network node, a removable storage device, a camera, etc. The electronic device 410 may encode a series of pictures of a video sequence that is provided to another UE, a network node, a removable storage device, or the like.

[0102] Figure 5 An objective performance table of a third exemplary embodiment according to the inventive concept is shown, which will be further described below, although other performances may be obtained without limitation to these embodiments. Figure 5 , tested on the VVC test model VTM-6.0. Figure 5The numbers in the table show the relative bit cost of achieving the same video quality as a reference codec for the tested devices that can encode and / or decode (also referred to as codecs). The reference codec used here is VTM-6.0 as is, and the tested codec is VTM-6.0 modified according to the third exemplary embodiment. Figure 5 In the example above, 0% means that there is no bit cost difference between the tested codec and the reference codec in achieving the same video quality. Based on these numbers, it can be asserted that the simplification has a negligible impact on objective performance.

[0103] Exemplary embodiments are now described.

[0104] The processing of a picture processing circuit (e.g., Figure 4 The picture processing circuit 400) performs encoding and decoding operations. The modules (also referred to as units) can be stored in Figure 4 The modules may be stored in the memory 404, and these modules may provide instructions, so that when the instructions of the modules are executed by the processor 402, the processor 402 performs corresponding operations according to any one or more embodiments disclosed herein.

[0105] In a first exemplary embodiment, a shift operation is applied to the refinement parameters Vx and Vy before using them to calculate the BDOF offset (bdofOffset) for each sample in a 4x4 sub-block. In other words, the shifted Vx and Vy are used to calculate bdofOffset instead. As described below, the shift operation in the calculation of bdofOffset for each sample in a 4x4 sub-block in the VVC specification draft 6 is removed.

[0106] The operation includes step 1: for a bi-predicted inter CU, a first prediction block predSampleL0 is generated using a first motion vector mvL0, and a second prediction block predSampleL1 is generated using a second motion vector mvL1.

[0107] The operation may also include step 2: for each prediction block (predSampleL0 and predSampleL1), generate two gradient blocks gradientH and gradientV. Each entry in gradientH is the MSB of the horizontal gradient of the corresponding prediction sample. Each entry in gradientV is the MSB of the vertical gradient of the corresponding prediction sample. The gradient is calculated by calculating the difference between two adjacent samples in the horizontal direction or the vertical direction. The two gradient blocks generated from predSampleL0 are called gradientHL0 and gradientVL0. The two gradient blocks generated from predSampleL1 are called gradientHL1 and gradientVL1.

[0108] The operation may further include step 3: generating a prediction sample difference block diff using the two prediction blocks predSampleL0 and predSampleL1. For each entry in the diff block, it is the MSB of the corresponding prediction sample difference in predSampleL0 and predSampleL1.

[0109] The operation may further include step 4: generating two gradient accumulation blocks tempH and tempV. Each entry in block tempH is calculated as the sum of corresponding entries in two gradient blocks gradientHL0 and gradientHL1. Each entry in block tempV is calculated as the sum of corresponding entries in two gradient blocks gradientVL0 and gradientVL1.

[0110] The operation may further include step 5: for a 4x4 sub-block SB within the inter CU, using the MSB of tempH, the MSB of tempV, and both the MSB and LSB of diff to calculate BDOF related parameters (sGx2, sGy2, sGxGy, sGxdI, and sGydI).

[0111] The operation may further include step 6: calculating a pair of refinement parameters Vx and Vy using the BDOF related parameters (sGx2, sGy2, sGxGy, sGxdI, and sGydI).

[0112] The operation may further include step 7: clipping the refinement parameters Vx and Vy to a predetermined range of values.

[0113] The operation may further include step 8: performing a shift operation on the refinement parameters Vx and Vy.

[0114] The operation may further include step 9: for each sample in the 4x4 sub-block SB, calculate a BDOF offset (bdofOffset) using the shifted versions of Vx, Vy and corresponding entries in gradientHL0, gradientHL1, gradientVL0 and gradientVL1.

[0115] The operation may further include step 10: for each sample in the 4x4 sub-block, using bdofOffset and corresponding entries in predSampleL0 and gradientHL0, calculating its final predicted sample value.

[0116] In the second exemplary embodiment, the operation of the first exemplary embodiment is performed by directly shifting Vx and Vy right by 1 bit to implement the shift operation of step 8. That is,

[0117] v x =v x >>1,

[0118] v y =v y >>1.

[0119] A second exemplary embodiment is shown below (where changes to the BDOF method of the above-mentioned VVC specification draft 6 are shown with underlining and strikethrough):

[0120] 8.5.6.5 Bidirectional Optical Flow Prediction Process

[0121] The inputs to this process are:

[0122] – Two variables nCbW and nCbH, specifying the width and height of the current coding block,

[0123] – Two (nCbW+2)x(nCbH+2) arrays of luma prediction samples predSamplesL0 and predSamplesL1,

[0124] – The prediction list uses the flags predFlagL0 and predFlagL1,

[0125] – reference indices refIdxL0 and refIdxL1,

[0126] – Bidirectional optical flow utilization flag bdofUtilizationFlag[xIdx][yIdx], where xIdx=0..(nCbW>>2)-1, yIdx=0..(nCbH>>2)-1.

[0127] The output of this process is an array pbSamples of luma prediction sample values ​​of (nCbW) x (nCbH).

[0128] The variables bitDepth, shift1, shift2, shift3, shift4, offset4, and mvRefineThres are derived as follows:

[0129] – Set the variable bitDepth equal to BitDepth Y .

[0130] – Set the variable shift1 equal to Max(6,bitDepth-6).

[0131] – Set the variable shift2 equal to Max(4,bitDepth-8).

[0132] – Set the variable shift3 equal to Max(1,bitDepth-11).

[0133] – Set the variable shift4 equal to Max(3,15-bitDepth) and the variable offset4 equal to 1<<(shift4-1).

[0134] – Set the variable mvRefineThres equal to 1< <Max(5,bitDepth-7)。

[0135] For xIdx = 0..(nCbW>>2)-1 and yIdx = 0..(nCbH>>2)–1, the following applies:

[0136] – Set the variable xSb to (xIdx<<2)+1 and the variable ySb to (yIdx<<2)+1.

[0137] – If bdofUtilizationFlag[xIdx][yIdx] is equal to FALSE, then for x=xSb-1..xSb+2, y=ySb-1..ySb+2, the predicted sample values ​​of the current sub-block are derived as follows:

[0138]

[0139] – Otherwise (bdofUtilizationFlag[xIdx][yIdx] is equal to TRUE), the predicted sample value of the current sub-block is derived as follows:

[0140] – For x = xSb-1..xSb+4, y = ySb-1..ySb+4, the following sequential steps apply:

[0141] 4. Predict the position (h) of each corresponding sample position (x, y) in the sample array x ,v y ) is exported as follows:

[0142] h x =Clip3(1,nCbW,x) (8-801)

[0143] v y =Clip3(1,nCbH,y) (8-802)

[0144] 5. The variables gradientHL0[x][y], gradientVL0[x][y], gradientHL1[x][y], and gradientVL1[x][y] are derived as follows:

[0145]

[0146]

[0147]

[0148]

[0149]

[0150] 6. The variables diff[x][y], tempH[x][y], and tempV[x][y] are derived as follows:

[0151] diff[x][y]=(predSamplesL0[h x ][v y ]>>shift2)-(predSamplesL1[h x ][v y ]>>shift2) (8-807)

[0152] tempH[x][y]=(gradientHL0[x][y]+gradientHL1[x][y])>>shift3 (8-808)

[0153] tempV[x][y]=(gradientVL0[x][y]+gradientVL1[x][y])>>shift3 (8-809)

[0154] – The variables sGx2, sGy2, sGxGy, sGxdI and sGydI are derived as follows:

[0155] sGx2=Σi Σ j Abs(tempH[xSb+i][ySb+j])

[0156] Where i, j = -1..4 (8-810)

[0157] sGy2=Σ i Σ j Abs(tempV[xSb+i][ySb+j])

[0158] Where i, j = -1..4 (8-811)

[0159] sGxGy=Σ i Σ j (Sign(tempV[xSb+i][ySb+j])*tempH[xSb+i][ySb+j])

[0160] Where i, j = -1..4 (8-812)

[0161] SXWh m =sGxGy>>12(8-813)

[0162] SXWh s =sGxGy&((1<<12)-1)(8-814)

[0163] sGxdI=Σ i Σ j (-Sign(tempH[xSb+i][ySb+j])*diff[xSb+i][ySb+j])

[0164] Where i, j = -1..4 (8-815)

[0165] sGydI=Σ i Σ j (-Sign(tempV[xSb+i][ySb+j])*diff[xSb+i][ySb+j])

[0166] Where i, j = -1..4 (8-816)

[0167] – The horizontal motion offset and vertical motion offset of the current sub-block are derived as:

[0168]

[0169]

[0170] – v x Further modified to v x =v x >>1, v y Further modified to v y =v y >>1

[0171] – For x=xSb-1..xSb+2, y=ySb-1..ySb+2, the predicted sample values ​​of the current sub-block are derived as follows:

[0172]

[0173]

[0174] As shown in the second exemplary embodiment described above, a direct 1-bit right shift of Vx and Vy is performed to obtain the shifted Vx and Vy refinement parameters. The two right shift operations for each position in the 4x4 sub-block when determining bdofOffset are removed.

[0175] In the third exemplary embodiment, the operation of the first exemplary embodiment is performed by implementing the shift operation of step 8 by adding an offset of 1 before the right shift. That is,

[0176] v x =(v x +1)>>1,

[0177] v y =(v y +1)>>1.

[0178] A third exemplary embodiment is shown below (where changes to the BDOF method of the above-mentioned VVC specification draft 6 are shown with underlining and strikethrough):

[0179] 8.5.6.5 Bidirectional Optical Flow Prediction Process

[0180] The inputs to this process are:

[0181] – Two variables nCbW and nCbH, specifying the width and height of the current coding block,

[0182] – Two (nCbW+2)x(nCbH+2) arrays of luma prediction samples predSamplesL0 and predSamplesL1,

[0183] – The prediction list uses the flags predFlagL0 and predFlagL1,

[0184] – reference indices refIdxL0 and refIdxL1,

[0185] – Bidirectional optical flow utilization flag bdofUtilizationFlag[xIdx][yIdx], where xIdx=0..(nCbW>>2)-1, yIdx=0..(nCbH>>2)-1.

[0186] The output of this process is an array pbSamples of luma prediction sample values ​​of (nCbW) x (nCbH).

[0187] …

[0188] – The horizontal motion offset and vertical motion offset of the current sub-block are derived as:

[0189]

[0190]

[0191] – v x Further modified to v x =(v x +1)>>1,v y Further modified to v y =(v y +1)>>1

[0192] – For x=xSb-1..xSb+2, y=ySb-1..ySb+2, the predicted sample values ​​of the current sub-block are derived as follows:

[0193]

[0194]

[0195]

[0196] As shown in the third exemplary embodiment described above, a shift operation of adding an offset of 1 before right shifting is performed to obtain shifted Vx and Vy refinement parameters. The two right shift operations for each position in the 4x4 sub-block when determining bdofOffset are removed.

[0197] Now will be Figure 6 These and other related operations are described in the context of operational flow diagrams of the invention, which may be performed by a video encoder and / or a video decoder.

[0198] Figure 6 Flowchart of operations that may be performed by picture decoding circuitry and / or picture encoding circuitry for determining a BDOF offset according to some embodiments, comprising two shift operations on a pair of refinement parameters, only one shift operation for each 4x4 sub-block, for performing prediction of luma sample values ​​during encoding and / or decoding of a video. Figure 6 The operation of may more generally be performed by a picture decoder circuit and / or a picture decoder circuit (e.g., Figure 4 The image processing circuit 400 in the image processing circuit is executed.

[0199] Reference Figure 6The operation includes: for each bidirectionally predicted inter-frame coding block, determining (600) a first prediction block using a first motion vector, and determining (600) a second prediction block using a second motion vector. The operation may also include: for each prediction block, determining (602) a first horizontal gradient block and a first vertical gradient block of the first prediction block and a second horizontal gradient block and a second vertical gradient block of the second prediction block.

[0200] The operations may also include determining (604) a block of prediction sample differences between the first prediction block and the second prediction block.

[0201] The operation may also include determining (606) each of a horizontal gradient accumulation block (temppH) and a vertical gradient accumulation block (tempV), wherein the horizontal gradient accumulation block (temppH) is determined by accumulating the first horizontal gradient block and the second horizontal gradient block, and the vertical gradient accumulation block (tempV) is determined by accumulating the first vertical gradient block and the second vertical gradient block.

[0202] Still refer to Figure 6 The operation may also include: for each sub-block within each bidirectionally predicted inter-frame coded block, using the most significant bit of the horizontal gradient accumulation block (tempH), the most significant bit of the vertical gradient accumulation block (tempV), and the most significant bit and the least significant bit of the prediction sample difference block to determine (608) a set of bidirectional optical flow related parameters.

[0203] The operations may also include determining (610) a first refinement parameter and a second refinement parameter using the set of bidirectional optical flow related parameters.

[0204] The operations may also include clipping (612) the value of each of the first refinement parameter and the second refinement parameter to within a defined range of values.

[0205] The operations may also include performing (614) a shift operation on each of the first refinement parameter and the second refinement parameter to obtain a shifted first refinement parameter and a shifted second refinement parameter.

[0206] The operations may also include determining (616) a bidirectional optical flow offset for each sample in each subblock within each bidirectionally predicted inter-frame coded block using the shifted first refinement parameter, the shifted second refinement parameter, the first horizontal gradient block and the first vertical gradient block of the first prediction block, and the second horizontal gradient block and the second vertical gradient block of the second prediction block.

[0207] In some embodiments, performing (614) a shift operation on each of the first refinement parameter and the second refinement parameter to obtain a shifted first refinement parameter and a shifted second refinement parameter is performed by directly right shifting each of the first refinement parameter and the second refinement parameter by one bit.

[0208] In some embodiments, the first refinement parameter is a variable denoted by vx, the second refinement parameter is a variable denoted by vy, and wherein directly right-shifting each of the first refinement parameter (vx) and the second refinement parameter (vy) by one bit comprises:

[0209] v x =v x >>1; and

[0210] v y =v y >>1

[0211] The >> indicates the right shift of the variable.

[0212] In some embodiments, determining (616) a bidirectional optical flow offset using the shifted first refinement parameter, the shifted second refinement parameter, the first horizontal gradient block and the first vertical gradient block of the first prediction block, and the second horizontal gradient block and the second vertical gradient block of the second prediction block includes:

[0213] bdofOffset=(v x *(gradientHL0[x+1][y+1]–gradientHL1[x+1][y+1]))+(v y *(gradientVL0[x+1][y+1]–gradientVL1[x+1][y+1]))

[0214] Where bdofOffset is the bidirectional optical flow offset, v x is the first refinement parameter after direct right shift, v y is the second refinement parameter that is directly right-shifted, gradientHL0[x+1][y+1] is the first horizontal gradient block, gradientHL1[x+1][y+1] is the second horizontal gradient block, gradientVL0[x+1][y+1] is the first vertical gradient block, and gradientVL1[x+1][y+1] is the second vertical gradient block.

[0215] In some embodiments, performing (614) a shift operation on each of the first refinement parameter and the second refinement parameter to obtain a shifted first refinement parameter and a shifted second refinement parameter is performed by adding an offset to each of the first refinement parameter and the second refinement parameter prior to right shifting.

[0216] In some embodiments, the first refinement parameter is a variable denoted by vx, the second refinement parameter is a variable denoted by vy, and wherein adding an offset to each of the first refinement parameter and the second refinement parameter before right shifting comprises:

[0217] v x =(v x +1)>>1; and

[0218] v y =(v y +1)>>1

[0219] The >> indicates the right shift of the variable.

[0220] In some embodiments, determining (616) a bidirectional optical flow offset using the shifted first refinement parameter, the shifted second refinement parameter, the first horizontal gradient block and the first vertical gradient block of the first prediction block, and the second horizontal gradient block and the second vertical gradient block of the second prediction block includes:

[0221] bdofOffset=(v x *(gradientHL0[x+1][y+1]–gradientHL1[x+1][y+1]))+(v y *(gradientVL0[x+1][y+1]–gradientVL1[x+1][y+1]))

[0222] Where bdofOffset is the bidirectional optical flow offset, v x is the first refinement parameter with added offset and right shift, v y are the second refinement parameters with added offset and right shift, gradientHL0[x+1][y+1] is the first horizontal gradient block, gradientHL1[x+1][y+1] is the second horizontal gradient block, gradientVL0[x+1][y+1] is the first vertical gradient block, and gradientVL1[x+1][y+1] is the second vertical gradient block.

[0223] Still refer to Figure 6 The operation may also include: performing (618) prediction of luma sample values ​​in each bidirectionally predicted inter-frame coded block using predicted sample values ​​of each sample in each sub-block within each bidirectionally predicted inter-frame coded block using the bidirectional optical flow offset, the first prediction block, and the first horizontal gradient block.

[0224] Therefore, go to Figure 7, the picture processing circuit performing the operation obtains a shifted pair of refinement parameters for each sub-block of the bidirectionally predicted inter-frame coding block to obtain a shifted pair of refinement parameters in block 700, wherein the shifted pair of refinement parameters includes a shifted first refinement parameter and a shifted second refinement parameter. In block 702, the picture processing circuit determines a bidirectional optical flow BDOF offset by the following operation without performing any right shift using at least the shifted pair of refinement parameters: for each sample in each sub-block within each bidirectionally predicted inter-frame coding block, determining the BDOF offset using the shifted first refinement parameter, the shifted second refinement parameter, the first horizontal gradient block and the first vertical gradient block of the first prediction block, and the second horizontal gradient block and the second vertical gradient block of the second prediction block.

[0225] Various aspects of the present disclosure have been described herein with reference to flowchart illustrations and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present disclosure. It should be understood that each box in the flowchart illustration and / or block diagram, and the combination of multiple boxes in the flowchart illustration and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a computer, a special-purpose computer, or other programmable data processing devices used to produce a machine, so that the instruction (executed via a processor of a computer or other programmable instruction execution device) creates a mechanism for implementing the function / action specified in one or more boxes of a flowchart and / or block diagram.

[0226] These computer program instructions may also be stored in a computer-readable medium, which, when executed, may direct a computer, other programmable data processing device, or other device to operate in a specific manner, so that the instructions, when stored in the computer-readable medium, produce an article of manufacture, which includes instructions that, when executed, cause the computer to implement the functions / actions specified in one or more boxes of the flowchart and / or block diagram. The computer program instructions may also be loaded onto a computer, other programmable instruction execution device, or other device, so that a series of operating steps are executed on the computer, other programmable device, or other device, thereby producing a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the functions / actions specified in one or more boxes of the flowchart and / or block diagram.

[0227] It should be understood that the terms used herein are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meanings as those commonly understood by those of ordinary skill in the art to which the present disclosure belongs. It will be further understood that terms such as those defined in general dictionaries should be interpreted as having meanings consistent with their meanings in the context of this specification and the relevant technology, and are not interpreted as ideal or overly formal meanings, unless such clear definitions are provided herein.

[0228] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architectures, functions and operations of the systems, methods and computer program products according to various aspects of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a code module, segment or portion, which includes one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative embodiments, the functions marked in the box may not occur in the order marked in the accompanying drawings. For example, the two boxes shown in succession may actually be substantially concurrently executed, or the boxes may be executed in reverse order at some point, depending on the functions involved. It should also be noted that each box of the block diagram and / or flow diagram and the combination of multiple boxes in the block diagram and / or flow diagram can be performed by a combination of a hardware system based on a specific purpose or a specific purpose hardware and computer instructions, and the system is used to perform a specific function or action.

[0229] The terms used herein are only for the purpose of describing specific aspects, and are not intended to limit the present disclosure. As used herein, the singular forms "a", "an" and "said" are intended to also include plural forms, unless the context clearly indicates the opposite. It will also be understood that the terms "include" and / or "comprise" when used in this specification specify the presence of the claimed features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof. The term "and / or" used herein includes any and all combinations of one or more of the associated listed items. Throughout the description of the drawings, similar reference numerals refer to similar elements.

[0230] The description of the present disclosure has been presented for the purpose of illustration and description, but is not intended to be exhaustive or limited to the disclosure in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the present disclosure. Various aspects of the embodiments are selected and described in order to best explain the principles and practical applications of the present disclosure, and to enable those of ordinary skill in the art to understand the present disclosure with various modifications suitable for the specific use contemplated.

[0231] Example embodiments are discussed below. Reference numerals / letters are provided in parentheses by way of example / illustration and not to limit the embodiments to the specific elements indicated by the reference numerals / letters.

[0232] Examples listed:

[0233] Embodiment 1. A method of processing a video sequence comprising a plurality of images, wherein each of the plurality of images comprises a plurality of blocks having inter-frame coded blocks with bidirectional prediction, the method comprising:

[0234] For each bi-directionally predicted inter-coded block, a first prediction block is determined (600) using a first motion vector, and a second prediction block is determined (600) using a second motion vector;

[0235] For each prediction block, determining (602) a first horizontal gradient block and a first vertical gradient block of a first prediction block, and a second horizontal gradient block and a second vertical gradient block of a second prediction block;

[0236] determining (604) a block of prediction sample differences between the first prediction block and the second prediction block;

[0237] determining (606) each of a horizontal gradient accumulation block and a vertical gradient accumulation block, wherein the horizontal gradient accumulation block is determined by accumulating the first horizontal gradient block and the second horizontal gradient block, and the vertical gradient accumulation block is determined by accumulating the first vertical gradient block and the second vertical gradient block;

[0238] For each subblock within each bidirectionally predicted inter-frame coded block, determine (608) a set of bidirectional optical flow related parameters using the most significant bit of the horizontal gradient accumulation block, the most significant bit of the vertical gradient accumulation block, and the most significant bit and the least significant bit of the prediction sample difference block;

[0239] determining (610) a first refinement parameter and a second refinement parameter using the set of bidirectional optical flow related parameters;

[0240] clipping (612) the value of each of the first refinement parameter and the second refinement parameter to a defined range of values;

[0241] performing (614) a shift operation on each of the first refinement parameter and the second refinement parameter to obtain a shifted first refinement parameter and a shifted second refinement parameter;

[0242] determining (616) a bidirectional optical flow offset for each sample in each subblock within each bidirectionally predicted inter-coded block using the shifted first refinement parameter, the shifted second refinement parameter, the first horizontal gradient block and the first vertical gradient block of the first prediction block, and the second horizontal gradient block and the second vertical gradient block of the second prediction block; and

[0243] Prediction of luma sample values ​​in each bidirectionally predicted inter-coded block is performed (618) using the predicted sample values ​​for each sample in each sub-block within each bidirectionally predicted inter-coded block using the bidirectional optical flow offsets, the first prediction block, and the first horizontal gradient block.

[0244] Embodiment 2. The method of embodiment 1, wherein performing (614) a shift operation on each of the first refinement parameter and the second refinement parameter to obtain a shifted first refinement parameter and a shifted second refinement parameter is performed by directly right shifting each of the first refinement parameter and the second refinement parameter by one bit.

[0245] Embodiment 3. The method according to any one of embodiments 1 to 2, wherein the first refinement parameter is v x The second refinement parameter is represented by v y denoted by the variable, and wherein the first refinement parameter (v x ) and the second refinement parameter (v y ) each of which is directly shifted right by one bit including:

[0246] v x =v x >>1; and

[0247] v y =v y >>1

[0248] The >> indicates the right shift of the variable.

[0249] Embodiment 4. The method of any one of embodiments 1 to 3, wherein determining (616) a bidirectional optical flow offset using the shifted first refinement parameter, the shifted second refinement parameter, the first horizontal gradient block and the first vertical gradient block of the first prediction block, and the second horizontal gradient block and the second vertical gradient block of the second prediction block comprises:

[0250] bdofOffset=(v x *(gradientHL0[x+1][y+1]–gradientHL1[x+1][y+1]))+(v y *(gradientVL0[x+1][y+1]–gradientVL1[x+1][y+1]))

[0251] Where bdofOffset is the bidirectional optical flow offset, v x is the first refinement parameter after direct right shift, v yis the second refinement parameter that is directly right-shifted, gradientHL0[x+1][y+1] is the first horizontal gradient block, gradientHL1[x+1][y+1] is the second horizontal gradient block, gradientVL0[x+1][y+1] is the first vertical gradient block, and gradientVL1[x+1][y+1] is the second vertical gradient block.

[0252] 5. The method of embodiment 1, wherein performing (614) a shift operation on each of the first refinement parameter and the second refinement parameter to obtain a shifted first refinement parameter and a shifted second refinement parameter is performed by adding an offset to each of the first refinement parameter and the second refinement parameter before right shifting.

[0253] Embodiment 6. The method according to embodiment 5, wherein the first refinement parameter is v x The second refinement parameter is represented by v y A variable represented by , and wherein adding an offset to each of the first refinement parameter and the second refinement parameter before right shifting comprises:

[0254] v x =(v x +1)>>1; and

[0255] v y =(v y +1)>>1

[0256] The >> indicates the right shift of the variable.

[0257] Embodiment 7. The method of any one of embodiments 5 to 6, wherein determining (616) a bidirectional optical flow offset using the shifted first refinement parameter, the shifted second refinement parameter, the first horizontal gradient block and the first vertical gradient block of the first prediction block, and the second horizontal gradient block and the second vertical gradient block of the second prediction block comprises:

[0258] bdofOffset=(v x *(gradientHL0[x+1][y+1]–gradientHL1[x+1][y+1]))+(v y *(gradientVL0[x+1][y+1]–gradientVL1[x+1][y+1]))

[0259] Where bdofOffset is the bidirectional optical flow offset, v x is the first refinement parameter with added offset and right shift, v yare the second refinement parameters with added offset and right shift, gradientHL0[x+1][y+1] is the first horizontal gradient block, gradientHL1[x+1][y+1] is the second horizontal gradient block, gradientVL0[x+1][y+1] is the first vertical gradient block, and gradientVL1[x+1][y+1] is the second vertical gradient block.

[0260] Embodiment 8. An electronic device (410) adapted to perform the operations according to any one of embodiments 1 to 7.

[0261] Embodiment 9. An electronic device (410) comprises:

[0262] The processor (402) is configured to perform the operation according to any one of embodiments 1 to 7.

[0263] Embodiment 10. An electronic device (410) is configured to perform operations comprising the following steps:

[0264] For each bidirectionally predicted inter-frame coding block, a first prediction block is determined using the first motion vector, and a second prediction block is determined using the second motion vector;

[0265] For each prediction block, determining a first horizontal gradient block and a first vertical gradient block of a first prediction block and a second horizontal gradient block and a second vertical gradient block of a second prediction block;

[0266] determining a prediction sample difference block between the first prediction block and the second prediction block;

[0267] Determine each of a horizontal gradient accumulation block and a vertical gradient accumulation block, wherein the horizontal gradient accumulation block is determined by accumulating the first horizontal gradient block and the second horizontal gradient block, and the vertical gradient accumulation block is determined by accumulating the first vertical gradient block and the second vertical gradient block;

[0268] For each sub-block in each bidirectionally predicted inter-frame coding block, a set of bidirectional optical flow related parameters is determined using the most significant bit of the horizontal gradient accumulation block, the most significant bit of the vertical gradient accumulation block, and the most significant bit and the least significant bit of the prediction sample difference block;

[0269] Determining a first refinement parameter and a second refinement parameter using a set of bidirectional optical flow related parameters;

[0270] clipping the value of each of the first refinement parameter and the second refinement parameter to a defined range of values;

[0271] performing a shift operation on each of the first refinement parameter and the second refinement parameter to obtain a shifted first refinement parameter and a shifted second refinement parameter;

[0272] determining, for each sample in each subblock within each bidirectionally predicted inter-coded block, a bidirectional optical flow offset using the shifted first refinement parameter, the shifted second refinement parameter, the first horizontal gradient block and the first vertical gradient block of the first prediction block, and the second horizontal gradient block and the second vertical gradient block of the second prediction block; and

[0273] Prediction of luma sample values ​​in each bidirectionally predicted inter-coded block is performed using predicted sample values ​​for each sample in each sub-block within each bidirectionally predicted inter-coded block using the bidirectional optical flow offset, the first prediction block and the first horizontal gradient block.

[0274] Further definitions are discussed below.

[0275] In the above description of various embodiments of the inventive concept, it is to be understood that the terms used herein are only used for the purpose of describing specific embodiments and are not intended to limit the inventive concept. Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meanings as commonly understood by ordinary technicians in the field to which the inventive concept belongs. It should also be understood that terms such as those defined in general dictionaries should be interpreted as having meanings consistent with their meanings in the context of this specification and the relevant technology, and not interpreted as ideal or overly superficial meanings, unless so clearly defined herein.

[0276] When an element is referred to as being "connected", "coupled", "responsive" or a variation thereof relative to another element, it may be directly connected, coupled to or responsive to the other element, or there may be an intermediate element. On the contrary, when an element is referred to as being "directly connected", "directly coupled", "directly responsive" or a variation thereof relative to another element, there is no intermediate element. Throughout the text, similar reference numerals represent similar elements. In addition, "coupling", "connection", "responsiveness" or a variation thereof used herein may include wireless coupling, connection or response. As used herein, the singular forms "one", "an" and "said" are intended to also include plural forms, unless the context clearly indicates otherwise. For the sake of brevity and / or clarity, well-known functions or structures may not be described in detail. The term "and / or" includes any and all combinations of one or more items listed in association.

[0277] It will be understood that although the terms first, second, third, etc. can be used herein to describe each element / operation, these elements / operations should not be limited by these terms. These terms are only used to distinguish one element / operation from another element / operation. Therefore, the first element / operation in some embodiments can be referred to as the second element / operation in other embodiments without departing from the teaching of the inventive concept. Throughout the specification, the same reference numerals or the same reference signs represent the same or similar elements.

[0278] As used herein, the terms "comprise, comprising, comprises, include, including, includes," "have, has, having," or variations thereof, are open ended and include one or more stated features, integers, elements, steps, components, or functions, but do not preclude the presence or addition of one or more other features, integers, elements, steps, components, functions, or combinations thereof. In addition, as used herein, the commonly used abbreviation "eg," derived from the Latin phrase "exempli gratia," may be used to introduce or specify one or more general examples of a previously mentioned item without being intended as a limitation of that item. The commonly used abbreviation "ie," derived from the Latin phrase "idest," may be used to specify a specific item of a more general reference.

[0279] Example embodiments are described herein with reference to block diagrams and / or flowchart illustrations of computer-implemented methods, devices (systems and / or equipment), and / or computer program products. It should be understood that the blocks of the block diagrams and / or flowchart illustrations and the combination of multiple blocks in the block diagrams and / or flowchart illustrations can be implemented by computer program instructions executed by one or more computer circuits. These computer program instructions can be provided to processor circuits of general-purpose computer circuits, special-purpose computer circuits, and / or other programmable data processing circuits to generate a machine, so that instructions executed by processors of computers and / or other programmable data processing devices convert and control transistors, values ​​stored in memory locations, and other hardware components within such circuits to implement the functions / actions specified in the block diagrams and / or flowcharts, and thereby create a device (functional body) and / or structure for implementing the functions / actions specified in the block diagrams and / or flowcharts.

[0280] These computer program instructions may also be stored in a tangible computer-readable medium that is capable of directing a computer or other programmable data processing device to act in a specific manner so that the instructions stored in the computer-readable medium produce an article of manufacture, the article of manufacture including instructions for implementing the functions / actions specified in the blocks of the block diagrams and / or flow charts. Thus, embodiments of the inventive concept may be implemented in hardware and / or in software (including firmware, stored software, microcode, etc.) running on a processor such as a digital signal processor, which may be collectively referred to as a "circuit", "module" or variations thereof.

[0281] It should also be noted that in some alternative implementations, the function / action marked in the frame may not occur in the order marked in the flow chart. For example, depending on the function / action involved, the two frames shown in succession can actually be performed substantially simultaneously, or the frame can sometimes be performed in the opposite order. In addition, the function of a given frame of a flow chart and / or block diagram can be divided into multiple frames and / or the function of two or more frames of a flow chart and / or block diagram can be integrated at least in part. Finally, without departing from the scope of the inventive concept, other frames can be added / inserted between the frames shown, and / or the frame / operation can be omitted. In addition, although some frames include arrows about the communication path for indicating the main direction of communication, it should be understood that communication can occur in the direction opposite to the arrow represented.

[0282] Under the premise of not departing from the principle of the inventive concept basically, many changes and modifications can be made to the embodiment. All these changes and modifications are intended to be included in the scope of the inventive concept herein. Therefore, the above-mentioned subject matter should be understood as exemplary and not restrictive, and the examples of the embodiments are intended to cover all these modifications, improvements and other embodiments that fall within the spirit and scope of the inventive concept. Therefore, to the maximum extent permitted by law, the scope of the inventive concept should be determined by the widest permissible interpretation of the present disclosure including the examples of the embodiments and their equivalents, and should not be limited to or restricted to the previous specific embodiments.

[0283] Where appropriate, any feature of any embodiment disclosed herein may be applied to any other embodiment. Likewise, any advantage of any embodiment may be applicable to any other embodiment, and vice versa. Other purposes, features and advantages of the attached embodiments will be apparent from the following description.

[0284] Any suitable steps, methods, features, functions or benefits disclosed herein may be performed by one or more functional units or modules of one or more virtual devices. Each virtual device may include a plurality of these functional units. These functional units may be implemented by a processing circuit, which may include one or more microprocessors or microcontrollers and other digital hardware (may include a digital signal processor (DSP), dedicated digital logic, etc.). The processing circuit may be configured to execute program codes stored in a memory, which may include one or more types of memory, such as a read-only memory (ROM), a random access memory (RAM), a cache memory, a flash memory device, an optical storage device, etc. The program code stored in the memory includes program instructions for executing one or more telecommunications and / or data communication protocols and instructions for executing one or more techniques described herein. In some implementations, the processing circuit may be used to cause the corresponding functional unit to perform a corresponding function according to one or more embodiments of the present disclosure.

[0285] The units may have conventional meanings in the field of electronics, electrical devices and / or electronic devices, and may include those described herein, such as electrical and / or electronic circuits, devices, modules, processors, memories, logical solid-state and / or discrete devices, computer programs or instructions for performing respective tasks, procedures, calculations, output and / or display functions, etc.

Claims

1. A method for performing bidirectional optical flow (BDOF) processing on a video sequence comprising a plurality of images, wherein each image of the plurality of images comprises a plurality of blocks having bidirectionally predicted inter-frame coded blocks, the method comprising: For each subblock of a bi-directionally predicted inter-coded block, obtaining (700) a shifted pair of refinement parameters, wherein the shifted pair of refinement parameters comprises a shifted first refinement parameter and a shifted second refinement parameter; and Using at least the shifted pair of refinement parameters, a BDOF offset is determined (702) by the following operation without performing any right shift when determining (702) the BDOF offset: for each sample in each subblock within each bidirectionally predicted inter-frame coded block, the BDOF offset is determined using the shifted first refinement parameter, the shifted second refinement parameter, a first horizontal gradient block and a first vertical gradient block of a first prediction block, and a second horizontal gradient block and a second vertical gradient block of a second prediction block.

2. The method according to claim 1, further comprising: For each bi-directionally predicted inter-coded block, determining (600) a first prediction block using a first motion vector, and determining (600) a second prediction block using a second motion vector; For each prediction block, determining (602) a first horizontal gradient block and a first vertical gradient block of the first prediction block and a second horizontal gradient block and a second vertical gradient block of the second prediction block; determining (604) a block of prediction sample differences between the first prediction block and the second prediction block; determining (606) each of a horizontal gradient accumulation block and a vertical gradient accumulation block, wherein the horizontal gradient accumulation block is determined by accumulating the first horizontal gradient block and the second horizontal gradient block, and the vertical gradient accumulation block is determined by accumulating the first vertical gradient block and the second vertical gradient block; For each subblock within each bidirectionally predicted inter-frame coded block, determine (608) a set of BDOF-related parameters using the most significant bit of the horizontal gradient accumulation block, the most significant bit of the vertical gradient accumulation block, and the most significant bit and the least significant bit of the predicted sample difference block; determining ( 610 ) the first refinement parameter and the second refinement parameter using the set of BDOF-related parameters; clipping (612) the value of each of the first refinement parameter and the second refinement parameter to within a defined range of values; and Prediction of luma sample values ​​in each bi-directionally predicted inter-coded block is performed (614) using predicted sample values ​​for each sample in each sub-block within each bi-directionally predicted inter-coded block using the BDOF offset, the first prediction block, and the first horizontal gradient block.

3. The method according to claim 1, wherein: Obtaining (700) a shifted pair of refinement parameters includes directly right-shifting each of the first refinement parameter and the second refinement parameter by one bit.

4. The method according to claim 3, wherein: The first refinement parameter is given by v x represents the variable, the second refinement parameter is given by v y represents a variable, and wherein the first refinement parameter v x and the second refinement parameter v y Each direct right shift of one bit in is determined by the following operation: v x = v x >> 1; and v y = v y >> 1 The >> indicates the right shift of the variable.

5. The method according to claim 1, wherein: Determining (702) a BDOF offset using the shifted pair of refinement parameters without performing any right shift in determining (702) the BDOF offset is determined according to the following operations: bdofOffset = ( v x * ( gradientHL0[ x + 1 ][ y + 1 ] – gradientHL1[ x + 1 ][ y + 1 ] ) ) + ( v y * (gradientVL0[ x + 1 ][ y + 1 ] – gradientVL1[ x + 1 ][ y + 1 ] ) ) Where bdofOffset is the bidirectional optical flow offset BDOF offset, v x is the first refinement parameter, v y is the second refinement parameter, gradientHL0[ x + 1 ][ y + 1 ] is the first horizontal gradient block, gradientHL1[ x + 1 ][ y + 1 ] is the second horizontal gradient block, gradientVL0[ x + 1 ][ y + 1 ] is the first vertical gradient block, and gradientVL1[ x + 1 ][ y + 1 ] is the second vertical gradient block.

6. The method according to claim 1, wherein: For each subblock of a bi-directionally predicted inter-coded block, obtaining (700) a shifted pair of refinement parameters to obtain the shifted pair of refinement parameters includes adding an offset to each of the first refinement parameter and the second refinement parameter before right shifting.

7. The method according to claim 6, wherein: The first refinement parameter is given by v x represents the variable, the second refinement parameter is given by v y A variable represented by , and wherein adding an offset to each of the first refinement parameter and the second refinement parameter before right shifting is performed according to the following operation: v x = (v x +1) >> 1; and v y = (v y +1) >> 1 The >> indicates the right shift of the variable.

8. The method according to claim 6, wherein: Determining (702) a BDOF offset using the shifted pair of refinement parameters without performing any right shift in determining (702) the BDOF offset includes: bdofOffset = ( v x * ( gradientHL0[ x + 1 ][ y + 1 ] – gradientHL1[ x + 1 ][ y + 1 ] ) ) + ( v y * (gradientVL0[ x + 1 ][ y + 1 ] – gradientVL1[ x + 1 ][ y + 1 ] ) ) Where bdofOffset is the bidirectional optical flow offset BDOF offset, v x is the first refinement parameter with the added offset, v y are the second refinement parameters with the added offset, gradientHL0[ x + 1 ][ y + 1 ] is the first horizontal gradient block, gradientHL1[ x + 1 ][ y + 1 ] is the second horizontal gradient block, gradientVL0[ x + 1 ][ y + 1 ] is the first vertical gradient block, and gradientVL1[ x + 1 ][ y + 1 ] is the second vertical gradient block.

9. An electronic device (410) for performing bidirectional optical flow (BDOF) processing on a video sequence comprising a plurality of images, wherein each image of the plurality of images comprises a plurality of blocks of inter-coded blocks having bidirectional prediction, the electronic device being adapted to perform an operation comprising the following steps: For each subblock of a bidirectionally predicted inter-coded block, a shifted pair of refinement parameters is obtained (700), where The shifted pair of refinement parameters comprises a shifted first refinement parameter and a shifted second refinement parameter; and Using at least the shifted pair of refinement parameters, a BDOF offset is determined (702) by the following operation without performing any right shift when determining (702) the BDOF offset: for each sample in each subblock within each bidirectionally predicted inter-frame coded block, the BDOF offset is determined using the shifted first refinement parameter, the shifted second refinement parameter, a first horizontal gradient block and a first vertical gradient block of a first prediction block, and a second horizontal gradient block and a second vertical gradient block of a second prediction block.

10. The electronic device (410) according to claim 9, adapted to perform the operations according to any one of claims 2 to 8.

11. An electronic device (410) for performing bidirectional optical flow (BDOF) processing on a video sequence comprising a plurality of images, wherein each image of the plurality of images comprises a plurality of blocks having bidirectionally predicted inter-frame coded blocks, the electronic device comprising: at least one processor (402, 420); At least one memory (404, 422) connected to the at least one processor (402, 420) and storing program code, the program code being executed by the at least one processor to perform operations comprising the following steps: For each subblock of a bi-directionally predicted inter-coded block, obtaining (700) a shifted pair of refinement parameters, wherein the shifted pair of refinement parameters comprises a shifted first refinement parameter and a shifted second refinement parameter; and Using at least the shifted pair of refinement parameters, a BDOF offset is determined (702) by the following operation without performing any right shift when determining (702) the BDOF offset: for each sample in each subblock within each bidirectionally predicted inter-frame coded block, the BDOF offset is determined using the shifted first refinement parameter, the shifted second refinement parameter, a first horizontal gradient block and a first vertical gradient block of a first prediction block, and a second horizontal gradient block and a second vertical gradient block of a second prediction block.

12. The electronic device (410) according to claim 11, wherein: The at least one memory further stores program code that is executed by the at least one processor to perform operations further comprising: For each bi-directionally predicted inter-coded block, determining (600) a first prediction block using a first motion vector, and determining (600) a second prediction block using a second motion vector; For each prediction block, determining (602) a first horizontal gradient block and a first vertical gradient block of the first prediction block and a second horizontal gradient block and a second vertical gradient block of the second prediction block; determining (604) a block of prediction sample differences between the first prediction block and the second prediction block; determining (606) each of a horizontal gradient accumulation block and a vertical gradient accumulation block, wherein the horizontal gradient accumulation block is determined by accumulating the first horizontal gradient block and the second horizontal gradient block, and the vertical gradient accumulation block is determined by accumulating the first vertical gradient block and the second vertical gradient block; For each subblock within each bidirectionally predicted inter-frame coded block, determine (608) a set of BDOF-related parameters using the most significant bit of the horizontal gradient accumulation block, the most significant bit of the vertical gradient accumulation block, and the most significant bit and the least significant bit of the predicted sample difference block; determining ( 610 ) the first refinement parameter and the second refinement parameter using the set of BDOF-related parameters; clipping (612) the value of each of the first refinement parameter and the second refinement parameter to within a defined range of values; and Prediction of luma sample values ​​in each bi-directionally predicted inter-coded block is performed (614) using predicted sample values ​​for each sample in each sub-block within each bi-directionally predicted inter-coded block using the BDOF offset, the first prediction block, and the first horizontal gradient block.

13. The electronic device (410) according to claim 11, wherein: Obtaining (700) a shifted pair of refinement parameters includes directly right-shifting each of the first refinement parameter and the second refinement parameter by one bit.

14. The electronic device (410) according to claim 13, wherein: The first refinement parameter is given by v x represents the variable, the second refinement parameter is given by v y represents a variable, and wherein the first refinement parameter v x and the second refinement parameter v y Each direct right shift of one bit in is determined by the following operation: v x = v x >> 1; and v y = v y >> 1 The >> indicates the right shift of the variable.

15. The electronic device (410) according to any one of claims 11 to 13, wherein: When determining (702) a BDOF offset using the shifted pair of refinement parameters without performing any right shifting in determining (702) the BDOF offset, the at least one memory stores program code that is executed by the at least one processor to perform operations including determining the BDOF offset according to the following steps: bdofOffset = ( v x * ( gradientHL0[ x + 1 ][ y + 1 ] – gradientHL1[ x + 1 ][ y + 1 ] ) ) + ( v y * (gradientVL0[ x + 1 ][ y + 1 ] – gradientVL1[ x + 1 ][ y + 1 ] ) ) Where bdofOffset is the bidirectional optical flow offset BDOF offset, v x is the shifted first refinement parameter, v y is the shifted second refinement parameter, gradientHL0[ x + 1 ][ y + 1 ] is the first horizontal gradient block, gradientHL1[ x + 1 ][ y + 1 ] is the second horizontal gradient block, gradientVL0[ x + 1 ][ y + 1 ] is the first vertical gradient block, and gradientVL1[ x + 1 ][ y + 1 ] is the second vertical gradient block.

16. The electronic device (410) according to claim 11, wherein: In the case where, for each subblock of a bidirectionally predicted inter-frame coded block, a shifted pair of refinement parameters is obtained (700) to obtain the shifted pair of refinement parameters, the at least one memory further stores program code that is executable by the at least one processor to perform operations including adding an offset to each of the first refinement parameter and the second refinement parameter prior to right shifting.

17. The electronic device (410) according to claim 16, wherein: The first refinement parameter is given by v x represents the variable, the second refinement parameter is given by v y wherein, when adding an offset to each of the first refinement parameter and the second refinement parameter before right shifting, the at least one memory further stores the following program code: the program code is executed by the at least one processor to perform the operation of adding the offset according to the following steps: v x = (v x +1) >> 1; and v y = (v y +1) >> 1 The >> indicates the right shift of the variable.

18. The electronic device (410) according to any one of claims 16 to 17, wherein: When determining (702) a BDOF offset using the shifted pair of refinement parameters and when no right shift is performed in determining (702) the BDOF offset, the at least one memory further stores program code that is executed by the at least one processor to perform operations including determining the BDOF offset according to the following steps: bdofOffset = ( v x * ( gradientHL0[ x + 1 ][ y + 1 ] – gradientHL1[ x + 1 ][ y + 1 ] ) ) + ( v y * (gradientVL0[ x + 1 ][ y + 1 ] – gradientVL1[ x + 1 ][ y + 1 ] ) ) Where bdofOffset is the bidirectional optical flow offset BDOF offset, v x is the first refinement parameter with the added offset, v y are the second refinement parameters with the added offset, gradientHL0[ x + 1 ][ y + 1 ] is the first horizontal gradient block, gradientHL1[ x + 1 ][ y + 1 ] is the second horizontal gradient block, gradientVL0[ x + 1 ][ y + 1 ] is the first vertical gradient block, and gradientVL1[ x + 1 ][ y + 1 ] is the second vertical gradient block.

19. A computer program product comprising a non-transitory storage medium, the non-transitory storage medium comprising program code to be executed by at least one processor (402, 420) of an electronic device (410), whereby execution of the program code causes the electronic device to perform operations comprising the following steps: For each subblock of a bidirectionally predicted inter-coded block, a shifted pair of refinement parameters is obtained (700), where The shifted pair of refinement parameters comprises a shifted first refinement parameter and a shifted second refinement parameter; and Using at least the shifted pair of refinement parameters, a BDOF offset is determined (702) in the following manner without performing any right shift when determining (702) the BDOF offset: for each sample in each subblock within each bidirectionally predicted inter-frame coded block, the BDOF offset is determined using the shifted first refinement parameter, the shifted second refinement parameter, a first horizontal gradient block and a first vertical gradient block of a first prediction block, and a second horizontal gradient block and a second vertical gradient block of a second prediction block.

20. The computer program product of claim 19, whereby execution of the program code causes the electronic device (410) to perform operations according to any one of claims 2 to 8.

Images