Reference frame acquisition method and device used in bidirectional inter-frame prediction

Abstract

The embodiment of the present application discloses a reference frame acquisition method and device for bidirectional inter-frame prediction, which relates to the technical field of video image coding and decoding, and improves coding and decoding efficiency. The method includes: determining a first reference frame list and a second reference frame list of a current image block; determining from the first reference frame list whether there is an X-direction reference frame x1 that satisfies condition A1; determining from the second reference frame list whether there is a Y-direction reference frame y1 that satisfies condition B1; and obtaining a bidirectional reference frame index value (R1, R2) of the current image block in response to the determined existence of the X-direction reference frame x1 and the determined existence of the Y-direction reference frame y1.

Description

[0001] This application claims priority to U.S. patent applications filed with the U.S. Patent Office on September 4, 2018, with application number US62 / 726,975 and application name “A method and device for bidirectional inter-frame prediction”; claims priority to U.S. patent applications filed with the U.S. Patent Office on September 5, 2018, with application number US62 / 727,534 and application name “A method and device for bidirectional inter-frame prediction”; and claims priority to U.S. patent applications filed with the U.S. Patent Office on September 20, 2018, with application number US62 / 734,226 and application name “A method and device for bidirectional inter-frame prediction”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the technical field of video image coding and decoding, and in particular to a bidirectional inter-frame prediction method and device. Background Art

[0003] In video coding and decoding technology, for a current image block, a predicted image block of the current image block can be generated based on only one reference image block (i.e., unidirectional inter-frame prediction), or a predicted image block of the current image block can be generated based on at least two reference image blocks (i.e., bidirectional inter-frame prediction), and the above-mentioned at least two reference image blocks can come from the same reference frame or different reference frames.

[0004] In order to make the decoder and encoder use the same reference image block, the encoder needs to send the motion information of each image block to the decoder in the bitstream. Generally, the motion information of the current image block includes the index value of the reference frame, the motion vector predictor (MVP) flag and the motion vector difference (MVD). The decoder can find the correct reference image block in the selected reference frame based on the reference frame index value, MVP flag and MVD.

[0005] Correspondingly, for bidirectional inter-frame prediction, the encoder needs to send the motion information of each image block in each direction to the decoder. In this way, the motion information occupies a large amount of transmission resources, reduces the effective utilization of transmission resources, reduces the transmission rate, and reduces the encoding and decoding compression efficiency. Summary of the invention

[0006] The embodiments of the present application provide a bidirectional inter-frame prediction method and device, which can reduce the transmission resources occupied by motion information, reduce the effective utilization of transmission resources, reduce the transmission rate, and reduce the problem of codec compression efficiency.

[0007] In order to achieve the above purpose, the embodiment of the present application adopts the following technical solution:

[0008] In a first aspect, a bidirectional inter-frame prediction method is provided, which obtains indication information for indicating how to determine second motion information based on first motion information, where the first motion information is motion information of a current image block in a first direction, and the second motion information is motion information of the current image block in a second direction; obtains the first motion information; and determines the second motion information based on the obtained first motion information, so that the predicted pixels of the current image block can be determined based on the obtained first motion information and the determined second motion information.

[0009] The bidirectional inter-frame prediction method provided by the present application determines the second motion information according to the first motion information after obtaining the indication information, so that the code stream only includes the indication information and the first motion information, and no longer needs to include the second motion information. Compared with the prior art, in which the code stream includes the motion information of each image block in each direction, the motion information included in the code stream is effectively reduced, the effective utilization of transmission resources is improved, the transmission rate is improved, and accordingly, the encoding and decoding rate is also improved.

[0010] Optionally, in a possible implementation of the present application, the above-mentioned method of "determining the second motion information based on the first motion information" is: obtaining the index value of the first reference frame in the first motion information, and determining the serial number of the first reference frame according to the index value of the first reference frame and the first reference frame list, where the first reference frame is a reference frame of the current image block in the first direction, and the index value of the first reference frame is the number of the first reference frame in the first reference frame list; obtaining the index value of the second reference frame, and determining the serial number of the second reference frame according to the index value of the second reference frame and the second reference frame list, where the second reference frame is a reference frame of the current image block in the second direction, and the index value of the second reference frame is the number of the second reference frame in the second reference frame list; determining the first motion vector according to the first motion vector difference and the first motion vector prediction value flag in the first motion information, the first motion vector being the motion vector of the current image block in the first direction; determining the second motion vector in the second motion information according to the following formula:

[0011]

[0012] In the formula, mv_lY represents the second motion vector, POC_Cur represents the sequence number of the current frame, POC_listX represents the sequence number of the first reference frame, POC_listY represents the sequence number of the second reference frame, mv_lX represents the first motion vector, and the second motion vector is the motion vector of the current image block in the second direction.

[0013] Optionally, in another possible implementation of the present application, the method of "determining the second motion information according to the first motion information" is: obtaining an index value of a first reference frame in the first motion information, and determining a serial number of the first reference frame according to the index value of the first reference frame and the first reference frame list, the first reference frame being a reference frame of the current image block in the first direction, and the index value of the first reference frame being the number of the first reference frame in the first reference frame list; obtaining an index value of a second reference frame, and determining a serial number of the second reference frame according to the index value of the second reference frame and the second reference frame list, the second reference frame being a reference frame of the current image block in the second direction, and the index value of the second reference frame being the number of the second reference frame in the second reference frame list; according to the first motion vector difference and the first motion vector prediction value flag in the first motion information, Determine a first motion vector, where the first motion vector is a motion vector of the current image block in a first direction; when the first reference frame is a forward reference frame of the current image block and the second reference frame is a backward reference frame of the current image block, or when the first reference frame is a backward reference frame of the current image block and the second reference frame is a forward reference frame of the current image block, or when both the first reference frame and the second reference frame are forward reference frames of the current image block, or when both the first reference frame and the second reference frame are backward reference frames of the current image block, determine a second motion vector in the second motion information according to a formula mv_lY=-mv_lX, where mv_lY represents the second motion vector, mv_lX represents the first motion vector, and the second motion vector is the motion vector of the current image block in the second direction.

[0014] Optionally, in another possible implementation of the present application, the above-mentioned method of "determining the second motion information based on the first motion information" is: determine the second motion vector in the second motion information according to the formula mv_lY=-mv_lX, in which formula, mv_lY represents the second motion vector, mv_lX represents the first motion vector, and the second motion vector is the motion vector of the current image block in the second direction.

[0015] The above-mentioned "the case where the first reference frame is the forward reference frame of the current image block, and the second reference frame is the backward reference frame of the current image block", or "the case where the first reference frame is the backward reference frame of the current image block, and the second reference frame is the forward reference frame of the current image block" can be expressed by the formula (POC_Cur-POC_listX)*(POC_listY-POC_Cur)>0, or by the formula POC_listY=2*POC_Cur-POC_listX, and the present application does not make any specific limitation on this.

[0016] In addition, the above "the case where the first reference frame and the second reference frame are both forward reference frames of the current image block" or "the case where the first reference frame and the second reference frame are both backward reference frames of the current image block" can be expressed by the formula (POC_Cur-POC_listX)*(POC_listY-POC_Cur)<0.

[0017] Optionally, in another possible implementation of the present application, the above-mentioned method of "determining the second motion information according to the first motion information" is: obtaining the index value of the first reference frame in the first motion information and the first motion vector difference, and determining the serial number of the first reference frame according to the index value of the first reference frame and the first reference frame list, the first reference frame is a reference frame of the current image block in the first direction, and the index value of the first reference frame is the number of the first reference frame in the first reference frame list; obtaining the index value of the second reference frame, and determining the serial number of the second reference frame according to the index value of the second reference frame and the second reference frame list, determining the second predicted motion vector according to the index value of the second reference frame and the second candidate predicted motion vector list, the second predicted motion vector is the predicted motion vector of the current image block in the second direction, the second reference frame is a reference frame of the current image block in the second direction, and the index value of the second reference frame is the number of the second reference frame in the second reference frame list; determining the second motion vector difference in the second motion information according to the following formula:

[0018]

[0019] In this formula, mvd_lY represents the second motion vector difference, POC_Cur represents the sequence number of the current frame, POC_listX represents the sequence number of the first reference frame, POC_listY represents the sequence number of the second reference frame, and mvd_lX represents the first motion vector difference; according to the second predicted motion vector and the second motion vector difference, the second motion vector is determined, and the second motion vector is the motion vector of the current image block in the second direction.

[0020] Optionally, in another possible implementation of the present application, the above-mentioned method of "determining the second motion information according to the first motion information" is: obtain the index value of the first reference frame and the first motion vector in the first motion information, and determine the serial number of the first reference frame according to the index value of the first reference frame and the first reference frame list, the first reference frame is a reference frame of the current image block in the first direction, and the index value of the first reference frame is the number of the first reference frame in the first reference frame list; obtain the index value of the second reference frame, and determine the serial number of the second reference frame according to the index value of the second reference frame and the second reference frame list, determine the second predicted motion vector according to the index value of the second reference frame and the second candidate predicted motion vector list, the second predicted motion vector is the predicted motion vector of the current image block in the second direction, the second reference frame is the reference frame of the current image block in the second direction, and the index value of the second reference frame is the first reference frame. the number of the second reference frame in the second reference frame list; when the first reference frame is the forward reference frame of the current image block and the second reference frame is the backward reference frame of the current image block, or when the first reference frame is the backward reference frame of the current image block and the second reference frame is the forward reference frame of the current image block, or when the first reference frame and the second reference frame are both the forward reference frames of the current image block, or when the first reference frame and the second reference frame are both the backward reference frames of the current image block, determining the second motion vector difference in the second motion information according to the formula mvd_lY=-mvd_lX, in which formula mvd_lY represents the second motion vector difference and mvd_lX represents the first motion vector difference; determining the second motion vector according to the second predicted motion vector and the second motion vector difference, the second motion vector being the motion vector of the current image block in the second direction.

[0021] Optionally, in another possible implementation of the present application, the above-mentioned method of "determining the second motion information based on the first motion information" is: determine the second motion vector difference in the second motion information according to the formula mvd_lY=-mvd_lX, in which formula, mvd_lY represents the second motion vector difference, and mvd_lX represents the first motion vector difference; determine the second motion vector based on the second predicted motion vector and the second motion vector difference, and the second motion vector is the motion vector of the current image block in the second direction.

[0022] Similarly, the above-mentioned "the first reference frame is the forward reference frame of the current image block, and the second reference frame is the backward reference frame of the current image block", or "the first reference frame is the backward reference frame of the current image block, and the second reference frame is the forward reference frame of the current image block" can be expressed by the formula (POC_Cur-POC_listX)*(POC_listY-POC_Cur)>0, or by the formula POC_listY=2*POC_Cur-POC_listX, and the present application does not make any specific limitation on this.

[0023] The above "the case where both the first reference frame and the second reference frame are forward reference frames of the current image block" or "the case where both the first reference frame and the second reference frame are backward reference frames of the current image block" can be expressed by the formula (POC_Cur-POC_listX)*(POC_listY-POC_Cur)<0.

[0024] It can be seen that the bidirectional inter-frame prediction method provided in the present application can determine the second motion vector according to the first motion vector, or can determine the second motion vector difference according to the first motion vector difference, and determine the second motion vector according to the second motion vector difference.

[0025] Optionally, in another possible implementation of the present application, the above-mentioned method of "obtaining the index value of the second reference frame" is: according to the sequence number of the current frame and the sequence number of the first reference frame, the first sequence number is calculated by the formula POC_listY0=2*POC_Cur-POC_listX, wherein POC_Cur represents the sequence number of the current frame, POC_listX represents the sequence number of the first reference frame, and POC_listY represents the first sequence number; when the second reference frame list includes the first sequence number, the number of the reference frame represented by the first sequence number in the second reference frame list is determined as the index value of the second reference frame.

[0026] Optionally, in another possible implementation of the present application, the above-mentioned method of "obtaining the index value of the second reference frame" is: according to the sequence number of the current frame and the sequence number of the first reference frame, the second sequence number is calculated by the formula (POC_Cur-POC_listX)*(POC_listY0′-POC_Cur)>0, where POC_listY0′ represents the second sequence number; when the second reference frame list includes the second sequence number, the number of the reference frame represented by the second sequence number in the second reference frame list is determined as the index value of the second reference frame.

[0027] Optionally, in another possible implementation of the present application, the above-mentioned method of "obtaining the index value of the second reference frame" is: according to the sequence number of the current frame and the sequence number of the first reference frame, calculate the third sequence number through the formula POC_listX≠POC_listY0″, where POC_listY0″ represents the third sequence number; determine the number of the reference frame represented by the third sequence number in the second reference frame list as the index value of the second reference frame.

[0028] Optionally, in another possible implementation of the present application, the above-mentioned method of "obtaining the index value of the second reference frame" is: according to the sequence number of the current frame and the sequence number of the first reference frame, the first sequence number is calculated by the formula POC_listY0=2*POC_Cur-POC_listX, wherein POC_Cur represents the sequence number of the current frame, POC_listX represents the sequence number of the first reference frame, and POC_listY represents the first sequence number. In the case where the second reference frame list includes the first sequence number, the number of the reference frame represented by the first sequence number in the second reference frame list is determined as the index value of the second reference frame. In the case where the second reference frame list does not include the first sequence number, according to the sequence number of the current frame and the sequence number of the first reference frame, the second sequence number is calculated by the formula (POC_Cur-POC_listX)*(POC_listY0′-POC_Cur)>0, wherein POC_listY0′ represents the second sequence number. In the case where the second reference frame list includes the second sequence number, the number of the reference frame represented by the second sequence number in the second reference frame list is determined as the index value of the second reference frame. In the case where the second reference frame list does not include the second sequence number, the third sequence number is calculated according to the sequence number of the current frame and the sequence number of the first reference frame by the formula POC_listX≠POC_listY0″, where POC_listY0″ represents the third sequence number; the number of the reference frame represented by the third sequence number in the second reference frame list is determined as the index value of the second reference frame.

[0029] Optionally, in another possible implementation of the present application, the above method of "obtaining the index value of the second reference frame" is: parsing the code stream to obtain the index value of the second reference frame.

[0030] It can be seen that there may be multiple methods for “obtaining the index value of the second reference frame” in the present application, and the specific method to be adopted to obtain the index value of the second reference frame needs to be determined according to actual needs or pre-settings.

[0031] In a second aspect, a bidirectional inter-frame prediction device is provided, and the bidirectional inter-frame prediction device includes an acquisition unit and a determination unit.

[0032] Specifically, the acquisition unit is used to acquire indication information, the indication information is used to indicate determining the second motion information according to the first motion information, the first motion information is the motion information of the current image block in the first direction, the second motion information is the motion information of the current image block in the second direction, and acquiring the first motion information. The determination unit is used to determine the second motion information according to the first motion information acquired by the acquisition unit, and to determine the predicted pixel of the current image block according to the first motion information and the second motion information.

[0033] Optionally, in a possible implementation of the present application, the above-mentioned determination unit is specifically used to: obtain an index value of a first reference frame in the first motion information, and determine a serial number of the first reference frame according to the index value of the first reference frame and the first reference frame list, the first reference frame being a reference frame of the current image block in the first direction, and the index value of the first reference frame being the number of the first reference frame in the first reference frame list; obtain an index value of a second reference frame, and determine a serial number of the second reference frame according to the index value of the second reference frame and the second reference frame list, the second reference frame being a reference frame of the current image block in the second direction, and the index value of the second reference frame being the number of the second reference frame in the second reference frame list; determine a first motion vector according to a first motion vector difference and a first motion vector prediction value flag in the first motion information, the first motion vector being the motion vector of the current image block in the first direction; and determine a second motion vector in the second motion information according to the following formula:

[0034]

[0035] Among them, mv_lY represents the second motion vector, POC_Cur represents the sequence number of the current frame, POC_listX represents the sequence number of the first reference frame, POC_listY represents the sequence number of the second reference frame, mv_lX represents the first motion vector, and the second motion vector is the motion vector of the current image block in the second direction.

[0036] Optionally, in another possible implementation of the present application, the above-mentioned determination unit is specifically used to: obtain an index value of a first reference frame in the first motion information, and determine a serial number of the first reference frame according to the index value of the first reference frame and the first reference frame list, the first reference frame being a reference frame of the current image block in the first direction, and the index value of the first reference frame being the number of the first reference frame in the first reference frame list; obtain an index value of a second reference frame, and determine a serial number of the second reference frame according to the index value of the second reference frame and the second reference frame list, the second reference frame being a reference frame of the current image block in the second direction, and the index value of the second reference frame being the number of the second reference frame in the second reference frame list; determine the first motion vector difference according to the first motion vector prediction value flag in the first motion information. The first motion vector is the motion vector of the current image block in the first direction; when the first reference frame is the forward reference frame of the current image block and the second reference frame is the backward reference frame of the current image block, or when the first reference frame is the backward reference frame of the current image block and the second reference frame is the forward reference frame of the current image block, or when the first reference frame and the second reference frame are both the forward reference frames of the current image block, or when the first reference frame and the second reference frame are both the backward reference frames of the current image block, the second motion vector in the second motion information is determined according to the formula mv_lY=-mv_lX, in which formula, mv_lY represents the second motion vector, mv_lX represents the first motion vector, and the second motion vector is the motion vector of the current image block in the second direction.

[0037] Optionally, in another possible implementation of the present application, the above-mentioned determination unit is specifically used to: determine the second motion vector in the second motion information according to the formula mv_lY=-mv_lX, in which formula, mv_lY represents the second motion vector, mv_lX represents the first motion vector, and the second motion vector is the motion vector of the current image block in the second direction.

[0038] Optionally, in another possible implementation of the present application, the above-mentioned determination unit is specifically used to: obtain the index value of the first reference frame and the first motion vector difference in the first motion information, and determine the serial number of the first reference frame according to the index value of the first reference frame and the first reference frame list, the first reference frame is a reference frame of the current image block in the first direction, and the index value of the first reference frame is the number of the first reference frame in the first reference frame list; obtain the index value of the second reference frame, and determine the serial number of the second reference frame according to the index value of the second reference frame and the second reference frame list, determine the second predicted motion vector according to the index value of the second reference frame and the second candidate predicted motion vector list, the second predicted motion vector is the predicted motion vector of the current image block in the second direction, the second reference frame is the reference frame of the current image block in the second direction, and the index value of the second reference frame is the number of the second reference frame in the second reference frame list; determine the second motion vector difference in the second motion information according to the following formula:

[0039]

[0040] Among them, mvd_lY represents the second motion vector difference, POC_Cur represents the sequence number of the current frame, POC_listX represents the sequence number of the first reference frame, POC_listY represents the sequence number of the second reference frame, and mvd_lX represents the first motion vector difference; according to the second predicted motion vector and the second motion vector difference, the second motion vector is determined, and the second motion vector is the motion vector of the current image block in the second direction.

[0041] Optionally, in another possible implementation of the present application, the above-mentioned determination unit is specifically used to: obtain the index value and the first motion vector of the first reference frame in the first motion information, and determine the serial number of the first reference frame according to the index value of the first reference frame and the first reference frame list, the first reference frame being a reference frame of the current image block in the first direction, and the index value of the first reference frame being the number of the first reference frame in the first reference frame list; obtain the index value of the second reference frame, and determine the serial number of the second reference frame according to the index value of the second reference frame and the second reference frame list, determine the second predicted motion vector according to the index value of the second reference frame and the second candidate predicted motion vector list, the second predicted motion vector being the predicted motion vector of the current image block in the second direction, the second reference frame being the reference frame of the current image block in the second direction, and the index value of the second reference frame being the number of the second reference frame in the first reference frame list. The number in the reference frame list; when the first reference frame is the forward reference frame of the current image block and the second reference frame is the backward reference frame of the current image block, or when the first reference frame is the backward reference frame of the current image block and the second reference frame is the forward reference frame of the current image block, or when the first reference frame and the second reference frame are both the forward reference frames of the current image block, or when the first reference frame and the second reference frame are both the backward reference frames of the current image block, determine the second motion vector difference in the second motion information according to the formula mvd_lY=-mvd_lX, wherein mvd_lY represents the second motion vector difference and mvd_lX represents the first motion vector difference; determine the second motion vector according to the second predicted motion vector and the second motion vector difference, the second motion vector being the motion vector of the current image block in the second direction.

[0042] Optionally, in another possible implementation of the present application, the above-mentioned determination unit is specifically used to: determine the second motion vector difference in the second motion information according to the formula mvd_lY=-mvd_lX, in which formula, mvd_lY represents the second motion vector difference, and mvd_lX represents the first motion vector difference; determine the second motion vector according to the second predicted motion vector and the second motion vector difference, and the second motion vector is the motion vector of the current image block in the second direction.

[0043] Optionally, in another possible implementation of the present application, the above-mentioned acquisition unit is specifically used to: calculate the first sequence number according to the sequence number of the current frame and the sequence number of the first reference frame by the formula POC_listY0=2*POC_Cur-POC_listX, wherein POC_Cur represents the sequence number of the current frame, POC_listX represents the sequence number of the first reference frame, and POC_listY represents the first sequence number; when the second reference frame list includes the first sequence number, the number of the reference frame represented by the first sequence number in the second reference frame list is determined as the index value of the second reference frame.

[0044] Optionally, in another possible implementation of the present application, the above-mentioned acquisition unit is specifically used to: calculate the second sequence number according to the sequence number of the current frame and the sequence number of the first reference frame by the formula (POC_Cur-POC_listX)*(POC_listY0′-POC_Cur)>0, where POC_listY0′ represents the second sequence number; when the second reference frame list includes the second sequence number, determine the number of the reference frame represented by the second sequence number in the second reference frame list as the index value of the second reference frame.

[0045] Optionally, in another possible implementation of the present application, the above-mentioned acquisition unit is specifically used to: calculate the third sequence number according to the sequence number of the current frame and the sequence number of the first reference frame by the formula POC_listX≠POC_listY0″, where POC_listY0″ represents the third sequence number; determine the number of the reference frame represented by the third sequence number in the second reference frame list as the index value of the second reference frame.

[0046] In a third aspect, a bidirectional inter-frame prediction method is provided. There are multiple implementations of the bidirectional inter-frame prediction method:

[0047] One implementation method is: parsing the bit stream, obtaining a first identifier, the first identifier is used to indicate whether to determine the second motion information based on the first motion information, the first motion information is the motion information of the current image block in the first direction, and the second motion information is the motion information of the current image block in the second direction; if the value of the first identifier is a first preset value, obtaining the first motion information, and determining the second motion information based on the first motion information; determining the predicted pixel of the current image block based on the first motion information and the second motion information.

[0048] Another implementation method is: parsing the bit stream, obtaining a second identifier, the second identifier is used to indicate whether to use a motion information derivation algorithm to calculate the motion information of the current image block; if the value of the second identifier is a second preset value, obtaining a third identifier, the third identifier is used to indicate whether to determine the second motion information based on the first motion information, the first motion information is the motion information of the current image block in the first direction, and the second motion information is the motion information of the current image block in the second direction; if the value of the third identifier is a third preset value, obtaining the first motion information, and determining the second motion information based on the first motion information; determining the predicted pixels of the current image block based on the first motion information and the second motion information.

[0049] Another implementation method is: parsing the bit stream to obtain a second identifier, the second identifier is used to indicate whether to use a motion information derivation algorithm to calculate the motion information of the current image block; if the value of the second identifier is a second preset value, obtaining the first motion information, and determining the second motion information based on the first motion information, the first motion information is the motion information of the current image block in a first direction, and the second motion information is the motion information of the current image block in a second direction; determining the predicted pixels of the current image block based on the first motion information and the second motion information.

[0050] Another implementation method is: parsing the bitstream to obtain a fourth identifier, the fourth identifier is used to indicate whether to use a motion information derivation algorithm to calculate the motion information of the current image block; if the value of the fourth identifier is a fourth preset value, determining the index value of the first reference frame and the index value of the second reference frame according to the first reference frame list and the second reference frame list, the first reference frame list is a reference frame list of the current image block in the first direction, the second reference frame list is a reference frame list of the current image block in the second direction, the first reference frame is a reference frame of the current image block in the first direction, and the second reference frame is a reference frame of the current image block in the second direction; obtaining a first motion vector difference and a first motion vector prediction value flag, and determining the second motion information according to the first motion information, the first motion information includes the index value of the first reference frame, the first motion vector difference and the first motion vector prediction value flag, and the second motion information is the motion information of the current image block in the second direction; determining the predicted pixel of the current image block according to the first motion information and the second motion information.

[0051] Another implementation method is: parsing the bit stream, obtaining a first identifier, the first identifier is used to indicate whether to determine the second motion information based on the first motion information, the first motion information is the motion information of the current image block in the first direction, and the second motion information is the motion information of the current image block in the second direction; if the value of the first identifier is the eighth preset value, obtaining a fifth identifier, the fifth identifier is used to indicate whether to determine the first motion information based on the second motion information; if the value of the fifth identifier is the fifth preset value, obtaining the second motion information, and determining the first motion information based on the second motion information; determining the predicted pixels of the current image block based on the first motion information and the second motion information.

[0052] Another implementation method is: parsing the bit stream, obtaining a second identifier, the second identifier is used to indicate whether to use a motion information derivation algorithm to calculate the motion information of the current image block; if the value of the second identifier is a second preset value, obtaining a third identifier, the third identifier is used to indicate whether to determine the second motion information based on the first motion information, the first motion information is the motion information of the current image block in the first direction, and the second motion information is the motion information of the current image block in the second direction; if the value of the third identifier is a sixth preset value, obtaining the second motion information, and determining the first motion information based on the second motion information; determining the predicted pixels of the current image block based on the first motion information and the second motion information.

[0053] For detailed descriptions of the first to fourth identifiers, please refer to the subsequent descriptions.

[0054] In the bidirectional inter-frame prediction method provided by the present application, after parsing the code stream to obtain a certain identifier, it is determined whether to determine the second motion information based on the first motion information according to the value of the identifier. After determining that the second motion information needs to be determined based on the first motion information, the first motion information is obtained, and then the second motion information is determined based on the obtained first motion information. In this way, the code stream only includes the corresponding identifier and the first motion information, and there is no need to include the second motion information. Compared with the prior art, in which the code stream includes the motion information of each image block in each direction, the motion information included in the code stream is effectively reduced, the effective utilization of transmission resources is improved, the transmission rate is improved, and correspondingly, the encoding and decoding rate is also improved.

[0055] According to a fourth aspect, a bidirectional inter-frame prediction device is provided, wherein the bidirectional inter-frame prediction device includes an acquisition unit and a determination unit.

[0056] Specifically, in one implementation, the acquisition unit is used to parse the bitstream and acquire the first identifier, the first identifier is used to indicate whether to determine the second motion information based on the first motion information, the first motion information is the motion information of the current image block in the first direction, the second motion information is the motion information of the current image block in the second direction, and is used to acquire the first motion information if the value of the first identifier is a first preset value. The determination unit is used to determine the second motion information based on the first motion information acquired by the acquisition unit, and is used to determine the predicted pixel of the current image block based on the first motion information and the second motion information.

[0057] In another implementation, the acquisition unit is used to parse the bitstream, acquire the second identifier, the second identifier is used to indicate whether to use the motion information derivation algorithm to calculate the motion information of the current image block, and to acquire the third identifier if the value of the second identifier is a second preset value, the third identifier is used to indicate whether to determine the second motion information based on the first motion information, the first motion information is the motion information of the current image block in the first direction, the second motion information is the motion information of the current image block in the second direction, and to acquire the first motion information if the value of the third identifier is a third preset value. The determination unit is used to determine the second motion information based on the first motion information acquired by the acquisition unit, and to determine the predicted pixels of the current image block based on the first motion information and the second motion information.

[0058] In another implementation, the acquisition unit is used to parse the bitstream to acquire the second identifier, the second identifier is used to indicate whether to use the motion information derivation algorithm to calculate the motion information of the current image block, and is used to acquire the first motion information if the value of the second identifier is a second preset value. The determination unit is used to determine the second motion information according to the first motion information acquired by the acquisition unit, the first motion information is the motion information of the current image block in the first direction, the second motion information is the motion information of the current image block in the second direction, and is used to determine the predicted pixel of the current image block according to the first motion information and the second motion information.

[0059] In another implementation, the acquisition unit is used to parse the code stream to obtain a fourth identifier, and the fourth identifier is used to indicate whether to use the motion information derivation algorithm to calculate the motion information of the current image block. The determination unit is used to determine the index value of the first reference frame and the index value of the second reference frame according to the first reference frame list and the second reference frame list if the value of the fourth identifier obtained by the acquisition unit is a fourth preset value, the first reference frame list is a reference frame list of the current image block in the first direction, the second reference frame list is a reference frame list of the current image block in the second direction, the first reference frame is a reference frame of the current image block in the first direction, and the second reference frame is a reference frame of the current image block in the second direction. The acquisition unit is also used to obtain the first motion vector difference and the first motion vector prediction value flag. The determination unit is also used to determine the second motion information according to the first motion information, the first motion information includes the index value of the first reference frame, the first motion vector difference and the first motion vector prediction value flag, and the second motion information is the motion information of the current image block in the second direction; according to the first motion information and the second motion information, determine the predicted pixel of the current image block.

[0060] In a fifth aspect, a terminal is provided, the terminal comprising: one or more processors, a memory, and a communication interface. The memory and the communication interface are coupled to the one or more processors; the memory is used to store computer program codes, the computer program codes include instructions, and when the one or more processors execute the instructions, the terminal executes the bidirectional inter-frame prediction method as described in the first aspect and any possible implementation thereof, or executes the bidirectional inter-frame prediction method as described in the third aspect and any possible implementation thereof.

[0061] In a sixth aspect, a video decoder is provided, comprising a non-volatile storage medium and a central processing unit, wherein the non-volatile storage medium stores an executable program, the central processing unit is connected to the non-volatile storage medium, and executes the executable program to implement the bidirectional inter-frame prediction method as described in the first aspect and any possible implementation thereof, or the bidirectional inter-frame prediction method as described in the third aspect and any possible implementation thereof.

[0062] In a seventh aspect, a decoder is provided, the decoder comprising the bidirectional inter-frame prediction device and a reconstruction module in the above-mentioned second aspect, wherein the reconstruction module is used to determine the reconstructed pixel value of the current image block based on the predicted pixels obtained by the bidirectional inter-frame prediction device; or, the decoder comprises the bidirectional inter-frame prediction device and a reconstruction module in the above-mentioned fourth aspect, wherein the reconstruction module is used to determine the reconstructed pixel value of the current image block based on the predicted pixels obtained by the bidirectional inter-frame prediction device.

[0063] In an eighth aspect, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium. When the instructions are executed on the terminal described in the fifth aspect, the terminal executes the bidirectional inter-frame prediction method described in the first aspect and any possible implementation thereof, or executes the bidirectional inter-frame prediction method described in the third aspect and any possible implementation thereof.

[0064] In the ninth aspect, a computer program product comprising instructions is provided. When the computer program product is run on the terminal described in the fifth aspect, the terminal executes the bidirectional inter-frame prediction method described in the first aspect and any possible implementation thereof, or executes the bidirectional inter-frame prediction method described in the third aspect and any possible implementation thereof.

[0065] In a tenth aspect, the present invention also provides methods, devices, terminals, storage media and codecs as described in claims 1-26.

[0066] In this application, the name of the above-mentioned bidirectional inter-frame prediction device does not limit the device or functional module itself. In actual implementation, these devices or functional modules may appear with other names. As long as the functions of each device or functional module are similar to those of this application, they belong to the scope of the claims of this application and their equivalent technologies.

[0067] For the specific description of the fifth to ninth aspects and their various implementations in the present application, reference may be made to the detailed description in the first aspect and its various implementations or the detailed description in the third aspect and its various implementations; and, for the beneficial effects of the fifth to ninth aspects and their various implementations, reference may be made to the beneficial effect analysis in the first aspect and its various implementations or the beneficial effect analysis in the third aspect and its various implementations, which will not be repeated here.

[0068] These and other aspects of the present application will become more apparent from the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 This is a schematic diagram of the structure of a video encoding and decoding system in an embodiment of the present application;

[0070] Figure 2 This is a schematic diagram of the structure of a video encoder in an embodiment of the present application;

[0071] Figure 3 This is a schematic diagram of the structure of a video decoder in an embodiment of the present application;

[0072] Figure 4 A schematic diagram of a bidirectional inter-frame prediction method flow chart provided in an embodiment of the present application;

[0073] Figure 5 A schematic diagram of the structure of the bidirectional inter-frame prediction device provided in the embodiment of the present application Figure 1 ;

[0074] Figure 6 A schematic diagram of the structure of the bidirectional inter-frame prediction device provided in the embodiment of the present application Figure 2 . DETAILED DESCRIPTION

[0075] The terms "first", "second", "third" and "fourth" etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects rather than to limit a specific order.

[0076] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0077] In order to facilitate understanding of the embodiments of the present application, relevant elements involved in the embodiments of the present application are first introduced here.

[0078] Image encoding: The process of compressing an image sequence into a code stream.

[0079] Image decoding: The process of restoring the code stream into a reconstructed image according to specific grammatical rules and processing methods.

[0080] At present, the encoding process of video images is as follows: the encoder first divides a frame of the original image into multiple non-overlapping parts, each of which can be regarded as an image block; then, the encoder performs prediction, transformation, and quantization operations on each image block to obtain a code stream corresponding to the image block; wherein, the prediction is to obtain a prediction block of the image block, so that only the difference between the image block and its prediction block (or called a residual or residual block) can be encoded and transmitted, thereby saving transmission overhead; finally, the encoder sends the code stream corresponding to the image block to the decoder.

[0081] Correspondingly, after receiving the code stream, the decoder performs a video decoding process. Specifically, the decoder performs operations such as prediction, inverse quantization, and inverse transformation on the received code stream to obtain a reconstructed image block (or reconstructed image block). This process is called an image reconstruction process (or image reconstruction process); then, the decoder assembles the reconstructed blocks of each image block in the original image to obtain a reconstructed image of the original image, and plays the reconstructed image.

[0082] Existing video image coding and decoding technologies include intra-frame prediction and inter-frame prediction. Among them, inter-frame prediction refers to the prediction completed by using the correlation between the current frame and its reference frame in units of coded image blocks / decoded image blocks. The current frame may have one or more reference frames. Specifically, the predicted image block of the current image block is generated based on the pixels in the reference frame of the current image block.

[0083] Generally, for the current image block, the predicted image block of the current image block can be generated based on only one reference image block, or the predicted image block of the current image block can be generated based on at least two reference image blocks. The above-mentioned generation of the predicted image block of the current image block based on one reference image block is called unidirectional prediction, and the above-mentioned generation of the predicted image block of the current image block based on at least two reference image blocks is called bidirectional inter-frame prediction. At least two reference image blocks in the bidirectional inter-frame prediction can come from the same reference frame or different reference frames. In other words, the "direction" involved in the present application is a broad definition. A direction in the present application corresponds to a reference image block. The first direction and the second direction below correspond to different reference image blocks, and these two reference image blocks can both be included in the forward reference frame / backward reference frame of the current image block, or one can be included in the forward reference frame of the current image block and the other can be included in the backward reference frame of the current image block.

[0084] Optionally, bidirectional inter-frame prediction may refer to inter-frame prediction performed by utilizing the correlation between the current video frame and video frames encoded and played before it, and the correlation between the current video frame and video frames encoded and played after it.

[0085] It can be seen that the above-mentioned bidirectional inter-frame prediction involves inter-frame prediction in two directions, generally referred to as: forward inter-frame prediction and backward inter-frame prediction. Forward inter-frame prediction refers to inter-frame prediction using the correlation between the current video frame and the video frame encoded before and played before it. Backward inter-frame prediction refers to inter-frame prediction using the correlation between the current video frame and the video frame encoded before and played after it.

[0086] Forward inter-frame prediction corresponds to the forward reference frame list L0, and backward inter-frame prediction corresponds to the backward reference frame list L1. The number of reference frames included in the two reference frame lists may be the same or different.

[0087] Motion compensation (MC) is the process of predicting the current image block using a reference image block.

[0088] In most coding frameworks, a video sequence includes a series of pictures, each of which is divided into at least one slice, and each slice is divided into blocks. Video encoding / decoding is based on blocks, and can be encoded / decoded from left to right, from top to bottom, and line by line starting from the upper left corner of the image. Here, the image block can be a macro block (MB) in the video coding standard H.264, or a coding unit (CU) in the High Efficiency Video Coding (HEVC) standard, and the embodiments of the present application do not specifically limit this.

[0089] In the present application, an image block that is being encoded / decoded is referred to as a current image block (current block), and the image in which the current image block is located is referred to as a current frame.

[0090] Generally, the current frame can be a unidirectional prediction frame (P frame) or a bidirectional prediction frame (B frame). When the current frame is a P frame, the current frame has one reference frame list. When the current frame is a B frame, the current frame has two reference frame lists, which are usually called L0 and L1. Each reference frame list contains at least one reconstructed frame used as a reference frame of the current frame. The reference frame is used to provide reference pixels for inter-frame prediction of the current frame.

[0091] In the current frame, image blocks adjacent to the current image block (for example, located on the left, above or right side of the current block) may have completed the encoding / decoding process and obtained reconstructed images, which are called reconstructed image blocks; information such as the encoding mode and reconstructed pixels of the reconstructed image blocks is available.

[0092] A frame that has completed encoding / decoding processing before the current frame is encoded / decoded is called a reconstructed frame.

[0093] Motion Vector (MV) is an important parameter in the inter-frame prediction process, which represents the spatial displacement of the encoded image block relative to the current image block. Generally, the motion vector can be obtained using the motion estimation (ME) method, such as motion search. In the early inter-frame prediction technology, the encoder transmits the motion vector of the current image block in the bitstream so that the decoder can reproduce the predicted pixels of the current image block and obtain the reconstructed block. In order to further improve the coding efficiency, it was later proposed to use the reference motion vector to differentially encode the motion vector, that is, only encode the motion vector difference (MVD).

[0094] In order to make the decoder and the encoder use the same reference image block, the encoder needs to send the motion information of each image block to the decoder in the bitstream. If the encoder directly encodes the motion vector of each image block, it will consume a lot of transmission resources. Since the motion vectors of adjacent image blocks in the spatial domain are highly correlated, the motion vector of the current image block can be predicted based on the motion vector of the adjacent encoded image block. The predicted motion vector is called MVP, and the difference between the motion vector of the current image block and MVP is called MVD.

[0095] The video codec standard H.264 uses multi-reference frame prediction to improve prediction accuracy during motion estimation, that is, a cache is established to store multiple reconstructed frames, and the optimal reference image block is found in all the reconstructed frames in the cache for motion compensation, so as to better remove redundancy in the time domain. The inter-frame prediction of the video codec standard H.264 uses two caches, namely reference frame list 0 (reference list0) and reference frame list 1 (reference list1). The reference frame where the optimal reference block in each list is located is marked with an index value, namely ref_idx_l0 and ref_idx_l1. In each reference frame list, the motion information of the reference image block includes the index value of the reference frame (ref_idx_l0 or ref_idx_l1), the MVP flag and the MVD. The decoder can find the correct reference image block in the selected reference frame based on the index value, MVP flag and MVD of the reference frame.

[0096] At present, inter-frame prediction modes commonly used in the HEVC standard are Advanced Motion Vector Prediction (AMVP) mode, Merge mode and non-translational motion model prediction mode.

[0097] For the AMVP mode, the encoder constructs a candidate motion vector list through the motion information of the coded image blocks adjacent to the current image block in the spatial domain or temporal domain, and determines the optimal motion vector from the candidate motion vector list as the MVP of the current image block according to the rate-distortion cost. In addition, the encoder performs a motion search in the neighborhood centered on the MVP to obtain the motion vector of the current image block. The encoder transmits the index value of the MVP in the candidate motion vector list (i.e., the above-mentioned MVP flag), the index value of the reference frame, and the MVD to the decoder.

[0098] For the merge mode, the encoder constructs a candidate motion information list through the motion information of the coded image blocks adjacent to the current image block in the spatial domain or temporal domain, and determines the optimal motion information from the candidate motion information list as the motion information of the current image block according to the rate-distortion cost. The encoder transmits the index value of the position of the optimal motion information in the candidate motion information list to the decoder.

[0099] For the non-translational motion model prediction mode, the codec uses the same motion model to derive the motion information of each sub-block in the current image block, and performs motion compensation based on the motion information of all sub-blocks to obtain the predicted image block, thereby improving the prediction efficiency. Among them, the motion models commonly used by the codec are the 4-parameter affine model, the 6-parameter affine transformation model, or the 8-parameter bilinear model.

[0100] Exemplarily, the 4-parameter affine transformation model can be represented by the motion vectors of two pixels and their coordinates relative to the upper left vertex pixel of the current image block. Here, the pixel points used to represent the motion model parameters are called control points. If the upper left vertex (0, 0) and upper right vertex (W, 0) pixel points of the current image block are control points, and the motion vectors of the upper left vertex and upper right vertex of the current image block are (vx0, vy0) and (vx1, vy1) respectively, then the motion information of each sub-block in the current image block is obtained according to the following formula (1). In the following formula (1), (x, y) is the coordinate of the sub-block relative to the upper left vertex pixel of the current image block, (vx, vy) is the motion vector of the sub-block, and W is the width of the current image block.

[0101]

[0102] Exemplarily, the 6-parameter affine transformation model can be represented by the motion vectors of three pixels and their coordinates relative to the upper left vertex pixel of the current image block. If the upper left vertex (0, 0), upper right vertex (W, 0) and lower left vertex (0, H) pixels of the current image block are control points, and the motion vectors of the upper left vertex, upper right vertex and lower left vertex of the current image block are (vx0, vy0), (vx1, vy1), (vx2, vy2), respectively, then the motion information of each sub-block in the current image block is obtained according to the following formula (2). In the following formula (2), (x, y) is the coordinate of the sub-block relative to the upper left vertex pixel of the current image block, (vx, vy) is the motion vector of the sub-block, and W and H are the width and height of the current image block, respectively.

[0103]

[0104] Exemplarily, the 8-parameter bilinear model can be represented by the motion vectors of four pixels and their coordinates relative to the upper left vertex pixel of the current image block. If the upper left vertex (0, 0), upper right vertex (W, 0), lower left vertex (0, H) and lower right vertex (W, H) pixels of the current image block are control points, and the motion vectors of the upper left vertex, upper right vertex, lower left vertex and lower right vertex of the current image block are (vx0, vy0), (vx1, vy1), (vx2, vy2), (vx3, vy3), respectively, then the motion information of each sub-block in the current image block is obtained according to the following formula (3). In the following formula (3), (x, y) is the coordinate of the sub-block relative to the upper left vertex pixel of the current image block, (vx, vy) is the motion vector of the sub-block, and W and H are the width and height of the current image block, respectively.

[0105]

[0106] It is easy to see that for any of the above inter-frame prediction modes, if the inter-frame prediction is bidirectional inter-frame prediction, the encoder needs to send the motion information of each image block in each direction to the decoder. In this way, the motion information occupies a large amount of transmission resources, reduces the effective utilization of transmission resources, reduces the transmission rate, and reduces the encoding and decoding compression efficiency.

[0107] In response to the above problems, the present application provides a bidirectional inter-frame prediction method. For bidirectional inter-frame prediction, the encoding end sends the motion information of the current image block in the first direction to the decoding end. After receiving the motion information of the current image block in the first direction, the decoding end calculates the motion information of the current image block in the second direction based on the motion information of the current image block in the first direction. In this way, the predicted pixels of the current image block can be calculated based on the motion information of the current image block in the first direction and the motion information of the current image block in the second direction.

[0108] The bidirectional inter-frame prediction method provided in the present application can be executed by a bidirectional inter-frame prediction device, a video encoding and decoding device, a video codec, and other devices with video encoding and decoding functions.

[0109] The bidirectional inter-frame prediction method provided in the present application is applicable to a video coding and decoding system. The video encoder 100 and the video decoder 200 of the video coding and decoding system are used to implement the calculation of the motion information of the current image block according to the bidirectional inter-frame prediction method example proposed in the present application. Specifically, the motion information of the current image block in the second direction is calculated based on the motion information of the current image block in the first direction, so as to determine the predicted pixels of the current image block based on the motion information of the current image block in the first direction and the motion information of the current image block in the second direction. In this way, only the motion information of the current image block in the first direction needs to be transmitted between the video encoder 10 and the video encoder 20, which effectively improves the utilization rate of transmission resources and improves the coding and decoding compression efficiency.

[0110] Figure 1 The structure of the video encoding and decoding system is shown. Figure 1 As shown, the video encoding and decoding system 1 includes a source device 10 and a destination device 20. The source device 10 generates encoded video data, and the source device 10 can also be called a video encoding device or a video encoding device. The destination device 20 can decode the encoded video data generated by the source device 10, and the destination device 20 can also be called a video decoding device or a video decoding device. The source device 10 and / or the destination device 20 may include at least one processor and a memory coupled to the at least one processor. The memory may include but is not limited to a read-only memory (ROM), a random access memory (RAM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, or any other medium that can be used to store the desired program code in the form of instructions or data structures that can be accessed by a computer, and the present application does not make specific limitations on this.

[0111] Source device 10 and destination device 20 may include a variety of devices, including desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, telephone handsets such as so-called "smart" phones, televisions, cameras, display devices, digital media players, video game consoles, in-vehicle computers, or the like.

[0112] The destination device 20 may receive the encoded video data from the source device 10 via the link 30. The link 30 may include one or more media and / or devices capable of moving the encoded video data from the source device 10 to the destination device 20. In one example, the link 30 may include one or more communication media that enable the source device 10 to transmit the encoded video data directly to the destination device 20 in real time. In this example, the source device 10 may modulate the encoded video data according to a communication standard (e.g., a wireless communication protocol) and may transmit the modulated video data to the destination device 20. The one or more communication media may include wireless and / or wired communication media, such as a radio frequency (RF) spectrum, one or more physical transmission lines. The one or more communication media may form part of a packet-based network, a part of a packet-based network (e.g., a local area network, a wide area network, or a global network (e.g., the Internet)). The one or more communication media may include routers, switches, base stations, or other devices that enable communication from the source device 10 to the destination device 20.

[0113] In another example, the encoded video data can be output from the output interface 140 to the storage device 40. Similarly, the encoded video data can be accessed from the storage device 40 through the input interface 240. The storage device 40 can include a variety of locally accessible data storage media, such as a Blu-ray disc, a high-density digital video disc (DVD), a compact disc read-only memory (CD-ROM), a flash memory, or other suitable digital storage media for storing encoded video data.

[0114] In another example, storage device 40 may correspond to a file server or another intermediate storage device that stores the encoded video data generated by source device 10. In this example, destination device 20 may obtain the video data stored therein from storage device 40 via streaming or downloading. The file server may be any type of server that is capable of storing the encoded video data and transmitting the encoded video data to destination device 20. For example, the file server may include a World Wide Web (Web) server (e.g., for a website), a File Transfer Protocol (FTP) server, a Network Attached Storage (NAS) device, and a local disk drive.

[0115] Destination device 20 may access the encoded video data through any standard data connection (e.g., an Internet connection). Example types of data connections include a wireless channel suitable for accessing the encoded video data stored on a file server, a wired connection (e.g., a cable modem, etc.), or a combination of both. The transmission of the encoded video data from the file server may be a streaming transmission, a download transmission, or a combination of both.

[0116] The bidirectional inter-frame prediction method of the present application is not limited to wireless application scenarios. For example, the bidirectional inter-frame prediction method of the present application can be applied to video encoding and decoding supporting various multimedia applications such as the following applications: over-the-air television broadcasting, cable television transmission, satellite television transmission, streaming video transmission (e.g., via the Internet), encoding of video data stored on a data storage medium, decoding of video data stored on a data storage medium, or other applications. In some examples, the video encoding and decoding system 1 can be configured to support unidirectional or bidirectional video transmission to support applications such as video streaming, video playback, video broadcasting, and / or video telephony.

[0117] It should be noted that Figure 1The video coding system 1 shown is only an example of a video coding system and is not intended to limit the video coding system in the present application. The bidirectional inter-frame prediction method provided in the present application may also be applicable to a scenario where there is no data communication between the coding device and the decoding device. In other examples, the video data to be coded or the coded video data may be retrieved from a local memory, or may be streamed over a network, etc. The video coding device may encode the video data to be coded and store the coded video data in a memory, and the video decoding device may also obtain the coded video data from the memory and decode the coded video data.

[0118] exist Figure 1 , source device 10 includes a video source 101, a video encoder 102, and an output interface 103. In some examples, output interface 103 may include a modem and / or a transmitter. Video source 101 may include a video capture device (e.g., a camera), a video archive containing previously captured video data, a video input interface for receiving video data from a video content provider, and / or a computer graphics system for generating video data, or a combination of such sources of video data.

[0119] Video encoder 102 may encode video data from video source 101. In some examples, source device 10 transmits the encoded video data directly to destination device 20 via output interface 103. In other examples, the encoded video data may also be stored on storage device 40 for later access by destination device 20 for decoding and / or playback.

[0120] exist Figure 1 In the example of , the destination device 20 includes a display device 201, a video decoder 202, and an input interface 203. In some examples, the input interface 203 includes a receiver and / or a modem. The input interface 203 can receive the encoded video data via the link 30 and / or from the storage device 40. The display device 201 can be integrated with the destination device 20 or can be external to the destination device 20. In general, the display device 201 displays the decoded video data. The display device 201 may include a variety of display devices, for example, a liquid crystal display, a plasma display, an organic light emitting diode display, or other types of display devices.

[0121] Alternatively, the video encoder 102 and the video decoder 202 may each be integrated with an audio encoder and decoder, and may include appropriate multiplexer-demultiplexer units or other hardware and software to handle the encoding of both audio and video in a common data stream or in separate data streams.

[0122] The video encoder 102 and the video decoder 202 may include at least one microprocessor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), discrete logic, hardware, or any combination thereof. If the bidirectional inter-frame prediction method provided in the present application is implemented in software, the instructions for the software may be stored in a suitable non-volatile computer-readable storage medium, and at least one processor may be used to execute the instructions in hardware to implement the present application. Any of the foregoing contents (including hardware, software, a combination of hardware and software, etc.) may be regarded as at least one processor. The video encoder 102 may be included in an encoder, and the video decoder 202 may be included in a decoder, and the encoder or decoder may be part of a combined encoder / decoder (encoder-decoder) in a corresponding device.

[0123] The video encoder 102 and the video decoder 202 in the present application may operate according to a video compression standard (eg, HEVC), or may operate according to other industry standards, which is not specifically limited in the present application.

[0124] The video encoder 102 is used to perform bidirectional motion estimation on the current image block to determine the motion information of the current image block in the first direction, and calculate the motion information of the current image block in the second direction based on the motion information of the current image block in the first direction. In this way, the video encoder 102 determines the predicted image block of the current image block based on the motion information of the current image block in the first direction and the motion information of the current image block in the second direction. Furthermore, the video encoder 102 performs operations such as transformation and quantization on the residual between the current image block and its predicted image block, generates a code stream and sends the code stream to the video decoder 202. The code stream includes the motion information of the current image block in the first direction and the indication information for indicating the determination of the second motion information based on the first motion information. The indication information can be represented by different identifiers. The representation method of this indication information can refer to the subsequent description.

[0125] Optionally, the above-mentioned method of "the video encoder 102 calculates the motion information of the current image block in the second direction based on the motion information of the current image block in the first direction" may be that the video encoder 102 determines the motion vector of the current image block in the second direction based on the motion vector of the current image block in the first direction, or may be that the video encoder 102 determines the motion vector difference of the current image block in the second direction based on the motion vector difference of the current image block in the first direction, and determines the motion vector of the current image block in the second direction based on the motion vector difference of the current image block in the second direction and the predicted motion vector of the current image block in the second direction.

[0126] Refer to the following Figure 4 The video decoder 202 is used to: obtain a code stream and parse the code stream to obtain indication information (S400) for indicating how to determine the second motion information based on the first motion information, that is, to determine from which direction the motion information is derived to calculate the motion information in another direction, the first motion information includes the motion information of the current image block in the first direction, and the second motion information includes the motion information of the current image block in the second direction, where the first direction is different from the second direction; obtain the first motion information (S401), and determine the second motion information (S402) based on the obtained first motion information, so that the video decoder 202 can determine the predicted pixels of the current image block based on the first motion information and the second motion information (S403).

[0127] The above-mentioned method of "the video decoder 202 calculates the motion information of the current image block in the second direction based on the motion information of the current image block in the first direction" can be that the video decoder 202 determines the motion vector of the current image block in the second direction based on the motion vector of the current image block in the first direction, or it can be that the video decoder 202 determines the motion vector difference of the current image block in the second direction based on the motion vector difference of the current image block in the first direction, and determines the motion vector of the current image block in the second direction based on the motion vector difference of the current image block in the second direction and the predicted motion vector of the current image block in the second direction.

[0128] Figure 2 1 is a schematic diagram of the structure of the video encoder 102 in the embodiment of the present application. Figure 2As shown, video encoder 102 is used to output video to post-processing entity 41. Post-processing entity 41 represents an example of a video entity that can process the encoded video data from video encoder 102, such as a media aware network element (MANE) or a splicing / editing device. In some cases, post-processing entity 41 may be an example of a network entity. In some video encoding systems, post-processing entity 41 and video encoder 102 may be parts of separate devices, while in other cases, the functionality described with respect to post-processing entity 41 may be performed by the same device that includes video encoder 100. In one example, post-processing entity 41 is Figure 1 An example of a storage device 40.

[0129] The video encoder 102 can calculate the motion information of the current image block in the second direction based on the motion information of the current image block in the first direction, and then determine the predicted image block of the current image block based on the motion information of the current image block in the first direction and the motion information of the current image block in the second direction, thereby completing bidirectional inter-frame prediction coding.

[0130] like Figure 2 As shown, the video encoder 102 includes a transformer 301, a quantizer 302, an entropy encoder 303, a filter 306, a memory 307, a prediction processing unit 308, and a summer 312. The prediction processing unit 308 includes an intra-frame predictor 309 and an inter-frame predictor 310. For image block reconstruction, the video encoder 102 also includes an inverse quantizer 304, an inverse transformer 305, and a summer 311. The filter 306 is intended to represent one or more loop filters, such as a deblocking filter, an adaptive loop filter, and a sample adaptive offset filter.

[0131] The memory 307 may store video data encoded by the components of the video encoder 102. The video data stored in the memory 307 may be obtained from the video source 101. The memory 307 may be a reference image memory that stores reference video data used by the video encoder 102 to encode the video data in intra-frame and inter-frame decoding modes. The memory 307 may be a dynamic random access memory (DRAM) of synchronous DRAM (SDRAM), a magnetic RAM (MRAM), a resistive RAM (RRAM), or other types of memory devices.

[0132] The video encoder 102 receives video data and stores the video data in a video data memory. A segmentation unit segments the video data into a number of image blocks, and these image blocks can be further segmented into smaller blocks, such as image block segmentation based on a quadtree structure or a binary tree structure. This segmentation can also include segmentation into slices, tiles, or other larger units. The video encoder 102 generally describes components for encoding image blocks within a video slice to be encoded. The slice can be divided into a plurality of image blocks (and possibly into sets of image blocks called slices).

[0133] The intra-frame predictor 309 within the prediction processing unit 308 may perform intra-frame predictive encoding of the current image block relative to one or more neighboring image blocks in the same frame or slice as the current image block to remove spatial redundancy. The inter-frame predictor 310 within the prediction processing unit 308 may perform inter-frame predictive encoding of the current image block relative to one or more prediction image blocks in one or more reference images to remove temporal redundancy.

[0134] The prediction processing unit 308 may provide the resulting intra-frame, inter-frame decoded image block to the summer 310 to generate a residual block, and to the summer 309 to reconstruct the coding block used as a reference image.

[0135] After the prediction processing unit 308 generates a prediction image block for the current image block via inter-frame prediction and intra-frame prediction, the video encoder 102 forms a residual image block by subtracting the prediction image block from the current image block to be encoded. The summer 312 represents one or more components that perform this subtraction operation. The residual video data in the residual block may be contained in one or more transform units (TU) and applied to the transformer 301. The transformer 301 transforms the residual video data into residual transform coefficients using a transform such as a discrete cosine transform (DCT) or a conceptually similar transform. The transformer 301 can convert the residual video data from a pixel value domain to a transform domain, such as a frequency domain.

[0136] The transformer 301 may send the resulting transform coefficients to the quantizer 302. The quantizer 302 quantizes the transform coefficients to further reduce the bit rate. In some examples, the quantizer 302 may then perform a scan of the matrix including the quantized transform coefficients. Alternatively, the entropy encoder 303 may perform the scan.

[0137] After quantization, the entropy encoder 303 performs entropy encoding on the quantized transform coefficients. For example, the entropy encoder 303 may perform context adaptive variable length coding (CAVLC), context based adaptive binary arithmetic coding (CABAC), or another entropy encoding method or technique. After entropy encoding by the entropy encoder 303, the encoded bitstream may be sent to the video decoder 202, or archived for later transmission or retrieval by the video decoder 202. The entropy encoder 303 may also perform entropy encoding on syntax elements of the current image block to be encoded.

[0138] The inverse quantizer 304 and the inverse transformer 305 apply inverse quantization and inverse transformation respectively to reconstruct the residual block in the pixel domain, for example, to be used as a reference block of a reference image later. The summer 311 adds the reconstructed residual block to the predicted image block generated by the inter-frame predictor 310 or the intra-frame predictor 309 to generate a reconstructed image block. The predicted image block of an image block can be obtained by processing (such as interpolation) the reference image block of the image block.

[0139] It should be understood that other structural changes of the video encoder 102 can be used to encode the video stream. For example, for some image blocks or image frames, the video encoder 102 can directly quantize the residual signal without being processed by the transformer 301, and accordingly, it does not need to be processed by the inverse transformer 305; or, for some image blocks or image frames, the video encoder 102 does not generate residual data, and accordingly, it does not need to be processed by the transformer 301, the quantizer 302, the inverse quantizer 304 and the inverse transformer 305; or, the video encoder 102 can directly store the reconstructed image block as a reference block without being processed by the filter 306; or, the quantizer 302 and the inverse quantizer 304 in the video encoder 102 can be combined together.

[0140] Figure 3 2 is a schematic diagram of the structure of the video decoder 202 in the embodiment of the present application. Figure 3 As shown, the video decoder 202 includes an entropy decoder 401, an inverse quantizer 402, an inverse transformer 403, a filter 404, a memory 405, a prediction processing unit 406, and a summer 409. The prediction processing unit 406 includes an intra-frame predictor 407 and an inter-frame predictor 408. In some examples, the video decoder 202 may perform substantially the same Figure 2 The encoding process described by the video encoder 102 is the inverse of the decoding process.

[0141] During the decoding process, the video decoder 202 receives a bitstream from the video encoder 102. The video decoder 202 may receive video data from the network entity 42, and optionally, may store the video data in a video data memory (not shown). The video data memory may store video data to be decoded by the components of the video decoder 202, such as an encoded bitstream. The video data stored in the video data memory may be obtained, for example, from the storage device 40, from a local video source such as a camera, via a wired or wireless network communication of video data, or by accessing a physical data storage medium. Although in Figure 3 The video data memory is not shown in the figure, but the video data memory and the memory 405 can be the same memory or can be separately arranged memories. The video data memory and the memory 405 can be formed by any of a variety of memory devices, such as dynamic random access memory (DRAM) including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. In various examples, the video data memory can be integrated on the chip with other components of the video decoder 200, or can be arranged off the chip relative to those components.

[0142] The network entity 42 may be, for example, a server, a MANE, a video editor / splicer, or other such device for implementing one or more of the techniques described above. The network entity 42 may or may not include a video encoder, such as the video encoder 102. Before the network entity 42 sends the code stream to the video decoder 202, the network entity 42 may implement some of the techniques described in this application. In some video decoding systems, the network entity 42 and the video decoder 202 may be parts of separate devices, while in other cases, the functionality described with respect to the network entity 42 may be performed by the same device including the video decoder 202. In some cases, the network entity 42 may be Figure 1 An example of a storage device 40.

[0143] The entropy decoder 401 of the video decoder 202 performs entropy decoding on the bitstream to generate quantized coefficients and some syntax elements. The entropy decoder 401 forwards the syntax elements to the filter 404. The video decoder 202 may receive syntax elements at the video slice level and / or the image block level. In the present application, in an example, the syntax elements here may include indication information related to the current image block, and the indication information is used to indicate that the second motion information is determined according to the first motion information. In addition, in some instances, the video encoder 102 may signal a specific syntax element indicating whether the second motion information is determined according to the first motion information.

[0144] The inverse quantizer 402 inversely quantizes, i.e., dequantizes, the quantized transform coefficients provided in the bitstream and decoded by the entropy decoder 401. The inverse quantization process may include using the quantization parameters calculated by the video encoder 102 for each image block in the video slice to determine the degree of quantization to be applied and likewise the degree of inverse quantization to be applied. The inverse transformer 403 applies an inverse transform to the transform coefficients, such as an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process, to generate a residual block in the pixel domain.

[0145] After the prediction processing unit 406 generates a predicted image block for the current image block or a sub-block of the current image block, the video decoder 202 obtains a reconstructed block, i.e., a decoded image block, by summing the residual block from the inverse transformer 403 with the corresponding predicted image block generated by the prediction processing unit 406. The summer 409 (also referred to as the reconstructor 409) represents a component that performs this summing operation. If necessary, a filter (in the decoding loop or after the decoding loop) can also be used to smooth pixel transitions or otherwise improve video quality. The filter 404 can be one or more loop filters, such as a deblocking filter, an adaptive loop filter (ALF), and a sample adaptive offset (SAO) filter.

[0146] It should be understood that other structural changes of the video decoder 202 can be used for decoding the bitstream. For example, for some image blocks or image frames, the entropy decoder 401 of the video decoder 202 does not decode the quantized coefficients, and accordingly does not need to be processed by the dequantizer 402 and the inverse transformer 403. For example, the dequantizer 402 and the inverse transformer 403 in the video decoder 202 can be combined together.

[0147] Below, combined with the above Figure 1 The video coding system 1 shown, Figure 2 The video encoder 102 is shown as well as Figure 3 The video decoder 202 shown describes the bidirectional inter-frame prediction method provided by the present application in detail.

[0148] Figure 4 Schematic diagram of the flow of the bidirectional inter-frame prediction method in the embodiment of the present application. Figure 4 The method shown is performed by a bidirectional inter-frame prediction device. The bidirectional inter-frame prediction device may be Figure 1 The video decoder 202 in. Figure 4 The bidirectional inter-frame prediction device is taken as the video decoder 202 as an example for description.

[0149] like Figure 4 As shown, the bidirectional inter-frame prediction method in the embodiment of the present application may include the following steps:

[0150] S400: The video decoder 202 parses the acquired code stream and obtains indication information.

[0151] Optionally, the video decoder 202 parses the bitstream and determines the inter-frame prediction mode for inter-frame prediction of the current image block in the current frame according to the value of the syntax element in the bitstream. When the inter-frame prediction mode is a bidirectional inter-frame prediction mode, the video decoder 202 obtains the indication information.

[0152] The video decoder 202 may receive the encoded bitstream sent by the video encoder 102 , or may obtain the encoded bitstream from the storage device 40 .

[0153] Optionally, the video decoder 202 in the embodiment of the present application determines the inter-frame prediction mode for inter-frame prediction of the current image block in the current frame according to the value of the syntax element inter_pred_idc. From the above description, it can be seen that inter-frame prediction includes unidirectional inter-frame prediction and bidirectional inter-frame prediction. When the value of the syntax element inter_pred_idc is 0, the video decoder 202 determines that the inter-frame prediction mode for inter-frame prediction of the current image block in the current frame is forward inter-frame prediction. When the value of the syntax element inter_pred_idc is 1, the video decoder 202 determines that the inter-frame prediction mode for inter-frame prediction of the current image block in the current frame is backward inter-frame prediction. When the value of the syntax element inter_pred_idc is 2, the video decoder 202 determines that the inter-frame prediction mode for inter-frame prediction of the current image block in the current frame is bidirectional inter-frame prediction.

[0154] Optionally, after determining that the value of the syntax element inter_pred_idc is 2, the video decoder 202 obtains indication information for indicating that the second motion information is determined according to the first motion information. The first motion information is motion information of the current image block in a first direction, and the second motion information is motion information of the current image block in a second direction, and the first direction is different from the second direction.

[0155] The image blocks involved in the present application may be basic units for performing video encoding or video decoding, such as coding units (CU), or may be basic units for performing prediction operations, such as prediction units (PU). The embodiments of the present application do not specifically limit this.

[0156] If the image block is a basic unit for performing video encoding or video decoding, the current image block in the embodiment of the present application includes at least one sub-block. Accordingly, the first motion information includes motion information of each sub-block in the first direction of at least one sub-block of the current image block, and the second motion information includes motion information of each sub-block in the second direction of at least one sub-block of the current image block. The indication information can be used to indicate the determination of the motion information of a sub-block in the second direction based on the motion information of the sub-block in the first direction.

[0157] The video decoder 202 may obtain the indication information in various ways.

[0158] In a first implementation, the video decoder 202 parses the first identifier. When the value of the first identifier is the first preset value, the video decoder 202 determines to parse the first motion information, and determines the second motion information based on the first motion information, that is, the video decoder 202 obtains the indication information. When the value of the first identifier is the eighth preset value, the video decoder 202 parses the code stream to obtain the fifth identifier. When the value of the fifth identifier is the fifth preset value, the video decoder 202 determines to parse the second motion information, and calculates the first motion information based on the second motion information. When the value of the fifth identifier is the ninth preset value, the video decoder 202 obtains the first motion information and the second motion information. Among them, the first preset value and the fifth preset value may be the same or different, and the embodiment of the present application does not specifically limit this.

[0159] Exemplarily, the first identifier is mv_derived_flag_l0, the fifth identifier is mv_derived_flag_l1, the first preset value and the fifth preset value are both 1, and the eighth preset value and the ninth preset value are both 0. The video decoder 202 first parses mv_derived_flag_l0. When the value of mv_derived_flag_l0 is 1, the video decoder 202 parses the first motion information and determines the second motion information according to the first motion information. When the value of mv_derived_flag_l0 is 0, the video decoder 202 parses mv_derived_flag_l1. When the value of mv_derived_flag_l1 is 1, the video decoder 202 parses the second motion information and calculates the first motion information according to the second motion information. When the values ​​of mv_derived_flag_l0 and mv_derived_flag_l1 are both 0, the video decoder 202 parses the first motion information and the second motion information.

[0160] In a second implementation, the video decoder 202 parses the second identifier. When the value of the second identifier is the second preset value, the video decoder 202 determines to use the motion information derivation algorithm to calculate the motion information of the current image block. Then, the video decoder 202 parses the third identifier. When the value of the third identifier is the third preset value, the video decoder 202 determines to parse the first motion information, and determines the second motion information based on the first motion information, that is, the video decoder 202 obtains the indication information. When the value of the third identifier is the sixth preset value, the video decoder 202 determines to parse the second motion information, and calculates the first motion information based on the second motion information.

[0161] Exemplarily, the second identifier is derived_mv_flag, the third identifier is derived_mv_direction, the third preset value is 1, and the sixth preset value is 0. The video decoder 202 first parses derived_mv_flag. When the value of derived_mv_flag is 1, the video decoder 202 determines to use the motion information derivation algorithm to calculate the motion information of the current image block. When the value of derived_mv_flag is 0, the video decoder 202 parses the first motion information and the second motion information. When the value of derived_mv_direction is 1, the video decoder 202 parses the first motion information and determines the second motion information based on the first motion information. When the value of derived_mv_direction is 0, the video decoder 202 determines to parse the second motion information and calculates the first motion information based on the second motion information.

[0162] In a third implementation, the video decoder 202 parses the second identifier. When the value of the second identifier is the second preset value, the video decoder 202 determines to use the motion information derivation algorithm to calculate the motion information of the current image block. Then, the video decoder 202 determines to parse the first motion information according to the preset derivation direction, and determines the second motion information according to the first motion information, that is, the video decoder 202 obtains the indication information. That is, in this implementation, "determining the second motion information according to the first motion information" is pre-set. When the value of the second identifier is the seventh preset value, the first group of motion information and the second motion information are parsed.

[0163] Exemplarily, the second identifier is derived_mv_flag, the second preset value is 1, and the seventh preset value is 0. The video decoder 202 parses derived_mv_flag. When the value of derived_mv_flag is 1, the video decoder 202 determines to use the motion information derivation algorithm to calculate the motion information of the current image block. Further, the video decoder 202 determines to parse the first motion information, and determines the second motion information according to the first motion information. When the value of derived_mv_flag is 0, the video decoder 202 parses the first motion information and the second motion information.

[0164] In a fourth implementation, the video decoder 202 parses the fourth flag (e.g., mv_derived_flag_10). When the value of the fourth flag is the fourth preset value, the video decoder 202 determines to use the motion information derivation algorithm to calculate the motion information of the current image block, and calculates the variable derived_ref_num according to the first reference frame list and the second reference frame list. The variable represents the number of reference frame pairs that can form a mirror / linear reference frame of the first reference frame and the second reference frame list. When the number of reference frame pairs is 1, the video decoder 202 directly determines the index value of the reference frame. Then, the video decoder 202 determines to parse the first motion information according to the preset derivation direction, and determines the second motion information according to the first motion information, that is, the video decoder 202 obtains the indication information. The first reference frame list is a reference frame list of the current image block in the first direction, the second reference frame list is a reference frame list of the current image block in the second direction, the first reference frame is a reference frame of the current image block in the first direction, and the second reference frame is a reference frame of the current image block in the second direction. The index values ​​of the reference frames in the embodiments of the present application are all numbers of the reference frames in the corresponding reference frame lists.

[0165] Exemplarily, the sequence number of the current frame is 4, the first reference frame list is [2, 0], and the second reference frame list is {6, 7}. According to the above condition B or condition C, it is determined that the reference frame with the sequence number 2 in the first reference frame list and the reference frame with the sequence number 6 in the second reference frame list can form a reference frame pair. Therefore, the index values ​​of the first reference frame and the second reference frame are both 0.

[0166] If the sequence number of the current frame is 4, the first reference frame list is {2, 0}, and the second reference frame list is {6, 7}, according to the above condition B or condition C, it is determined that the reference frame with sequence number 2 in the first reference frame list and the reference frame with sequence number 6 in the second reference frame list can form a reference frame pair, and the reference frame with sequence number 0 in the first reference frame list and the reference frame with sequence number 8 in the second reference frame list can also form a reference frame pair. At this time, the video decoder 202 needs to parse the index value of the reference frame.

[0167] Further, the video decoder 202 may also determine whether the characteristic information of the current frame satisfies a preset condition when determining that the inter-frame prediction mode is a bidirectional inter-frame prediction mode. Thus, when the characteristic information of the current frame satisfies the preset condition, the video decoder 202 obtains the indication information. That is, the process of S401 may specifically be: when determining that the inter-frame prediction mode is a bidirectional inter-frame prediction mode and the characteristic information of the current frame satisfies the first preset condition, the video decoder 202 obtains the indication information.

[0168] The characteristic information of the current frame includes at least one of a sequence number, a temporal level ID (TID), and a reference frame number. The bitstream acquired by the video decoder 202 includes a sequence parameter set (SPS), a picture parameter set (PPS), a slice header or a slice segment header, and encoded image data. Afterwards, the video decoder 202 parses the bitstream to obtain the characteristic information of the current frame.

[0169] The above-mentioned preset conditions include at least one of the following conditions:

[0170] Condition A: There are at least two reference frames for the current image block.

[0171] Condition B, the sequence number of the current frame, the sequence number of the first reference frame, and the sequence number of the second reference frame satisfy the following formula:

[0172] POC_Cur-POC_listX=POC_listY-POC_Cur

[0173] Among them, POC_Cur represents the serial number of the current frame, POC_listX represents the serial number of the first reference frame, POC_listY represents the serial number of the second reference frame, the first reference frame is the reference frame of the current image block in the first direction, and the second reference frame is the reference frame of the current image block in the second direction.

[0174] Condition C, the sequence number of the current frame, the sequence number of the first reference frame, and the sequence number of the second reference frame satisfy the following formula:

[0175] (POC_Cur-POC_listX)*(POC_listY-POC_Cur)>0

[0176] Among them, POC_Cur represents the serial number of the current frame, POC_listX represents the serial number of the first reference frame, POC_listY represents the serial number of the second reference frame, the first reference frame is the reference frame of the current image block in the first direction, and the second reference frame is the reference frame of the current image block in the second direction.

[0177] Condition D: The TID of the current frame is greater than or equal to a preset value.

[0178] The preset conditions in the embodiments of the present application may be pre-set or may be specified in a high-level syntax, such as a parameter set such as SPS, PPS, slice header or slice segment header, and the embodiments of the present application do not specifically limit this.

[0179] Condition E: mvd_l1_zero_flag of the current frame is 0.

[0180] Among them, mvd_l1_zero_flag is a flag bit obtained by parsing the video decoder 202. mvd_l1_zero_flag is 1, indicating that the MVD of List1 does not need to be parsed. (mvd_l1_zero_flag equal to 1 indicates that themvd_coding(x0, y0, 1)syntax structure is not parsed and MvdL1[x0][y0][compIdx]is set equal to 0 for compIdx=0..1.mvd_l1_zero_flag equal to 0 indicatesthat the mvd_coding(x0, y0, 1)syntax structure is parsed.)

[0181] Specifically, for the above condition B (or condition C), the video decoder 202 obtains a reference frame serial number from the first reference frame list and the second reference frame list respectively, and determines whether the obtained reference frame serial number and the current frame serial number meet the above condition B or condition C. If the above condition B (or condition C) is met, the indication information is obtained.

[0182] Optionally, for the above condition A, the video decoder 202 sets the reference frame index value (SymRefIdxX) of the first direction to 0, and sets the reference frame index value (SymRefIdxY) of the second direction to 0.

[0183] Optionally, for the above condition B, the video decoder 202 searches for a reference frame POC_listX in the first reference frame list that is closest to the current frame and has a sequence number smaller than that of the current frame, and the video decoder 202 searches for a reference frame POC_listY in the second reference frame list that satisfies POC_Cur-POC_listX=POC_listY-POC_Cur. If the reference frames POC_listX and POC_listY that meet the requirements can be found according to the search method, the reference frame index value (SymRefIdxX) in the first direction is set to the index value of POC_listX in the first reference frame list, and the reference frame index value (SymRefIdxY) in the second direction is set to the index value of POC_listY in the second reference frame list. Optionally, if the reference frames POC_listX and POC_listY that meet the requirements cannot be found, then the reference frame POC_listX that is closest to the current frame and has a sequence number greater than that of the current frame is further searched, and the video decoder 202 searches for the reference frame POC_listY that satisfies POC_Cur-POC_listX=POC_listY-POC_Cur in the second reference frame list. If the reference frames POC_listX and POC_listY that meet the requirements can be found according to this search method, the reference frame index value (SymRefIdxX) of the first direction is set to the index value of POC_listX in the first reference frame list, and the reference frame index value (SymRefIdxY) of the second direction is set to the index value of POC_listY in the second reference frame list.

[0184] Optionally, for the above condition C, the video decoder 202 searches for the reference frame POC_listX in the first reference frame list that is closest to the current frame and has a sequence number smaller than the sequence number of the current frame, and the video decoder 202 searches for the reference frame POC_listY in the second reference frame list that is closest to the current frame and has a sequence number larger than the sequence number of the current frame. If the reference frames POC_listX and POC_listY that meet the requirements can be found according to this search method, the reference frame index value (SymRefIdxX) in the first direction is set to the index value of POC_listX in the first reference frame list, and the reference frame index value (SymRefIdxY) in the second direction is set to the index value of POC_listY in the second reference frame list. Optionally, if the reference frames POC_listX and POC_listY that meet the requirements cannot be found, then the reference frame POC_listX that is closest to the current frame and has a sequence number greater than the sequence number of the current frame is further searched, and the video decoder 202 searches for the reference frame POC_listY that is closest to the current frame and has a sequence number less than the sequence number of the current frame in the second reference frame list. If the reference frames POC_listX and POC_listY that meet the requirements can be found according to this search method, the reference frame index value (SymRefIdxX) of the first direction is set to the index value of POC_listX in the first reference frame list, and the reference frame index value (SymRefIdxY) of the second direction is set to the index value of POC_listY in the second reference frame list.

[0185] Optionally, the reference frame index value (SymRefIdxX) in the first direction and the reference frame index value (SymRefIdxY) in the second direction may also be determined according to the following method:

[0186] The video decoder 202 searches the first reference frame list for the reference frame POC_listX_0 that is closest to the current frame and has a sequence number smaller than the sequence number of the current frame, and the reference frame POC_listX_1 that is closest to the current frame and has a sequence number larger than the sequence number of the current frame. The video decoder 202 searches the second reference frame list for the reference frame POC_listY_0 that is closest to the current frame and has a sequence number larger than the sequence number of the current frame, and the reference frame POC_listY_1 that is closest to the current frame and has a sequence number smaller than the sequence number of the current frame.

[0187] If the reference frame combination POC_listX_0 and POC_listY_0 that meets the requirements can be found according to the search method, the reference frame index value (SymRefIdxX) of the first direction is set to the index value of POC_listX_0 in the first reference frame list, and the reference frame index value (SymRefIdxY) of the second direction is set to the index value of POC_listY_0 in the second reference frame list;

[0188] Otherwise, if the reference frame combination POC_listX_1 and POC_listY_1 that meets the requirements can be found according to this search method, the reference frame index value (SymRefIdxX) in the first direction is set to the index value of POC_listX_1 in the first reference frame list, and the reference frame index value (SymRefIdxY) in the second direction is set to the index value of POC_listY_1 in the second reference frame list.

[0189] Optionally, the reference frame index value (SymRefIdxX) in the first direction and the reference frame index value (SymRefIdxY) in the second direction may also be determined according to the following method:

[0190] The video decoder 202 searches the first reference frame list for the reference frame POC_listX_0 that is closest to the current frame and has a sequence number smaller than the sequence number of the current frame, and the reference frame POC_listX_1 that is closest to the current frame and has a sequence number larger than the sequence number of the current frame. The video decoder 202 searches the second reference frame list for the reference frame POC_listY_0 that is closest to the current frame and has a sequence number larger than the sequence number of the current frame, and the reference frame POC_listY_1 that is closest to the current frame and has a sequence number smaller than the sequence number of the current frame.

[0191] If only the reference frame combination POC_listX_0 and POC_listY_0 that meet the requirements are found according to the search method, the reference frame index value (SymRefIdxX) of the first direction is set to the index value of POC_listX_0 in the first reference frame list, and the reference frame index value (SymRefIdxY) of the second direction is set to the index value of POC_listY_0 in the second reference frame list;

[0192] If only the reference frame combination POC_listX_1 and POC_listY_1 that meet the requirements are found according to the search method, the reference frame index value (SymRefIdxX) of the first direction is set to the index value of POC_listX_1 in the first reference frame list, and the reference frame index value (SymRefIdxY) of the second direction is set to the index value of POC_listY_1 in the second reference frame list;

[0193] If the reference frame combination POC_listX_0 and POC_listY_0, as well as the reference frame combination POC_listX_1 and POC_listY_1 that meet the requirements are found at the same time according to this search method, the distance POC_DIS_0 (POC_listY_0-POC_listX_0) between the reference frame combination POC_listY_0 and POC_listX_0 and the distance POC_DIS_1 (POC_listX_1-POC_listY_1) between the reference frame combination POC_listX_1 and POC_listY_1 are further calculated. If POC_DIS_0 is less than or equal to POC_DIS_i, the reference frame index value (SymRefIdxX) of the first direction is set to the index value of POC_listX_0 in the first reference frame list, and the reference frame index value (SymRefIdxY) of the second direction is set to the index value of POC_listY_0 in the second reference frame list; otherwise, the reference frame index value (SymRefIdxX) of the first direction is set to the index value of POC_listX_1 in the first reference frame list, and the reference frame index value (SymRefIdxY) of the second direction is set to the index value of POC_listY_1 in the second reference frame list.

[0194] Optionally, the reference frame index value (SymRefIdxX) in the first direction and the reference frame index value (SymRefIdxY) in the second direction may also be determined according to the following method:

[0195] The video decoder 202 searches the first reference frame list for the reference frame POC_listX_0 that is closest to the current frame and has a sequence number smaller than the sequence number of the current frame, and the reference frame POC_listX_1 that is closest to the current frame and has a sequence number larger than the sequence number of the current frame. The video decoder 202 searches the second reference frame list for the reference frame POC_listY_0 that is closest to the current frame and has a sequence number larger than the sequence number of the current frame, and the reference frame POC_listY_1 that is closest to the current frame and has a sequence number smaller than the sequence number of the current frame.

[0196] If only the reference frame combination POC_listX_0 and POC_listY_0 that meet the requirements are found according to the search method, the reference frame index value (SymRefIdxX) of the first direction is set to the index value of POC_listX_0 in the first reference frame list, and the reference frame index value (SymRefIdxY) of the second direction is set to the index value of POC_listY_0 in the second reference frame list;

[0197] If only the reference frame combination POC_listX_1 and POC_listY_1 that meet the requirements are found according to the search method, the reference frame index value (SymRefIdxX) of the first direction is set to the index value of POC_listX_1 in the first reference frame list, and the reference frame index value (SymRefIdxY) of the second direction is set to the index value of POC_listY_1 in the second reference frame list;

[0198] If the reference frame combination POC_listX_0 and POC_listY_0, as well as the reference frame combination POC_listX_1 and POC_listY_1 that meet the requirements are found at the same time according to the search method, then the absolute distances POC_DIS_listX0, POC_DIS_listY0, POC_DIS_listX1, POC_DIS_listY1 between the four reference frames POC_listY_0, POC_listX_0, POC_listX_1, POC_listY_1 and the current frame POC_Cur are further calculated. The reference frame combination with the smallest reference frame among the four distances is selected as the optimal combination k, and the reference frame index value (SymRefIdxX) of the first direction is set to the index value of the optimal combination POC_listX_k in the first reference frame list, and the reference frame index value (SymRefIdxY) of the second direction is set to the index value of POC_listY_k in the second reference frame list.

[0199] Optionally, the reference frame index value (SymRefIdxX) in the first direction and the reference frame index value (SymRefIdxY) in the second direction may also be determined according to the following method:

[0200] The video decoder 202 searches for all reference frame combinations POC_listX_i and POC_listY_i that satisfy POC_Cur-POC_listX_i=POC_listY_i-POC_Cur in the first reference frame list and the second reference frame list, where i ranges from 1 to the maximum number of reference frame combinations that meet the condition, N. If the number of reference frame combinations found is 1, the reference frame index value (SymRefIdxX) of the first direction is directly set to the index value of POC_listX_1 in the first reference frame list, and the reference frame index value (SymRefIdxY) of the second direction is set to the index value of POC_listY_1 in the second reference frame list. If the number of reference frame combinations found is greater than 1, the absolute distance POC_DIS_i (abs(POC_listY_i--POC_listX_i)) between each reference frame combination POC_listY_i and POC_listX_i is further calculated, and the reference frame combination with the smallest distance is selected, and the reference frame index value (SymRefIdxX) in the first direction is set to the index value of POC_listX_i in the first reference frame list, and the reference frame index value (SymRefIdxY) in the second direction is set to the index value of POC_listY_i in the second reference frame list.

[0201] Optionally, the reference frames POC_listX and POC_listY found above are both short-term reference frames.

[0202] Optionally, the reference frames POC_listX and POC_listY found above are of the same reference frame type, for example, both are short-term reference frames, or both are long-term reference frames.

[0203] In an embodiment of the present application, the method of "when it is determined that the inter-frame prediction mode is a bidirectional inter-frame prediction mode and the characteristic information of the current frame meets the preset conditions, the video decoder 202 obtains indication information" is the same as the above-mentioned method of "when it is determined that the inter-frame prediction mode is a bidirectional inter-frame prediction mode, the video decoder 202 obtains indication information".

[0204] In combination with the above description, Table 1 is a syntax table of the video decoder 202 using the first implementation method to obtain indication information when the inter-frame prediction mode is determined to be a bidirectional inter-frame prediction mode and the characteristic information of the current frame meets the preset conditions. prediction_unit() is a syntax structure of a predicted image block, which describes a method for determining motion information of each sub-block in the current image block.

[0205] In Table 1, x0 and y0 respectively represent the horizontal coordinate offset and vertical coordinate offset of the sub-block in the current image block relative to the upper left vertex of the current image block, nPbW represents the width of the current image block, and nPbH represents the height of the current image block. When the value of inter_pred_idc[x0][y0] is PRED_L0, it indicates that the inter-frame prediction of the current sub-block is forward prediction. When the value of inter_pred_idc[x0][y0] is PRED_L1, it indicates that the inter-frame prediction of the current sub-block is backward prediction. When the value of inter_pred_idc[x0][y0] is PRED_BI, it indicates that the inter-frame prediction of the current sub-block is bidirectional prediction.

[0206] In the case of bidirectional inter-frame prediction (i.e., inter_pred_idc[x0][y0]==PRED_BI), if the preset conditions are met, mv_derived_flag_l0[x0][y0] is parsed. If the value of mv_derived_flag_l0 is not the first preset value, mv_derived_flag_l1[x0][y0] is parsed. When the value of mv_derived_flag_l0 is the first preset value or the value of mv_derived_flag_l1[x0][y0] is the fifth preset value, the motion information of the sub-block of the current image block is determined, that is, the index value ref_idx_l0[x0][y0] of the reference frame, the motion vector prediction value flag mvp_l0_flag[x0][y0] and the motion vector difference mvd_coding(x0, y0, 0) are determined.

[0207] Table 1

[0208]

[0209]

[0210] In combination with the above description, Table 2 is a syntax table for the video decoder 202 to obtain indication information using the third implementation method when the inter-frame prediction mode is determined to be a bidirectional inter-frame prediction mode and the characteristic information of the current frame meets the preset conditions.

[0211] In Table 2, in the case of bidirectional inter-frame prediction (i.e., inter_pred_idc[x0][y0]==PRED_BI), if the preset conditions are met, derived_mv_flag[x0][y0] is parsed. If the value of derived_mv_flag[x0][y0] is a second preset value, the motion information of the sub-block of the current image block is determined, that is, the index value ref_idx_lx[x0][y0] of the reference frame, the motion vector prediction value flag mvp_lx_flag[x0][y0], and the motion vector difference mvd_coding(x0, y0, x) are determined.

[0212] Table 2

[0213]

[0214]

[0215] In combination with the above description, Table 3 is a syntax table for the video decoder 202 to obtain indication information using the fourth implementation method when the inter-frame prediction mode is determined to be a bidirectional inter-frame prediction mode and the characteristic information of the current frame meets the first preset condition.

[0216] In Table 3, in the case of bidirectional inter-frame prediction (i.e., inter_pred_idc[x0][y0]==PRED_BI), if the preset conditions are met, derived_mv_flag[x0][y0] is parsed. If the value of derived_mv_flag[x0][y0] is the fourth preset value, derived_ref_num is determined, and when the value of derived_ref_num is greater than 1, the motion information of the sub-block of the current image block is determined, that is, the index value ref_idx_lx[x0][y0] of the reference frame, the motion vector prediction value flag mvp_lx_flag[x0][y0], and the motion vector difference mvd_coding(x0, y0, x) are determined.

[0217] Table 3

[0218]

[0219]

[0220] The above-mentioned first identifier, second identifier, third identifier, and fourth identifier can all be pre-set, or can be specified in a high-level syntax, such as: SPS, PPS, slice header or slice segment header and other parameter sets, and the embodiments of the present application do not make specific limitations on this.

[0221] When the video decoder 202 determines that the inter-frame prediction mode is the bidirectional inter-frame prediction mode and the characteristic information of the current frame meets the preset conditions, it obtains the indication information, thereby effectively improving the decoding rate of the video decoder 202 and reducing information redundancy.

[0222] S401 : The video decoder 202 obtains first motion information.

[0223] Optionally, the video decoder 202 parses the bitstream to obtain the index value of the first reference frame, the first motion vector prediction value flag, and the first motion vector difference, that is, to obtain the first motion information. The first motion vector prediction value flag is used to indicate the index value of the first prediction motion vector in the first candidate prediction motion vector list, the first prediction motion vector is the prediction motion vector of the current image block in the first direction, the first motion vector difference is the difference between the first prediction motion vector and the first motion vector, and the first motion vector is the motion vector of the current image block in the first direction.

[0224] Optionally, generally speaking, the reference frame at the first position in the reference frame list is the frame closest to the current frame, so the index value ref_lX_idx of the first reference frame can be directly set to 0.

[0225] Optionally, if the video decoder 202 decides whether to obtain the indication information according to the preset condition B or C, the index value ref_lX_idx of the first reference frame may be set to a predetermined value SymRefIdxX.

[0226] Optionally, a first candidate prediction motion vector list is constructed according to the index value of the first reference frame. Specifically, in the process of constructing the first candidate prediction motion vector list, the motion vector of the adjacent decoded image block of the current image block is obtained. According to the index value of the first reference frame and the first reference frame list, the serial number of the first reference frame is determined. If the serial number of the reference frame according to the adjacent decoded image block is different from the serial number of the first reference frame, the motion vector of the adjacent decoded image block needs to be scaled to point to the first reference frame and then added to the first candidate prediction motion vector list.

[0227] In the syntax tables shown in Tables 1 to 3 above, the video decoder 202 determines the motion information of the sub-blocks of the current image block in the first direction.

[0228] S402: The video decoder 202 determines second motion information according to the first motion information.

[0229] In a first implementation, the method for the video decoder 202 to determine the second motion information is as follows: the video decoder 202 selects the index value of the first reference frame from the first motion information, and determines the serial number of the first reference frame according to the index value of the first reference frame and the first reference frame list; calculates the serial number of the second reference frame according to the serial number of the current frame and the serial number of the first reference frame by a preset formula; determines the index value of the second reference frame according to the serial number of the second reference frame and the second reference frame list; determines the second motion information according to the first motion information and the index of the second reference frame.

[0230] Here, the preset formula may be POC_listY=2*POC_Cur-POC_listX, wherein POC_Cur represents the sequence number of the current frame, POC_listX represents the sequence number of the first reference frame, and POC_listY represents the sequence number of the second reference frame.

[0231] For example, if the serial number of the current frame is 4, the serial number of the first reference frame is 2, and the second reference frame list is [6, 8], according to the formula POC_listY=2*POC_Cur-POC_listX, the serial number of the second reference frame is determined to be 6, then the video decoder 202 determines that the index value ref_lY_idx of the second reference frame is 0.

[0232] Optionally, the preset formula may also be (POC_Cur-POC_listX)*(POC_listY-POC_Cur)>0. It should be noted that if there are multiple reference frames in the second reference frame list whose serial numbers satisfy the formula, the video decoder 202 first selects the reference frame with the smallest abs((POC_listY-POC_Cur)-(POC_Cur-POC_listX)), and then selects the reference frame with the smallest abs(POC_listY-POC_Cur) to determine the index value of the second reference frame. Wherein, abs is an absolute value function.

[0233] For example, if the sequence number of the current frame is 4, the sequence number of the first reference frame is 2, and the second reference frame list is [5, 7, 8], according to the formula (POC_Cur-POC_listX)*(POC_listY-POC_Cur)>0, the sequence number of the second reference frame is determined to be 5, then the video decoder 202 determines that the index value ref_lY_idx of the second reference frame is 0.

[0234] Optionally, the preset formula may also be POC_listX≠POC_listY. It should be noted that if there are multiple reference frames in the second reference frame list whose serial numbers satisfy the formula, the video decoder 202 first selects the reference frame with the smallest abs((POC_listY-POC_Cur)-(POC_Cur-POC_listX)), and then selects the reference frame with the smallest abs(POC_listY-POC_Cur) to determine the index value of the second reference frame. Wherein, abs is an absolute value function.

[0235] For example, if the sequence number of the current frame is 4, the sequence number of the first reference frame is 2, and the second reference frame list is [3, 2, 1, 0], according to the formula POC_listX≠POC_listY, the sequence number of the second reference frame is determined to be 3, then the video decoder 202 determines that the index value ref_lY_idx of the second reference frame is 0.

[0236] Optionally, the preset formula may also be POC_listY0=2*POC_Cur-POC_listX, (POC_Cur-POC_listX)*(POC_listYO′-POC_Cur)>0, and POC_listX≠POC_listY0″. In this case, the method for the video decoder 202 to determine the index value of the second reference frame is specifically: according to the sequence number of the current frame and the sequence number of the first reference frame, the first sequence number is calculated by the formula POC_listY0=2*POC_Cur-POC_listX, wherein POC_Cur represents the sequence number of the current frame, POC_listX represents the sequence number of the first reference frame, and POC_listY0 represents the first sequence number; when the second reference frame list includes the first sequence number, the reference frame represented by the first sequence number in the second reference frame list is numbered as the index value of the second reference frame; when the second reference frame list does not include the first sequence number, the second sequence number is calculated according to the sequence number of the current frame and the sequence number of the first reference frame by (POC_Cur-POC_listX)*(POC_listY0′-POC_Cur)>0, wherein POC_listY0′ represents the second sequence number; when the second reference frame list includes the second sequence number, the number of the reference frame represented by the second sequence number in the second reference frame list is determined as the index value of the second reference frame; when the second reference frame list does not include the second sequence number, the third sequence number is calculated according to the sequence number of the current frame and the sequence number of the first reference frame by the formula POC_listX≠POC_listY0″, wherein POC_listY0″ represents the third sequence number; the number of the reference frame represented by the third sequence number in the second reference frame list is determined as the index value of the second reference frame.

[0237] Optionally, generally speaking, the reference frame at the first position in the reference frame list is the frame closest to the current frame, so the index value ref_lY_idx of the second reference frame can be directly set to 0, and setting it to 0 indicates pointing to the first position.

[0238] Optionally, if the video decoder 202 decides whether to obtain the indication information according to the preset condition B or C, the index value ref_lY_idx of the second reference frame may be set to a predetermined value SymRefIdxY.

[0239] In a second implementation, the method for the video decoder 202 to determine the second motion information is as follows: the video decoder 202 parses the bitstream, obtains the index value of the second reference frame, and determines the second motion information according to the first motion information and the index value of the second reference frame. The index value of the second reference frame may also be predefined, or specified in a parameter set such as an SPS, a PPS, a slice header, or a slice segment header, and the embodiment of the present application does not specifically limit this.

[0240] It can be seen that in both the first implementation and the second implementation, the video decoder 202 determines the second motion information according to the first motion information and the index value of the second reference frame.

[0241] Optionally, the video decoder 202 may calculate all motion information of the current image block in the second direction, or may calculate partial motion information of the current image block in the second direction.

[0242] The process of the video decoder 202 determining the second motion information according to the first motion information and the index value of the second reference frame is now described.

[0243] Optionally, the method of "the video decoder 202 determines the second motion information according to the first motion information and the index value of the second reference frame" may be: obtaining the index value of the first reference frame in the first motion information, and determining the sequence number of the first reference frame according to the index value of the first reference frame and the first reference frame list; obtaining the index value of the second reference frame, and determining the sequence number of the second reference frame according to the index value of the second reference frame and the second reference frame list; determining the first motion vector (the motion vector of the current image block in the first direction) according to the first motion vector difference and the first motion vector prediction value flag in the first motion information; and determining the second motion vector in the second motion information according to the following formula:

[0244]

[0245] Among them, mv_lY represents the second motion vector, POC_Cur represents the sequence number of the current frame, POC_listX represents the sequence number of the first reference frame, POC_listY represents the sequence number of the second reference frame, mv_lX represents the first motion vector, and the second motion vector is the motion vector of the current image block in the second direction.

[0246] Among them, the video decoder 202 constructs a candidate motion information list in the same way as the encoding end constructs the candidate motion information list in the above-mentioned AMVP mode or merge mode, and determines the first predicted motion vector in the candidate motion information list according to the first motion vector prediction flag. In this way, the video decoder 202 can determine the sum of the first motion prediction vector and the difference between the first motion vector and the first motion vector as the first motion vector.

[0247] Optionally, when the first reference frame is the forward reference frame of the current image block and the second reference frame is the backward reference frame of the current image block, or when the first reference frame is the backward reference frame of the current image block and the second reference frame is the forward reference frame of the current image block, or when the first reference frame and the second reference frame are both the forward reference frames of the current image block, or when the first reference frame and the second reference frame are both the backward reference frames of the current image block, the video decoder 202 may directly set mv_lY=-mv_lX.

[0248] Optionally, the video decoder 202 may not make any judgment and directly set mv_lY=-mv_lX.

[0249] Exemplarily, "the first reference frame is the forward reference frame of the current image block, and the second reference frame is the backward reference frame of the current image block", or "the first reference frame is the backward reference frame of the current image block, and the second reference frame is the forward reference frame of the current image block" can be expressed by the formula (POC_Cur-POC_listX)*(POC_listY-POC_Cur)>0, or by the formula POC_listY=2*POC_Cur-POC_listX.

[0250] “The case where the first reference frame is the current image block and the second reference frame are both forward reference frames of the current image block” or “the case where the first reference frame is the current image block and the second reference frame are both backward reference frames of the current image block” can be expressed by the formula (POC_Cur-POC_listX)*(POC_listY-POC_Cur)<0.

[0251] Optionally, the method of "the video decoder 202 determines the second motion information according to the first motion information and the index value of the second reference frame" may be: obtaining the index value of the first reference frame in the first motion information and the first motion vector difference, and determining the sequence number of the first reference frame according to the index value of the first reference frame and the first reference frame list; obtaining the index value of the second reference frame, and determining the sequence number of the second reference frame according to the index value of the second reference frame and the second reference frame list, determining the second predicted motion vector according to the index value of the second reference frame and the second candidate predicted motion vector list, the second predicted motion vector being the predicted motion vector of the current image block in the second direction; and determining the second motion vector difference in the second motion information according to the following formula:

[0252]

[0253] Among them, mvd_lY represents the second motion vector difference, POC_Cur represents the sequence number of the current frame, POC_listX represents the sequence number of the first reference frame, POC_listY represents the sequence number of the second reference frame, and mvd_lX represents the first motion vector difference; according to the second predicted motion vector and the second motion vector difference, the second motion vector is determined, and the second motion vector is the motion vector of the current image block in the second direction.

[0254] Optionally, a second candidate prediction motion vector list is constructed according to the index value of the second reference frame. Specifically, in the process of constructing the second candidate prediction motion vector list, the motion vector of the adjacent decoded image block of the current image block is obtained. According to the index value of the second reference frame and the second reference frame list, the serial number of the second reference frame is determined. If the serial number of the reference frame according to the adjacent decoded image block is different from the serial number of the second reference frame, the motion vector of the adjacent decoded image block needs to be scaled to point to the second reference frame and then added to the second candidate prediction motion vector list.

[0255] Optionally, when the first reference frame is a forward reference frame of the current image block and the second reference frame is a backward reference frame of the current image block, or when the first reference frame is a backward reference frame of the current image block and the second reference frame is a forward reference frame of the current image block, or when both the first reference frame and the second reference frame are forward reference frames of the current image block, or when both the first reference frame and the second reference frame are backward reference frames of the current image block, the video decoder 202 may directly set mvd_lY=-mvd_lX. Exemplarily, if (POC_Cur-POC_listX)*(POC_listY-POC_Cur)>0, POC_listY=2*POC_Cur-POC_listX or (POC_Cur-POC_listX)*(POC_listY-POC_Cur)<0, the video decoder 202 directly sets mvd_lY=-mvd_lX.

[0256] Optionally, the video decoder 202 may not make any judgment, and directly set mvd_lY=-mvd_lX. S403: The video decoder 202 determines the predicted pixel of the current image block according to the first motion information and the second motion information.

[0257] Optionally, the video decoder 202 determines the first motion vector and the second motion vector in S402, so that the video decoder 202 can determine the first reference image block according to the first motion vector, the first reference frame index value and the first reference frame list, and determine the second reference image block according to the second motion vector, the second reference frame index value and the second reference frame list, and then determine the predicted pixels of the current image block according to the first reference image block and the second reference image block, that is, the video decoder 202 completes the motion compensation process.

[0258] The method by which the video decoder 202 determines the predicted pixels of the current image block according to the first reference image block and the second reference image block may refer to any existing method, and the embodiment of the present application does not specifically limit this.

[0259] In the bidirectional inter-frame prediction method provided in the embodiment of the present application, the video decoder 202 can only obtain the first motion information from the encoded bitstream. After obtaining the first motion information, the video encoder 202 calculates the second motion information based on the first motion information, and further determines the predicted pixels of the current image block based on the first motion information and the second motion information. Compared with the prior art, the method provided by the present application does not need to transmit the motion information of each image block in all directions, effectively reducing the amount of motion information transmission, improving the effective utilization of transmission resources, improving the transmission rate, and improving the coding and decoding compression efficiency.

[0260] Figure 4The bidirectional inter-frame prediction method shown is described for the current image block, that is, it can be understood that the current image block performs inter-frame prediction based on the AMVP mode.

[0261] It is easy to understand that the bidirectional inter-frame prediction method provided by the present application is also applicable to non-translational motion model prediction modes, such as 4-parameter affine transformation motion model, 6-parameter affine transformation motion model, 8-parameter bilinear motion model, etc. In this scenario, the current image block includes at least one sub-block, and the motion information of the current image block includes the motion information of each sub-block in all sub-blocks of the current image block. The video decoder 202 determines the motion information of each sub-block (motion information in the first direction and motion information in the second direction) similarly to the method in which the video decoder 202 determines the motion information of the current image block.

[0262] For the non-translational motion model prediction mode, the video decoder 202 calculates the motion vector of the ith control point in the second direction using the following formula according to the motion vector of the ith control point in the first direction:

[0263]

[0264] In this formula, mvi_lY represents the motion vector of the i-th control point in the second direction, mvi_lX represents the motion vector of the i-th control point in the first direction, POC_Cur represents the sequence number of the current frame, POC_listY represents the sequence number of the second reference frame, and POC_listX represents the sequence number of the first reference frame.

[0265] Accordingly, the video decoder 202 calculates the motion vector difference of the ith control point in the second direction using the following formula according to the motion vector difference of the ith control point in the first direction:

[0266]

[0267] In this formula, mvdi_lY represents the motion vector difference of the i-th control point in the second direction, mvdi_lX represents the motion vector difference of the i-th control point in the first direction, POC_Cur represents the sequence number of the current frame, POC_listY represents the sequence number of the second reference frame, and POC_listX represents the sequence number of the first reference frame.

[0268] Compared with the video decoder 202, the video encoder 102 in the embodiment of the present application performs bidirectional motion estimation on the current image block to determine the motion information of the current image block in the first direction, and calculates the motion information of the current image block in the second direction based on the motion information of the current image block in the first direction. In this way, the video encoder 102 determines the predicted image block of the current image block based on the motion information of the current image block in the first direction and the motion information of the current image block in the second direction. Afterwards, the video encoder 102 performs operations such as transformation and quantization on the residual between the current image block and its predicted image block, generates a code stream and sends the code stream to the video decoder 202. The code stream includes the motion information of the current image block in the first direction.

[0269] The method of "the video encoder 102 calculates the motion information of the current image block in the second direction based on the motion information of the current image block in the first direction" can refer to the above-mentioned "the video decoder 202 determines the second motion information based on the first motion information" method, that is, refer to the description of S402 above, and this application will not go into details.

[0270] In summary, for bidirectional inter-frame prediction, the bidirectional inter-frame prediction method provided by the present application does not need to transmit the motion information of each image block in all directions, but only needs to transmit the motion information in a certain direction, which effectively reduces the amount of motion information transmission, improves the effective utilization of transmission resources, increases the transmission rate, and improves the encoding and decoding compression efficiency.

[0271] The embodiment of the present application provides a bidirectional inter-frame prediction device, which can be a video decoder. Specifically, the bidirectional inter-frame prediction device is used to perform the steps performed by the video decoder 202 in the above bidirectional inter-frame prediction method. The bidirectional inter-frame prediction device provided by the embodiment of the present application may include modules corresponding to the corresponding steps.

[0272] The embodiment of the present application can divide the functional modules of the bidirectional inter-frame prediction device according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules. The division of modules in the embodiment of the present application is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0273] In the case of dividing each functional module into corresponding functional modules, Figure 5 A possible structural diagram of the bidirectional inter-frame prediction device involved in the above embodiment is shown. Figure 5 As shown, the bidirectional inter-frame prediction device 5 includes an acquisition unit 50 and a determination unit 51.

[0274] The acquisition unit 50 is used to support the bidirectional inter-frame prediction apparatus to execute S400, S401, etc. in the above embodiments, and / or other processes used in the technology described herein.

[0275] The determination unit 51 is used to support the bidirectional inter-frame prediction apparatus to execute S402, S403, etc. in the above embodiments, and / or other processes used in the technology described herein.

[0276] Among them, all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, and will not be repeated here.

[0277] Of course, the bidirectional inter-frame prediction device provided in the embodiment of the present application includes but is not limited to the above modules. For example, the bidirectional inter-frame prediction device may also include a storage unit 52.

[0278] The storage unit 52 may be used to store program codes and data of the bidirectional inter-frame prediction apparatus.

[0279] In the case of using an integrated unit, the structural diagram of the bidirectional inter-frame prediction device provided in the embodiment of the present application is as follows: Figure 6 As shown. Figure 6 In the embodiment, the bidirectional inter-frame prediction device 6 includes: a processing module 60 and a communication module 61. The processing module 60 is used to control and manage the actions of the bidirectional inter-frame prediction device, for example, to execute the steps executed by the acquisition unit 50 and the determination unit 51, and / or to execute other processes of the technology described herein. The communication module 61 is used to support the interaction between the bidirectional inter-frame prediction device and other devices. Figure 6 As shown, the bidirectional inter-frame prediction apparatus may further include a storage module 62 , and the storage module 62 is used to store program codes and data of the bidirectional inter-frame prediction apparatus, such as storing the content stored in the above-mentioned storage unit 52 .

[0280] Among them, the processing module 60 can be a processor or a controller, for example, a central processing unit (CPU), a general processor, a digital signal processor (DSP), an ASIC, an FPGA or other programmable logic device, a transistor logic device, a hardware component or any combination thereof. It can implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements a computing function, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like. The communication module 61 can be a transceiver, an RF circuit or a communication interface, and the like. The storage module 62 can be a memory.

[0281] Among them, all relevant contents of each scenario involved in the above method embodiment can be referred to the functional description of the corresponding functional module, which will not be repeated here.

[0282] Both the bidirectional inter-frame prediction device 5 and the bidirectional inter-frame prediction device 6 can perform the above Figure 4 The bidirectional inter-frame prediction method shown, the bidirectional inter-frame prediction device 5 and the bidirectional inter-frame prediction device 6 can be specifically a video decoding device or other equipment with video encoding and decoding functions. The bidirectional inter-frame prediction device 5 and the bidirectional inter-frame prediction device 6 can be used to perform image prediction during the decoding process.

[0283] The present application also provides a terminal, which includes: one or more processors, a memory, and a communication interface. The memory and the communication interface are coupled to the one or more processors; the memory is used to store computer program codes, and the computer program codes include instructions. When the one or more processors execute the instructions, the terminal executes the bidirectional inter-frame prediction method of the embodiment of the present application.

[0284] The terminal here can be a video display device, a smart phone, a portable computer, or other device that can process or play video.

[0285] The present application also provides a video decoder, including a non-volatile storage medium and a central processing unit, wherein the non-volatile storage medium stores an executable program, the central processing unit is connected to the non-volatile storage medium, and executes the executable program to implement the bidirectional inter-frame prediction method of the embodiment of the present application.

[0286] The present application also provides a decoder, which includes a bidirectional inter-frame prediction device (bidirectional inter-frame prediction device 5 or bidirectional inter-frame prediction device 6) in an embodiment of the present application and a reconstruction module, wherein the reconstruction module is used to determine the reconstructed pixel value of the current image block based on the predicted image pixels obtained by the bidirectional inter-frame prediction device.

[0287] Another embodiment of the present application further provides a computer-readable storage medium, the computer-readable storage medium includes one or more program codes, the one or more programs include instructions, when the processor in the terminal executes the program code, the terminal executes as follows Figure 4 The bidirectional inter-frame prediction method shown.

[0288] In another embodiment of the present application, a computer program product is further provided, the computer program product comprising computer execution instructions, the computer execution instructions being stored in a computer-readable storage medium; at least one processor of the terminal can read the computer execution instructions from the computer-readable storage medium, and at least one processor executes the computer execution instructions so that the terminal implements the execution Figure 4The steps of the video decoder 202 in the bidirectional inter-frame prediction method are shown.

[0289] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by a software program, all or part of the embodiments may appear in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated.

[0290] The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0291] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0292] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0293] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0294] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0295] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program code.

[0296] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A reference frame acquisition method for bidirectional inter-frame prediction, characterized in that include: Determine a first reference frame list and a second reference frame list for a current image block; Determine whether there is an X-direction reference frame x1 satisfying condition A1 from the first reference frame list; Determine whether there is a Y-direction reference frame y1 satisfying condition B1 from the second reference frame list; In response to the determined existence of the X-direction reference frame x1 and the determined existence of the Y-direction reference frame y1, acquiring a bidirectional reference frame index value (R1, R2) of the current image block; The condition A1 is the reference frame POC_listX_0 that is closest to the current frame and has a sequence number smaller than that of the current frame, wherein the current frame is the frame where the current image block is located; the condition B1 is the reference frame POC_listY_0 that is closest to the current frame and has a sequence number larger than that of the current frame.

2. The method according to claim 1, characterized in that: In response to determining that the X-direction reference frame x1 does not exist and determining that the Y-direction reference frame y1 exists, the method further includes: Determine whether there is an X-direction reference frame x2 satisfying condition A2 from the first reference frame list; Determine whether there is a Y-direction reference frame y2 satisfying condition B2 from the second reference frame list; In response to determining that the X-direction reference frame x2 exists and determining that the Y-direction reference frame y2 exists, a bidirectional reference frame index value (R1, R2) of the current image block is obtained.

3. The method according to claim 2, characterized in that: The condition A2 is the reference frame POC_listX_1 that is closest to the current frame and has a sequence number greater than that of the current frame; the condition B2 is the reference frame POC_listY_1 that is closest to the current frame and has a sequence number less than that of the current frame.

4. The method according to claim 1, characterized in that: In response to determining the existence of the X-direction reference frame x1 and determining the existence of the Y-direction reference frame y1, the method further includes: Determine whether there is an X-direction reference frame x2 satisfying condition A2 from the first reference frame list; Determine whether there is a Y-direction reference frame y2 satisfying condition B2 from the second reference frame list; In response to determining the existence of the X-direction reference frame x1 and determining the existence of the Y-direction reference frame y1, determining the non-existence of the X-direction reference frame x2 and determining the existence of the Y-direction reference frame y2, obtaining a bidirectional reference frame index value (R1, R2) of the current image block, wherein the bidirectional reference frame index value (R1, R2) is respectively set to the index value of the X-direction reference frame x1 in the first reference frame list and the index value of the Y-direction reference frame y1 in the second reference frame list.

5. The method according to claim 4, characterized in that: The condition A2 is the reference frame POC_listX_1 that is closest to the current frame and has a sequence number greater than that of the current frame; the condition B2 is the reference frame POC_listY_1 that is closest to the current frame and has a sequence number less than that of the current frame.

6. The method according to claim 4, characterized in that: In response to determining that the X-direction reference frame x1 exists and the Y-direction reference frame y1 exists, and determining that the X-direction reference frame x2 exists and the Y-direction reference frame y2 exists, obtaining a bidirectional reference frame index value (R1, R2) of the current image block, the method further includes: According to a preset condition C, a bidirectional reference frame index value (R1, R2) of the current image block is obtained based on the determined existence of the X-direction reference frame x1 and the determined existence of the Y-direction reference frame y1, or the determined existence of the X-direction reference frame x2 and the determined existence of the Y-direction reference frame y2.

7. The method according to claim 6, characterized in that: The condition C is that the X-direction reference frame x1 and the Y-direction reference frame y1, and the X-direction reference frame x2 and the Y-direction reference frame y2 with the smallest distance (x1-y1, x2-y2) are used as the basis for obtaining the bidirectional reference frame index value (R1, R2) of the current image block.

8. The method according to claim 6, characterized in that: The condition C is that the reference frame group in which the X-direction reference frame x1 and the Y-direction reference frame y1, and the reference frame x2 and the Y-direction reference frame y2 with the shortest distance from the current frame are located is used as a basis for obtaining the bidirectional reference frame index value (R1, R2) of the current image block, wherein the reference frame group includes a reference frame group (x1, y1), and a reference frame group (x2, y2).

9. The method according to claim 1, characterized in that: The method further comprises: Determine whether there is an X-direction reference frame x2 satisfying condition A2 from the first reference frame list; Determine whether there is a Y-direction reference frame y2 satisfying condition B2 from the second reference frame list; The determining of the existing X-direction reference frame x1 and the determining of the existing Y-direction reference frame y1 to obtain the bidirectional reference frame index value (R1, R2) of the current image block includes: In response to a reference frame pair consisting of the determined existing X-direction reference frames x1 and x2 and the determined existing Y-direction reference frames y1 and y2, a bidirectional reference frame index value (R1, R2) of the current image block is obtained, wherein the reference frame pair includes, when x1 and y1 exist at the same time, a first reference frame pair (x1, y1) consisting of the X-direction reference frame x1 and the Y-direction reference frame y1, and when x2 and y2 exist at the same time, a second reference frame pair (x2, y2) consisting of the X-direction reference frame x2 and the Y-direction reference frame y2.

10. The method according to claim 9, characterized in that: The condition A2 is the reference frame POC_listX_1 that is closest to the current frame and has a sequence number greater than that of the current frame; the condition B2 is the reference frame POC_listY_1 that is closest to the current frame and has a sequence number less than that of the current frame.

11. The method according to claim 9, characterized in that: In response to the first reference frame pair existing and the second reference frame pair not existing, the obtaining the bidirectional reference frame index value (R1, R2) of the current image block includes: The bidirectional reference frame index values ​​(R1, R2) are respectively set to the index value of the X-direction reference frame x1 in the first reference frame list and the index value of the Y-direction reference frame y1 in the second reference frame list.

12. The method according to claim 9, characterized in that: In response to the second reference frame pair existing and the first reference frame pair not existing, the obtaining the bidirectional reference frame index value (R1, R2) of the current image block includes: The bidirectional reference frame index values ​​(R1, R2) are respectively set to the index value of the X-direction reference frame x2 in the first reference frame list and the index value of the Y-direction reference frame y2 in the second reference frame list.

13. The method according to claim 9, characterized in that: In response to the first reference frame pair existing and the second reference frame also existing, the method further includes: According to a preset condition C, a bidirectional reference frame index value (R1, R2) of the current image block is obtained based on the determined existence of the X-direction reference frame x1 and the determined existence of the Y-direction reference frame y1, or the determined existence of the X-direction reference frame x2 and the determined existence of the Y-direction reference frame y2.

14. The method according to claim 13, characterized in that: The condition C is that the X-direction reference frame x1 and the Y-direction reference frame y1, and the X-direction reference frame x2 and the Y-direction reference frame y2 with the smallest distance (x1-y1, x2-y2) are used as the basis for obtaining the bidirectional reference frame index value (R1, R2) of the current image block.

15. The method according to claim 13, characterized in that: The condition C is that the reference frame group in which the X-direction reference frame x1 and the Y-direction reference frame y1, and the reference frame x2 and the Y-direction reference frame y2 with the shortest distance from the current frame are located is used as a basis for obtaining the bidirectional reference frame index value (R1, R2) of the current image block, wherein the reference frame group includes a reference frame group (x1, y1), and a reference frame group (x2, y2).

16. A terminal, characterized in that: The terminal includes: one or more processors, a memory and a communication interface; The memory and the communication interface are connected to the one or more processors; the terminal communicates with other devices through the communication interface, the memory is used to store computer program code, and the computer program code includes instructions. When the one or more processors execute the instructions, the terminal executes the reference frame acquisition method applied to bidirectional inter-frame prediction as described in any one of claims 1-15.

17. A computer program product comprising instructions, characterized in that When the computer program product is run on a terminal, the terminal is enabled to execute the reference frame acquisition method applied to bidirectional inter-frame prediction as claimed in any one of claims 1 to 15.

18. A computer-readable storage medium comprising instructions, characterized in that: When the instruction is executed on a terminal, the terminal executes the reference frame acquisition method applied to bidirectional inter-frame prediction as claimed in any one of claims 1 to 15.

19. A video codec, comprising a non-volatile storage medium and a central processing unit, characterized in that: The non-volatile storage medium stores an executable program, and the central processing unit is connected to the non-volatile storage medium. When the central processing unit executes the executable program, the video decoder executes the reference frame acquisition method applied to bidirectional inter-frame prediction as described in any one of claims 1-15.

20. A reference frame acquisition device used in bidirectional inter-frame prediction, characterized in that include: One or more modules independently or in combination have the functions of implementing the method steps of claims 1 to 15 above.

Images