Method, apparatus and computer-readable storage medium for determining reference frames

Abstract

The present application provides a method, device, electronic device and computer-readable storage medium for determining a reference frame, including: determining the frame type of the image frame where the current block is located; obtaining the identifier of the reference frame of at least one adjacent block of the current block; using each single reference frame prediction mode to predict the current block respectively, and obtaining the rate-distortion cost corresponding to each single reference frame prediction mode; based on the frame type, the identifier of the reference frame of each adjacent block and the rate-distortion cost corresponding to each single reference frame prediction mode, obtaining the target reference frame template corresponding to each combined reference frame prediction mode; based on the target reference frame template corresponding to each combined reference frame prediction mode, determining the corresponding target combined reference frame when predicting the current block using each combined reference frame prediction mode. This scheme reduces the number of prediction combinations involved in the prediction in the process of selecting the optimal combined reference frame prediction combination, saves calculation time and improves the encoding speed.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and more specifically, to a method, device, and computer-readable storage medium for determining a reference frame, a method, a device, an electronic device, and a computer-readable storage medium. Background Art

[0002] In the video encoding process, a frame of image is sent to the encoder. First, it is divided into coding tree units (Coding Tree Uint, CTU) according to a certain size, and then the coding unit (Coding Uint, CU) is obtained through depth division. Each CU contains multiple prediction modes and transform units (TransformUnit, TU). Then, each CU is predicted to obtain a prediction value, which is subtracted from the input data to obtain a residual, which is then transformed and quantized to obtain a residual coefficient, which is then sent to the entropy coding module to output the bitstream. At the same time, the residual coefficient is dequantized and detransformed to obtain the residual value of the reconstructed image, which is then added to the prediction value to obtain the reconstructed image. After filtering, the reconstructed image enters the reference frame queue as the reference image of the next frame, and is then encoded backward in sequence.

[0003] Each CU corresponds to two prediction types: intra-frame prediction and inter-frame prediction. During the prediction process, different prediction modes within the same prediction type are first compared to determine the optimal intra-frame prediction mode and the optimal inter-frame prediction mode. Then, the optimal intra-frame prediction mode and the optimal inter-frame prediction mode are compared to determine the optimal prediction mode for the current CU.

[0004] For various video coding standards, the inter-frame prediction type corresponds to multiple prediction modes, each prediction mode corresponds to multiple reference frames, and each prediction mode and a corresponding reference frame form a prediction combination. In the process of determining the optimal inter-frame prediction mode, the prior art needs to use each prediction combination to predict the current CU once, that is, traverse all prediction combinations. When the number of prediction combinations is large, it will cause the problem of slow encoding speed. Summary of the invention

[0005] The purpose of this application is to solve at least one of the above technical defects. The technical solutions provided by the embodiments of this application are as follows:

[0006] In a first aspect, an embodiment of the present application provides a method for determining a reference frame, including:

[0007] Determine the frame type of the image frame where the current block is located;

[0008] Obtaining an identifier of a reference frame of at least one neighboring block of the current block;

[0009] Predicting the current block using each single reference frame prediction mode, and obtaining the rate distortion cost corresponding to each single reference frame prediction mode;

[0010] Based on the frame type, the identifiers of the reference frames of each adjacent block and the rate-distortion cost corresponding to each single reference frame prediction mode, a target reference frame template corresponding to each combined reference frame prediction mode is obtained, wherein each target reference frame template stores the identifiers of at least two target single reference frames, and the at least two target single reference frames are used to combine and form a target combined reference frame corresponding to the corresponding combined reference frame prediction mode;

[0011] Based on the target reference frame templates corresponding to the combined reference frame prediction modes, the target combined reference frames corresponding to the current block when the combined reference frame prediction modes are used to predict the current block are determined.

[0012] In an optional embodiment of the present application, based on the frame type, the identifier of the reference frame of each adjacent block, and the rate-distortion cost corresponding to each single reference frame prediction mode, obtaining the target reference frame template corresponding to each combined reference frame prediction mode includes:

[0013] Acquire a first reference frame template based on identifiers of reference frames of each adjacent block and rate-distortion costs corresponding to each single reference frame prediction mode;

[0014] Based on the frame type and the rate-distortion cost corresponding to each single reference frame prediction mode, obtaining a second reference frame template corresponding to each single reference frame prediction mode;

[0015] Based on the frame type, obtaining a third reference frame template;

[0016] Based on the first reference frame template, each second reference frame template and the third reference frame template, a target reference frame template corresponding to each combined reference frame mode is obtained.

[0017] In an optional embodiment of the present application, obtaining a first reference frame template based on identifiers of reference frames of each adjacent block and rate-distortion costs corresponding to each single reference frame prediction mode includes:

[0018] Taking the single reference frame corresponding to the minimum rate-distortion cost corresponding to each single reference frame prediction mode as the first optimal single reference frame, and obtaining an identifier of the first optimal single reference frame;

[0019] The identifiers of the reference frames of the adjacent blocks and the identifier of the first optimal single reference frame are stored in a predetermined format to obtain a first reference frame template.

[0020] In an optional embodiment of the present application, the method further includes:

[0021] Taking the single reference frame corresponding to the minimum rate-distortion cost corresponding to the specified single reference frame prediction mode as the second best single reference frame, and obtaining the identifier of the second best single reference frame;

[0022] The reference frame identifiers of the reference frames of each adjacent block and the identifier of the first optimal single reference frame are stored in a predetermined form to obtain a first reference frame template, including:

[0023] The reference frame identifiers of the reference frames of each adjacent block, the identifier of the first optimal single reference frame, and the identifier of the second optimal single reference frame are stored in a predetermined format to obtain a first reference frame template.

[0024] In an optional embodiment of the present application, based on the frame type and the rate-distortion cost corresponding to each single reference frame prediction mode, obtaining a second reference frame template corresponding to each single reference frame prediction mode includes:

[0025] Obtain corresponding weight level based on frame type;

[0026] Based on the weight level and the minimum rate-distortion cost corresponding to each single reference frame prediction mode, obtaining a corresponding judgment threshold;

[0027] A single reference frame corresponding to a rate-distortion cost less than a judgment threshold among rate-distortion costs corresponding to each single reference frame prediction mode is obtained, and an identifier of the single reference frame is stored in a predetermined form to obtain a second reference frame template corresponding to the single reference frame prediction mode.

[0028] In an optional embodiment of the present application, based on the weight level and the minimum rate-distortion cost corresponding to each single reference frame prediction mode, the corresponding judgment threshold is obtained, including:

[0029] Obtaining parameter values ​​corresponding to weight levels from a preset correspondence table, where the preset correspondence table stores correspondences between weight levels and parameter values;

[0030] Multiply the parameter value by the minimum rate-distortion cost to obtain the judgment threshold.

[0031] In an optional embodiment of the present application, obtaining a third reference frame template based on a frame type includes:

[0032] Obtain corresponding weight level based on frame type;

[0033] If the weight level is less than or equal to the preset level, the first number of specified single reference frames are stored in a preset form to obtain a third reference frame template. If the weight level is greater than the preset level, the second number of specified single reference frames are stored in a preset form to obtain a third reference frame template, wherein the first number is greater than the second number.

[0034] In an optional embodiment of the present application, based on the first reference frame template, each second reference frame template and the third reference frame template, obtaining a target reference frame template corresponding to each combined reference frame mode includes:

[0035] The single reference frame prediction mode corresponding to the minimum rate distortion cost is used as the optimal single reference frame mode;

[0036] For each combined reference frame prediction mode, if the combined reference frame prediction mode is obtained by combining only the optimal single reference frame mode, the second reference frame template, the first reference frame template and the third reference frame template corresponding to the optimal single reference frame prediction mode are combined to obtain a target reference frame template corresponding to the combined reference frame prediction mode; and

[0037] If the combined reference frame prediction mode is not obtained by combining only the optimal single reference frame mode, the second reference frame template and the first reference frame template corresponding to the single reference frame prediction mode that is used to obtain the combined reference frame prediction mode are combined to obtain the target reference frame template corresponding to the combined reference frame prediction mode.

[0038] In an optional embodiment of the present application, obtaining a target reference frame template by combining various reference frame templates includes:

[0039] Obtaining the identifier of the target single reference frame stored in each reference frame template;

[0040] The identifier of the target single reference frame is stored in a predetermined format to obtain a target reference frame template.

[0041] In an optional embodiment of the present application, based on the target reference frame templates corresponding to the combined reference frame prediction modes, determining the target combined reference frame corresponding to the current block when using the combined reference frame prediction modes to predict the current block includes:

[0042] An identification set consisting of identifications of target single reference frames stored in a target reference frame template corresponding to each combined reference frame prediction mode is obtained, and a combined reference frame whose label corresponds to a single reference frame in each combined reference frame and belongs to the identification set is determined as the target combined reference frame corresponding to the combined reference frame prediction mode.

[0043] In a second aspect, an embodiment of the present application provides a device for determining a reference frame, including:

[0044] A frame type determination module, used to determine the frame type of the image frame where the current block is located;

[0045] A module for obtaining an identifier of a reference frame of an adjacent block, used to obtain an identifier of a reference frame of at least one adjacent block of the current block;

[0046] A single reference frame prediction module, used to predict the current block using each single reference frame prediction mode, and obtain the rate distortion cost corresponding to each single reference frame prediction mode;

[0047] A target reference frame template acquisition module is used to acquire a target reference frame template corresponding to each combined reference frame prediction mode based on a frame type, an identifier of a reference frame of each adjacent block, and a rate-distortion cost corresponding to each single reference frame prediction mode, wherein each target reference frame template stores identifiers of at least two target single reference frames, and the at least two target single reference frames are used to combine and form a target combined reference frame corresponding to the corresponding combined reference frame prediction mode;

[0048] The target combined reference frame determination module is used to determine the corresponding target combined reference frame when predicting the current block using each combined reference frame prediction mode based on the target reference frame template corresponding to each combined reference frame prediction mode.

[0049] In an optional embodiment of the present application, the target reference frame template acquisition module includes a first reference frame template acquisition submodule, a second reference frame template acquisition submodule, a third reference frame template acquisition submodule and a target reference frame template acquisition submodule, wherein:

[0050] A first reference frame template acquisition submodule, used to acquire a first reference frame template based on an identifier of a reference frame of each adjacent block and a rate-distortion cost corresponding to each single reference frame prediction mode;

[0051] A second reference frame template acquisition submodule, used to acquire a second reference frame template corresponding to each single reference frame prediction mode based on a frame type and a rate-distortion cost corresponding to each single reference frame prediction mode;

[0052] A third reference frame template acquisition submodule, used to acquire a third reference frame template based on a frame type;

[0053] The target reference frame template acquisition submodule is used to acquire the target reference frame template corresponding to each combined reference frame mode based on the first reference frame template, each second reference frame template and the third reference frame template.

[0054] In an optional embodiment of the present application, the first reference frame template acquisition submodule is specifically used for:

[0055] Taking the single reference frame corresponding to the minimum rate-distortion cost corresponding to each single reference frame prediction mode as the first optimal single reference frame, and obtaining an identifier of the first optimal single reference frame;

[0056] The identifiers of the reference frames of the adjacent blocks and the identifier of the first optimal single reference frame are stored in a predetermined format to obtain a first reference frame template.

[0057] In an optional embodiment of the present application, the device further includes a second optimal single reference frame acquisition module, configured to:

[0058] Taking the single reference frame corresponding to the minimum rate-distortion cost corresponding to the specified single reference frame prediction mode as the second best single reference frame, and obtaining the identifier of the second best single reference frame;

[0059] The first reference frame template acquisition submodule is also used for:

[0060] The reference frame identifiers of the reference frames of each adjacent block, the identifier of the first optimal single reference frame, and the identifier of the second optimal single reference frame are stored in a predetermined format to obtain a first reference frame template.

[0061] In an optional embodiment of the present application, the second reference frame template acquisition submodule is specifically used for:

[0062] Obtain corresponding weight level based on frame type;

[0063] Based on the weight level and the minimum rate-distortion cost corresponding to each single reference frame prediction mode, obtaining a corresponding judgment threshold;

[0064] A single reference frame corresponding to a rate-distortion cost less than a judgment threshold among rate-distortion costs corresponding to each single reference frame prediction mode is obtained, and an identifier of the single reference frame is stored in a predetermined form to obtain a second reference frame template corresponding to the single reference frame prediction mode.

[0065] In an optional embodiment of the present application, the second reference frame template acquisition submodule is further used for:

[0066] Obtaining parameter values ​​corresponding to weight levels from a preset correspondence table, where the preset correspondence table stores correspondences between weight levels and parameter values;

[0067] Multiply the parameter value by the minimum rate-distortion cost to obtain the judgment threshold.

[0068] In an optional embodiment of the present application, the third reference frame template acquisition submodule is further used for:

[0069] Obtain corresponding weight level based on frame type;

[0070] If the weight level is less than or equal to the preset level, the first number of specified single reference frames are stored in a preset form to obtain a third reference frame template. If the weight level is greater than the preset level, the second number of specified single reference frames are stored in a preset form to obtain a third reference frame template, wherein the first number is greater than the second number.

[0071] In an optional embodiment of the present application, the target reference frame template acquisition submodule is specifically used for:

[0072] The single reference frame prediction mode corresponding to the minimum rate distortion cost is used as the optimal single reference frame mode;

[0073] For each combined reference frame prediction mode, if the combined reference frame prediction mode is obtained by combining only the optimal single reference frame mode, the second reference frame template, the first reference frame template and the third reference frame template corresponding to the optimal single reference frame prediction mode are combined to obtain a target reference frame template corresponding to the combined reference frame prediction mode; and

[0074] If the combined reference frame prediction mode is not obtained by combining only the optimal single reference frame mode, the second reference frame template and the first reference frame template corresponding to the single reference frame prediction mode that is used to obtain the combined reference frame prediction mode are combined to obtain the target reference frame template corresponding to the combined reference frame prediction mode.

[0075] In an optional embodiment of the present application, the target reference frame template acquisition submodule is further used to:

[0076] Obtaining the identifier of the target single reference frame stored in each reference frame template;

[0077] The identifier of the target single reference frame is stored in a predetermined format to obtain a target reference frame template.

[0078] In an optional embodiment of the present application, the target combined reference frame determination module is specifically used to:

[0079] An identification set consisting of identifications of target single reference frames stored in a target reference frame template corresponding to each combined reference frame prediction mode is obtained, and a combined reference frame whose label corresponds to a single reference frame in each combined reference frame and belongs to the identification set is determined as the target combined reference frame corresponding to the combined reference frame prediction mode.

[0080] In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor;

[0081] A computer program is stored in the memory;

[0082] A processor is used to execute a computer program to implement the method provided in the embodiment of the first aspect.

[0083] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the method provided in the embodiment of the first aspect is implemented.

[0084] In a fifth aspect, an embodiment of the present application provides a computer program product or a computer program, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that when the computer device executes, the method provided in the first aspect embodiment or the second aspect embodiment is implemented.

[0085] The beneficial effects of the technical solution provided by this application are:

[0086] In the inter-frame prediction process, the current block is first predicted using the prediction combination corresponding to the single reference frame prediction mode to obtain the corresponding rate-distortion cost, and the frame type of the image frame where the current block is located and the reference frame identifiers of each adjacent block are obtained. Then, this information is used to determine the target combined reference frame of each combined reference frame mode. In the process of selecting the optimal combined reference frame prediction combination, the number of prediction combinations involved in the prediction can be reduced, which saves calculation time and improves the encoding speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in describing the embodiments of the present application.

[0088] Figure 1 A schematic diagram of a flow chart of a method for determining a reference frame provided in an embodiment of the present application;

[0089] Figure 2a A schematic diagram of information obtained before combined reference frame prediction in an example of an embodiment of the present application;

[0090] Figure 2b Based on Figure 2a A schematic diagram of obtaining a target reference frame template of each combined reference frame prediction mode using the information obtained in;

[0091] Figure 3 A schematic diagram of the position distribution of adjacent blocks of a current block in an example of an embodiment of the present application;

[0092] Figure 4 A schematic diagram of performing single reference frame prediction on a current block in an example of an embodiment of the present application;

[0093] Figure 5 A schematic diagram of reference relationships corresponding to different frame types in an embodiment of the present application;

[0094] Figure 6 This is a reference relationship diagram of GOP16 in the embodiment of the present application;

[0095] Figure 7A schematic diagram of a process of obtaining a second reference frame template for each single reference frame prediction mode in an example of an embodiment of the present application;

[0096] Figure 8 A schematic diagram of a process of determining a target combined reference frame of a combined reference frame prediction mode in an example of an embodiment of the present application;

[0097] Fig. 9 A structural block diagram of a device for determining a reference frame provided in an embodiment of the present application;

[0098] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0099] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present invention.

[0100] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0101] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0102] In order to facilitate the detailed explanation of the technical solution provided by this application, this application will take the AV1 video compression standard as an example for explanation, but the scope of application of the technical solution provided by the embodiments of this application is not limited to the AV1 video compression standard, and can also be applied to compression standards such as H.266 and avs3.

[0103] In the AV1 video compression standard, the inter-frame prediction type includes a plurality of single reference frame prediction modes and a plurality of combined reference frame prediction modes obtained by combining the single reference frame prediction modes, wherein each single reference frame prediction mode and a plurality of single reference frames respectively constitute a corresponding single reference frame prediction combination, and each combined reference frame prediction mode and a plurality of combined reference frames respectively constitute a corresponding combined reference frame prediction combination. It should be noted that a plurality of combined reference frame prediction modes are obtained by combining a plurality of single reference frame prediction modes in a corresponding form, and a plurality of combined reference frames are also obtained by combining a plurality of single reference frames in a corresponding form. Specifically, 4 single reference frame prediction modes are combined to obtain 8 combined reference frame prediction modes, and 7 single reference frames are combined to obtain 16 combined reference frames. For example, the combined reference frame prediction mode NEW_NEARESTMV is composed of the single reference frame prediction modes NEWMV and NEARESTMV, and the combined reference frame {LAST_FRAME, ALTREF_FRAME} is composed of the single reference frames LAST_FRAME and ALTREF_FRAME.

[0104] Specifically, the single reference frame prediction modes include: NEARESTMV, NEARMV, GLOBALMV and NEWMV 4 types, each single reference frame prediction mode corresponds to the single reference frames: LAST_FRAME, LAST2_FRAME, LAST3_FRAME, GOLDEN_FRAME, BWDREF_FRAME, ALTREF2_FRAME and ALTREF_FRAME 7 types, then, there are 28 single reference frame prediction combinations corresponding to the single reference frame prediction mode and the single reference frame; the combined reference frame prediction modes include: NEAREST_NEARESTMV, NEAR_NEARMV, NEAREST_NEWMV, NEW_NEARESTMV, NEAR_NEWMV, NEW_NEARMV, GLOBAL_GLOBALMV and NEW_NEWMV There are 8 combined reference frames corresponding to each combined frame reference prediction mode: {LAST_FRAME, ALTREF_FRAME}, {LAST2_FRAME, ALTREF_FRAME}, {LAST3_FRAME, ALTREF_FRAME}, {GOLDEN_FRAME, ALTREF_FRAME}, {LAST_FRAME, BWDREF_FRAME}, {LAST2_FRAME, BWDREF_FRAME}, {LAST3_FRAME, BWDREF_FRAME}, {GOLDEN_FRAME, BWDREF_FRAME}, {LAST_FRAME, ALTREF_FRAME} 2_FRAME}, {LAST2_FRAME, ALTREF2_FRAME}, {LAST3_FRAME, ALTREF2_FRAME}, {GOLDEN_FRAME, ALTREF2_FRAME}, {LAST_FRAME, LAST2_FRAME}, {LAST_FRAME, LAST3_FRAME}, {LAST_FRAME, GOLDEN_FRAME} and {BWDREF_FRAME, ALTREF_FRAME}, then there are 128 combined reference frame prediction combinations corresponding to the combined reference frame prediction modes and the combined reference frames, so the total number of prediction combinations corresponding to inter-frame prediction is 156.

[0105] In the prior art, in the process of predictive coding, in order to determine the optimal inter-frame prediction combination corresponding to the current block from the above 156 prediction combinations, it is necessary to use each prediction combination to predict the current block, and determine the optimal prediction combination according to the size of the rate-distortion cost obtained after the prediction of each prediction combination. Since it is necessary to traverse 156 prediction combinations, a lot of computing time is consumed, which reduces the coding speed. In view of the above problems, the embodiment of the present application provides a method for determining a reference frame to improve the coding speed.

[0106] Figure 1 A flowchart of a method for determining a reference frame provided in an embodiment of the present application is shown as follows: Figure 1 As shown, the method may include:

[0107] Step S101, determining the frame type of the image frame where the current block is located.

[0108] Step S102: Obtain an identifier of a reference frame of at least one adjacent block of the current block.

[0109] Step S103 : predicting the current block respectively by using each single reference frame prediction mode, and obtaining the rate distortion cost corresponding to each single reference frame prediction mode.

[0110] Specifically, in the process of obtaining the optimal inter-frame prediction combination, first, the current block is predicted in a single reference frame prediction mode, that is, each single reference frame prediction combination is used to predict the current block, and the rate distortion cost obtained after the current block is predicted using each single reference frame prediction mode is obtained, that is, the rate distortion cost corresponding to each single reference frame prediction combination is obtained; at the same time, the frame type of the image frame where the current block is located is determined, and the identifier of the reference frame of each adjacent block of each current block is obtained. Then, after obtaining the above information, the above information is used to guide the prediction of the combined reference frame prediction mode of the current block.

[0111] Step S104, based on the frame type, the identifiers of the reference frames of each adjacent block and the rate-distortion cost corresponding to each single reference frame prediction mode, obtain the target reference frame template corresponding to each combined reference frame prediction mode, wherein each target reference frame template stores the identifiers of at least two target single reference frames, and the at least two target single reference frames are used to combine to form a target combined reference frame corresponding to the corresponding combined reference frame prediction mode.

[0112] Among them, the frame type of the image where the current block is located can indicate the reference relationship of the current image frame in the multi-frame image frame, and the reference relationship can guide the selection of the target single reference frame of the current block. The reference frames of each adjacent block can be directly used as the target single reference frame of the current block. The rate-distortion cost corresponding to each single reference frame prediction mode is the rate-distortion cost of each prediction combination corresponding to the single reference frame prediction mode. The single reference frame in the prediction combination with a smaller rate-distortion cost can be used as the target single reference frame. In summary, the information obtained in steps S101-S103 can be used to determine the target single reference frame corresponding to each single reference frame prediction mode. Since the combined reference frame prediction mode is obtained by combining the single reference frame prediction modes, and each combined reference frame is obtained by combining the single reference frames, the target single reference frame corresponding to each combined reference frame prediction mode can be determined according to the target single reference frame, and then the target combined reference frame corresponding to each combined reference frame prediction mode is determined. This process can be understood as the determination process of the target combined reference frame corresponding to each combined reference frame prediction mode.

[0113] Specifically, based on the frame type, the identifiers of the reference frames of each adjacent block and the rate-distortion cost corresponding to each single reference frame prediction mode, the identifiers of multiple target single reference frames are obtained, and then based on the obtained identifiers of the multiple target single reference frames, the target reference frame templates corresponding to each reference frame template are obtained.

[0114] Step S105 : determining a target combined reference frame corresponding to each combined reference frame prediction mode when predicting the current block based on the target reference frame template corresponding to each combined reference frame prediction mode.

[0115] Among them, in the prior art, in the process of determining the optimal inter-frame prediction mode, it is necessary to use the prediction combination corresponding to each single reference frame prediction mode and the prediction combination corresponding to each combined reference frame prediction mode to predict the current block, that is, to traverse all single reference frame prediction combinations and combined reference frame prediction combinations. In the embodiment of the present application, in order to reduce the number of predictions and save calculation time, step S103 traverses all single reference frame prediction combinations, and before using the combined reference frame prediction mode to predict the current block, the corresponding target combined reference frame is determined for each combined reference frame prediction mode, and then only the prediction combination corresponding to each combined reference frame prediction mode and the corresponding target combined reference frame is used to predict the current block. Then, in order to save calculation time and improve the encoding speed, the key to the present application is to determine the target combined reference frame corresponding to each combined reference frame prediction mode.

[0116] Specifically, before predicting the current block using the combined reference frame prediction mode, multiple corresponding target single reference frames are obtained based on the target reference frame models corresponding to each combined reference frame prediction mode, and the corresponding target combined reference frame is obtained based on the combination of the multiple target single reference frames.

[0117] For example, if the target single reference frame identifier contained in the target reference frame template corresponding to the determined combined reference frame prediction mode NEAREST_NEWMV is: the identifier of LAST3_FRAME and the identifier of ALTREF_FRAME, then the target combined reference frame corresponding to NEAREST_NEWMV is {LAST3_FRAME, ALTREF_FRAME}. Further, in the process of predicting the current block using NEAREST_NEWMV, only the prediction combination corresponding to NEAREST_NEWMV and {LAST3_FRAME, ALTREF_FRAME} is used to predict the current block, which reduces the prediction by up to 15 times compared with the prior art.

[0118] The solution provided by the present application, in the inter-frame prediction process, first uses the prediction combination corresponding to the single reference frame prediction mode to predict the current block to obtain the corresponding rate-distortion cost, and obtains the frame type of the image frame where the current block is located and the reference frame identifiers of each adjacent block, and then uses this information to determine the target combined reference frame of each combined reference frame mode. In the process of selecting the optimal combined reference frame prediction combination, the number of prediction combinations participating in the prediction can be reduced, thereby saving calculation time and improving the encoding speed.

[0119] In an optional embodiment of the present application, based on the frame type, the identifier of the reference frame of each adjacent block, and the rate-distortion cost corresponding to each single reference frame prediction mode, obtaining the target reference frame template corresponding to each combined reference frame prediction mode includes:

[0120] Acquire a first reference frame template based on identifiers of reference frames of each adjacent block and rate-distortion costs corresponding to each single reference frame prediction mode;

[0121] Based on the frame type and the rate-distortion cost corresponding to each single reference frame prediction mode, obtaining a second reference frame template corresponding to each single reference frame prediction mode;

[0122] Based on the frame type, obtaining a third reference frame template;

[0123] Based on the first reference frame template, each second reference frame template and the third reference frame template, a target reference frame template corresponding to each combined reference frame mode is obtained.

[0124] Specifically, before obtaining the target reference frame template, different reference frame templates are obtained based on different information. Specifically, based on the identifier of the reference frame of each adjacent block and the rate-distortion cost corresponding to each single reference frame prediction mode, a first reference frame template is obtained; based on the frame type and the rate-distortion cost corresponding to each single reference frame prediction mode, a second reference frame template corresponding to each single reference frame prediction mode is obtained; based on the frame type of the image frame where the current block is located, a third reference frame template is obtained. Then, based on the single reference frame prediction mode corresponding to each combined reference frame prediction mode and other information, a target reference frame template corresponding to each combined reference frame prediction mode is obtained.

[0125] Specifically, before obtaining the target reference model, the above information needs to be collected. Figure 2a As shown, it mainly includes the acquisition of the following information: acquiring the neighboring block information of the current block, acquiring the rate-distortion cost of each prediction combination corresponding to the single reference frame prediction mode, acquiring the optimal single reference frame prediction mode information, and acquiring the frame type of the image frame where the current block is located. After acquiring the above information, the first reference frame template, the second reference frame template, and the third reference frame template are acquired based on the above information, and then the target reference frame template corresponding to each combined reference frame prediction mode is obtained, such as Figure 2b shown.

[0126] For example, if Figure 3 As shown, the positional relationship between the current block E and the adjacent blocks A, B, C, D. If A, B, C, D exist and are inter-frame predicted, then the respective reference frame information (i.e., reference frame identifier) ​​of A, B, C, D is recorded, which can be recorded as ref_nb[4] = {ref_A, ref_B, ref_C, ref_D}.

[0127] like Figure 4As shown, the current block is predicted using prediction combinations corresponding to the four single reference frame prediction modes. For the prediction combination corresponding to each single reference frame prediction mode, the rate-distortion cost corresponding to the prediction combination corresponding to each single reference frame prediction mode is recorded (that is, the rate-distortion cost corresponding to each single reference frame under each single reference frame prediction mode). The rate-distortion costs corresponding to the prediction combinations corresponding to the four single reference frame prediction modes can be recorded as: rd_nearestmv[8], rd_newmv[8], rd_nearmv[8] and rd_globalmv[8]. It can be understood that the "8" represents the size of the array used for storage, which corresponds to 7 inter-frame types and 1 intra-frame type. For example, rd_nearestmv[8] stores respectively: rd_nearestmv[LAST_FRAME], rd_nearestmv[LAST2_FRAME], rd_nearestmv[LAST3_FRAME], rd_ nearestmv[GOLDEN_FRAME], rd_nearestmv[BWDREF_FRAME], rd_nearestmv[ALTREF2_FRAME], and rd_nearestmv[ALTREF_FRAME].

[0128] It should be noted that Figure 4 The prediction order in can be adjusted according to actual needs, and this application does not impose any restrictions on this.

[0129] After the prediction is completed, the single reference frame prediction mode corresponding to the prediction combination corresponding to the minimum rate-distortion cost is determined to be the optimal single reference frame prediction mode, and the corresponding single reference frame is determined to be the first optimal single reference frame, which can be recorded as single_best_mode (optimal single reference frame prediction mode), single_best_ref1 (first optimal single reference frame) and best_rd1 respectively. Further, the single reference frame corresponding to the minimum rate-distortion cost in the prediction combination corresponding to NEARESTMV is determined to be the second optimal single reference frame, which can be recorded as ref_nearestmv (second optimal single reference frame).

[0130] Get the frame type of the image frame where the current block is located. The frame types are sorted from most important to least important according to the reference relationship: I frame>P frame>B frame>b frame>non-reference B frame, such as Figure 5 In specific implementation, the reference relationship of the frame type can also be represented by a GOP (Group of Pictures) structure, such as Figure 6As shown, poc16 refers to poc0; poc8 refers to poc0 and poc16; poc4 refers to poc0 and poc8; poc2 refers to poc0 and poc4; and poc1, poc3, poc5, poc7, poc9, poc11, poc13 and poc15 are not referenced. The weight levels corresponding to each frame type are shown in Table 1. The weights are sorted from large to small as follows: poc0>poc16>poc8>poc4>poc2>poc1. The selection of the reference frame can be adjusted in combination with the weight level corresponding to the frame type corresponding to the current block. It should be noted that the frame type of the image frame where the current block is located has been set before prediction, so the current weight level can be obtained before prediction, which can be recorded as slice_level.

[0131] Table 1

[0132]

[0133]

[0134] In an optional embodiment of the present application, obtaining a first reference frame template based on identifiers of reference frames of each adjacent block and rate-distortion costs corresponding to each single reference frame prediction mode includes:

[0135] Taking the single reference frame corresponding to the minimum rate-distortion cost corresponding to each single reference frame prediction mode as the first optimal single reference frame, and obtaining an identifier of the first optimal single reference frame;

[0136] The identifiers of the reference frames of the adjacent blocks and the identifier of the first optimal single reference frame are stored in a predetermined format to obtain a first reference frame template.

[0137] Specifically, the first optimal single reference frame is used as the target single reference frame, and then the identifier of the target single reference frame and the identifiers of the reference frames of each adjacent block are stored in a predetermined form to obtain the first reference frame template.

[0138] Among them, the storage in a predetermined form may be storing the identification information in an array of a predetermined number of bits. For example, the template is an array containing 8 bits, and each bit corresponds to a type of single reference frame. In the process of generating a template based on the target single reference frame, the type of the target single reference frame is first determined, and then the number of bits in the array corresponding to the type is set to 1, and the other bits are set to 0, and the corresponding template is obtained.

[0139] Furthermore, the method further comprises:

[0140] Taking the single reference frame corresponding to the minimum rate-distortion cost corresponding to the specified single reference frame prediction mode as the second best single reference frame, and obtaining the identifier of the second best single reference frame;

[0141] Store the identifiers of the reference frames of adjacent blocks and the identifier of the first optimal single reference frame in a predetermined form to obtain a first reference frame template, including:

[0142] Store the identifiers of the reference frames of adjacent blocks, the identifier of the first optimal single reference frame, and the identifier of the second optimal single reference frame in a predetermined form to obtain a first reference frame template.

[0143] Specifically, since the probability of NEARESTMV being the optimal single reference frame prediction mode is the highest, the specified single reference frame prediction mode can be NEARESTMV. Therefore, the second optimal single reference frame can also be used as the target single reference frame. That is, during the prediction of the current block using NEARESTMV, the single reference frame corresponding to the minimum rate-distortion cost is used as the target single reference frame. Therefore, store the identifiers of the reference frames of adjacent blocks, the identifier of the first optimal single reference frame, and the identifier of the second optimal single reference frame in a predetermined form to obtain a first reference frame template.

[0144] The above process of generating the first reference frame template can be implemented by the following code:

[0145] A. If ref_nearestmv is greater than 0, then mask_init |= (1 << ref_nearestmv) (that is, if the second optimal single reference frame is obtained, store the identifier of the second optimal single reference frame in the first reference frame template in a predetermined form, where mask_init represents the first reference frame template);

[0146] B. If single_best_ref is greater than 0, then mask_init |= (1 << single_best_ref) (that is, if the first optimal single reference frame is obtained, store the identifier of the first optimal single reference frame in the first reference frame template in a predetermined form);

[0147] C. Loop through the number of adjacent blocks and judge each block in turn. Since adjacent blocks can be dual reference frames, two reference frames need to be judged.

[0148] for(int i = 0; i < 4; i++){

[0149] if(ref_nb[i][0] > 0) mask_init |= (1 << ref_nb[i][0]);

[0150] if (ref_nb[i][1] > 0) mask_init |= (1 << ref_nb[i][1]);} (That is, loop through each adjacent block in sequence. If the first reference frame (ref_nb[i][0]) of the i-th adjacent block is obtained, its identifier is stored in the first reference frame template in a predetermined form. If the second reference frame (ref_nb[i][1]) of the i-th adjacent block is obtained, its identifier is stored in the first reference frame template in a predetermined form).

[0151] In an alternative embodiment of the present application, obtaining the second reference frame template corresponding to each single reference frame prediction mode based on the frame type and the rate distortion cost corresponding to each single reference frame prediction mode includes:

[0152] Obtaining the corresponding weight level based on the frame type;

[0153] Obtaining the corresponding judgment threshold based on the weight level and the minimum rate distortion cost corresponding to each single reference frame prediction mode;

[0154] Obtaining the single reference frame corresponding to the rate distortion cost less than the judgment threshold in the rate distortion cost corresponding to each single reference frame prediction mode, and storing the identifier of the single reference frame in a predetermined form to obtain the second reference frame template corresponding to the single reference frame prediction mode.

[0155] Further, obtaining the corresponding judgment threshold based on the weight level and the minimum rate distortion cost corresponding to each single reference frame prediction mode includes:

[0156] Obtaining the parameter value corresponding to the weight level from a preset correspondence table, where the preset correspondence table stores the correspondence between each weight level and each parameter value;

[0157] Multiplying the parameter value by the minimum rate distortion cost to obtain the judgment threshold.

[0158] Specifically, each single reference frame prediction mode has its corresponding second reference frame template. For each single reference frame prediction mode, first obtain a judgment threshold, and then compare the rate distortion cost of each prediction combination corresponding to the single reference frame prediction mode with the judgment threshold. If the rate distortion cost of a certain prediction combination corresponding to the single reference frame prediction mode is less than the judgment threshold, the single reference frame corresponding to the prediction combination is used as the target single reference frame, and the identifier of the target single reference frame is stored in a predetermined form. After traversing the prediction combinations corresponding to the single reference frame prediction mode in the above manner, the second reference frame template corresponding to the single reference frame prediction mode is obtained.

[0159] In specific implementation, such as Figure 7As shown, the traversal of each single-reference frame prediction combination can be performed in the form of cycling through 7 single-reference frames (in this example, the rate-distortion cost is multiplied by 16 and then compared with the judgment threshold). Among them, mask_nearest, mask_new, mask_near, and mask_global respectively represent the second-reference frame templates of 4 single-reference frame prediction modes. mask_nearest|=(1<<i) means storing the i-th single-reference frame into the second-reference frame template corresponding to NEARESTMV. Thr represents the judgment threshold, and its calculation formula can be:

[0160] thr = param[slice_level]*best_rd

[0161] Wherein, best_rd is the first optimal rate-distortion cost, and param[slice_level] is the preset parameter corresponding to the weight level slice_level, which can be set according to actual situations. For example, param[6] = {0, 22, 22, 20, 20, 18} (that is, 6 different preset parameters corresponding to six different weight levels).

[0162] In an alternative embodiment of the present application, obtaining a third-reference frame template based on the frame type includes:

[0163] Obtaining the corresponding weight level based on the frame type;

[0164] If the weight level is less than or equal to the preset level, store the first number of specified single-reference frames in a preset form to obtain the third-reference frame template. If the weight level is greater than the preset level, store the second number of specified single-reference frames in a preset form to obtain the third-reference frame template, where the first number is greater than the second number.

[0165] Specifically, the third-reference frame template is determined according to the frame type of the image frame where the current block is located, that is, the third-reference frame template is determined according to the weight level of the frame type, and different weight levels result in different numbers of specified single-reference frames included in the third-reference frame template.

[0166] For example, if the weight level slice_level is less than or equal to 3, the corresponding third-reference template is intmask_add0. If the weight level slice_level is greater than 3, the corresponding third-reference template is int mask_add1, where:

[0167] int mask_add0 = (1 << LAST_FRAME) | (1 << GOLDEN_FRAME) | (1 << BWDREF_FRAME) | (1 << ALTREF_FRAME) (That is, the third reference template is that the number of target single reference frames included in int mask_add0 is 4, namely LAST_FRAME, GOLDEN_FRAME, BWDREF_FRAME, and ALTREF_FRAME);

[0168] int mask_add1 = (1 << LAST_FRAME) | (1 << GOLDEN_FRAME) | (1 << BWDREF_FRAME) (That is, the third reference template is that the number of target single reference frames included in int mask_add1 is 3, namely LAST_FRAME, GOLDEN_FRAME, and BWDREF_FRAME).

[0169] In an alternative embodiment of the present application, based on the first reference frame template, each second reference frame template, and the third reference frame template, obtaining the target reference frame template corresponding to each combined reference frame mode includes:

[0170] Taking the single reference frame prediction mode corresponding to the minimum rate distortion cost as the optimal single reference frame mode;

[0171] For each combined reference frame prediction mode, if the combined reference frame prediction mode is only obtained by combining the optimal single reference frame mode, then combining the second reference frame template, the first reference frame template, and the third reference frame template corresponding to the optimal single reference frame prediction mode to obtain the target reference frame template corresponding to the combined reference frame prediction mode; and,

[0172] If the combined reference frame prediction mode is not only obtained by combining the optimal single reference frame mode, then combining the second reference frame template and the first reference frame template corresponding to the single reference frame prediction mode that combines to obtain the combined reference frame prediction mode to obtain the target reference frame template corresponding to the combined reference frame prediction mode.

[0173] Further, obtaining the target reference frame template by combining each reference frame template includes:

[0174] Obtaining the identifiers of the target single reference frames stored in each reference frame template;

[0175] Storing the identifiers of the target single reference frames in a predetermined form to obtain the target reference frame template.

[0176] Specifically, after obtaining the first reference frame template, the second reference frame template and the third reference frame template, for each combined reference frame prediction mode, the corresponding target reference frame is obtained by combining the first reference frame template, the second reference frame template and the third reference frame template. Specifically, the target reference frame template of each combined reference frame prediction mode includes the identifier of the target single reference frame in the first reference frame template, and also includes the target single reference frame in the second reference frame template corresponding to its corresponding single reference frame prediction mode, but whether the target single reference frame in the third reference frame template is included depends on whether the combined reference frame template is obtained by combining the optimal single reference frame mode. If so, the target single reference frame in the third reference frame template is included; if not, the target single reference frame in the third reference frame template is not included.

[0177] For example, the above process of obtaining the target reference frame template can be expressed as follows:

[0178] (1) First, the second reference frame template of the single reference frame prediction mode corresponding to each combined reference frame prediction mode is combined with the first reference frame template as follows:

[0179] mask[NEW_NEWMV] = (mask_new|mask_init) (i.e., the target reference frame template of NEW_NEWMV is obtained by combining the second reference frame template mask_new and the first reference frame template of NEWMV);

[0180] mask[NEAREST_NEARESTMV] = (mask_nearest|mask_init) (i.e., the target reference frame template of NEAREST_NEARESTMV is obtained by combining the second reference frame template mask_nearest and the first reference frame template of NEARESTMV);

[0181] mask[NEW_NEARESTMV]=mask[NEAREST_NEWMV]=(mask_new|mask_init|mask_nearest) (i.e., the target reference frame template of NEW_NEARESTMV and NEAREST_NEWMV is obtained by combining the second reference frame template mask_new of NEWMV, the second reference frame template mask_nearest of NEARESTMV and the first reference frame template);

[0182] mask[NEW_NEARMV]=mask[NEAR_NEWMV]=(mask_new|mask_init|mask_near) (i.e., the target reference frame templates of NEW_NEARMV and NEAR_NEWMV are obtained by combining the second reference frame template mask_new of NEWMV, the second reference frame template mask_near of NEARMV, and the first reference frame template);

[0183] mask[NEAR_NEARMV] = (mask_near|mask_init) (i.e., the target reference frame template of NEAR_NEARMV is obtained by combining the second reference frame template mask_nearest of NEARMV and the first reference frame template);

[0184] mask[GLOBAL_GLOBALMV] = (mask_global|mask_init), that is, the target reference frame template of GLOBAL_GLOBALMV is obtained by combining the second reference frame template mask_global and the first reference frame template of GLOBALMV);

[0185] (2) If the combined reference frame prediction mode is obtained only by combining the best single reference frame mode, the number of the target single reference frame in the third reference frame template is added on the basis of (1), as follows:

[0186] If the best single reference frame prediction best_mode is NEWMV, mask[NEW_NEWMV]|=mask_add (ie, the target single reference frame in the third reference frame template is added to mask[NEW_NEWMV]);

[0187] If best_mode is NEARESTMV, mask[NEAREST_NEARESTMV]|=mask_add (ie, the target single reference frame in the third reference frame template is added to mask[NEAREST_NEARESTMV]);

[0188] If best_mode is NEARMV, mask[NEAR_NEARMV]|=mask_add (ie, the target single reference frame in the third reference frame template is added to mask[NEAR_NEARMV]);

[0189] If best_mode is GLOBALMV or DC_PRED, mask[GLOBAL_GLOBALMV]|=mask_add (ie, the target single reference frame in the third reference frame template is added to mask[GLOBAL_GLOBALMV]).

[0190] In an alternative embodiment of the present application, based on the target reference frame templates corresponding to each combined reference frame prediction mode, determining the target combined reference frame corresponding to using each combined reference frame prediction mode to predict the current block includes:

[0191] Obtaining a set of identifiers composed of the identifiers of the target single reference frames stored in the target reference frame template corresponding to each combined reference frame prediction mode, and determining, as the target combined reference frame corresponding to this combined reference frame prediction mode, the combined reference frame in which the label of the corresponding single reference frame in each combined reference frame belongs to the set of identifiers.

[0192] Specifically, during the specific combined frame prediction process, it is possible to traverse the prediction combinations corresponding to the combined reference frame prediction mode to determine whether the combined reference frame corresponding to each prediction combination is consistent with the target reference frame template of the corresponding combined reference frame prediction mode, that is, to determine whether the identifier of the single reference frame corresponding to the adopted combined reference frame is stored in the target reference frame template. If they are consistent, use this prediction combination to predict the current block; if not, do not perform the prediction.

[0193] For example, as Figure 8 shown, for all combined reference frame prediction modes, traverse the prediction combinations corresponding to each combined reference frame. According to the foregoing description, there are a total of 156 prediction combinations, where i = 0 to 27 are single reference frame prediction combinations, and i = 28 to 156 are combined reference frame prediction combinations. Therefore, start traversing from i = 28 until i = 156. In the figure, mode represents the combined reference frame prediction mode being currently traversed, ref0 and ref1 represent the identifiers of the two single reference frames corresponding to the combined reference frame being currently traversed, mask[mode] represents the target reference frame template corresponding to this combined reference frame prediction mode, and mask[mode]&(1<<ref0)&(1<<ref1) represents that ref0 and ref1 are stored in the target reference frame template.

[0194] Fig. 9 Shown is a structural block diagram of a device for determining a reference frame provided by an embodiment of the present application. As Fig. 9 shown, the device 900 may include: a frame type determination module 901, an identifier acquisition module 902 for the reference frames of adjacent blocks, a single reference frame prediction module 903, a target reference frame template acquisition module 904, and a target combined reference frame determination module 905, where:

[0195] The frame type determination module 901 is used to determine the frame type of the image frame where the current block is located;

[0196] The identifier acquisition module 902 for the reference frames of adjacent blocks is used to acquire the identifiers of the reference frames of at least one adjacent block of the current block;

[0197] The single reference frame prediction module 903 is used to predict the current block using each single reference frame prediction mode and obtain the rate distortion cost corresponding to each single reference frame prediction mode;

[0198] The target reference frame template acquisition module 904 is used to acquire the target reference frame template corresponding to each combined reference frame prediction mode based on the frame type, the identifier of the reference frame of each adjacent block and the rate-distortion cost corresponding to each single reference frame prediction mode, wherein each target reference frame template stores the identifier of at least two target single reference frames, and the at least two target single reference frames are used to combine and form a target combined reference frame corresponding to the corresponding combined reference frame prediction mode;

[0199] The target combined reference frame determination module 905 is used to determine the corresponding target combined reference frame when predicting the current block using each combined reference frame prediction mode based on the target reference frame template corresponding to each combined reference frame prediction mode.

[0200] In the inter-frame prediction process, the current block is first predicted using the prediction combination corresponding to the single reference frame prediction mode to obtain the corresponding rate-distortion cost, and the frame type of the image frame where the current block is located and the reference frame identifiers of each adjacent block are obtained. Then, this information is used to determine the target combined reference frame of each combined reference frame mode. In the process of selecting the optimal combined reference frame prediction combination, the number of prediction combinations involved in the prediction can be reduced, which saves calculation time and improves the encoding speed.

[0201] In an optional embodiment of the present application, the target reference frame template acquisition module includes a first reference frame template acquisition submodule, a second reference frame template acquisition submodule, a third reference frame template acquisition submodule and a target reference frame template acquisition submodule, wherein:

[0202] A first reference frame template acquisition submodule, used to acquire a first reference frame template based on an identifier of a reference frame of each adjacent block and a rate-distortion cost corresponding to each single reference frame prediction mode;

[0203] A second reference frame template acquisition submodule, used to acquire a second reference frame template corresponding to each single reference frame prediction mode based on a frame type and a rate-distortion cost corresponding to each single reference frame prediction mode;

[0204] A third reference frame template acquisition submodule, used to acquire a third reference frame template based on a frame type;

[0205] The target reference frame template acquisition submodule is used to acquire the target reference frame template corresponding to each combined reference frame mode based on the first reference frame template, each second reference frame template and the third reference frame template.

[0206] In an optional embodiment of the present application, the first reference frame template acquisition submodule is specifically used for:

[0207] Taking the single reference frame corresponding to the minimum rate-distortion cost corresponding to each single reference frame prediction mode as the first optimal single reference frame, and obtaining an identifier of the first optimal single reference frame;

[0208] The identifiers of the reference frames of the adjacent blocks and the identifier of the first optimal single reference frame are stored in a predetermined format to obtain a first reference frame template.

[0209] In an optional embodiment of the present application, the device further includes a second optimal single reference frame acquisition module, configured to:

[0210] Taking the single reference frame corresponding to the minimum rate-distortion cost corresponding to the specified single reference frame prediction mode as the second best single reference frame, and obtaining the identifier of the second best single reference frame;

[0211] The first reference frame template acquisition submodule is also used for:

[0212] The reference frame identifiers of the reference frames of each adjacent block, the identifier of the first optimal single reference frame, and the identifier of the second optimal single reference frame are stored in a predetermined format to obtain a first reference frame template.

[0213] In an optional embodiment of the present application, the second reference frame template acquisition submodule is specifically used for:

[0214] Obtain corresponding weight level based on frame type;

[0215] Based on the weight level and the minimum rate-distortion cost corresponding to each single reference frame prediction mode, obtaining a corresponding judgment threshold;

[0216] A single reference frame corresponding to a rate-distortion cost less than a judgment threshold among rate-distortion costs corresponding to each single reference frame prediction mode is obtained, and an identifier of the single reference frame is stored in a predetermined form to obtain a second reference frame template corresponding to the single reference frame prediction mode.

[0217] In an optional embodiment of the present application, the second reference frame template acquisition submodule is further used for:

[0218] Obtaining parameter values ​​corresponding to weight levels from a preset correspondence table, where the preset correspondence table stores correspondences between weight levels and parameter values;

[0219] Multiply the parameter value by the minimum rate-distortion cost to obtain the judgment threshold.

[0220] In an optional embodiment of the present application, the third reference frame template acquisition submodule is further used for:

[0221] Obtain corresponding weight level based on frame type;

[0222] If the weight level is less than or equal to the preset level, the first number of specified single reference frames are stored in a preset form to obtain a third reference frame template. If the weight level is greater than the preset level, the second number of specified single reference frames are stored in a preset form to obtain a third reference frame template, wherein the first number is greater than the second number.

[0223] In an optional embodiment of the present application, the target reference frame template acquisition submodule is specifically used for:

[0224] The single reference frame prediction mode corresponding to the minimum rate distortion cost is used as the optimal single reference frame mode;

[0225] For each combined reference frame prediction mode, if the combined reference frame prediction mode is obtained by combining only the optimal single reference frame mode, the second reference frame template, the first reference frame template and the third reference frame template corresponding to the optimal single reference frame prediction mode are combined to obtain a target reference frame template corresponding to the combined reference frame prediction mode; and

[0226] If the combined reference frame prediction mode is not obtained by combining only the optimal single reference frame mode, the second reference frame template and the first reference frame template corresponding to the single reference frame prediction mode that is used to obtain the combined reference frame prediction mode are combined to obtain the target reference frame template corresponding to the combined reference frame prediction mode.

[0227] In an optional embodiment of the present application, the target reference frame template acquisition submodule is further used to:

[0228] Obtaining the identifier of the target single reference frame stored in each reference frame template;

[0229] The identifier of the target single reference frame is stored in a predetermined format to obtain a target reference frame template.

[0230] In an optional embodiment of the present application, the target combined reference frame determination module is specifically used to:

[0231] An identification set consisting of identifications of target single reference frames stored in a target reference frame template corresponding to each combined reference frame prediction mode is obtained, and a combined reference frame whose label corresponds to a single reference frame in each combined reference frame and belongs to the identification set is determined as the target combined reference frame corresponding to the combined reference frame prediction mode.

[0232] Based on the same principle, an embodiment of the present application further provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method provided in any optional embodiment of the present application is implemented, and specifically the following situations can be implemented:

[0233] Determine the frame type of the image frame where the current block is located; obtain the identifier of the reference frame of at least one adjacent block of the current block; use each single reference frame prediction mode to predict the current block respectively, and obtain the rate-distortion cost corresponding to each single reference frame prediction mode; based on the frame type, the identifier of the reference frame of each adjacent block and the rate-distortion cost corresponding to each single reference frame prediction mode, obtain the target reference frame template corresponding to each combined reference frame prediction mode, wherein each target reference frame template stores the identifier of at least two target single reference frames, and the at least two target single reference frames are used to combine and form a target combined reference frame corresponding to the corresponding combined reference frame prediction mode; based on the target reference frame template corresponding to each combined reference frame prediction mode, determine the corresponding target combined reference frame when using each combined reference frame prediction mode to predict the current block.

[0234] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the method shown in any embodiment of the present application is implemented.

[0235] It is understandable that what is stored in the medium may be a computer program corresponding to the interactive method in video playback.

[0236] Fig.10 A schematic diagram of the structure of an electronic device applicable to an embodiment of the present application is shown in FIG. Fig.10 As shown, Fig.10 The electronic device 1000 shown includes: a processor 1001 and a memory 1003. The processor 1001 and the memory 1003 are connected, such as through a bus 1002. Further, the electronic device 1000 may also include a transceiver 1004, and the electronic device 1000 may interact with other electronic devices through the transceiver 1004. It should be noted that in actual applications, the transceiver 1004 is not limited to one, and the structure of the electronic device 1000 does not constitute a limitation on the embodiments of the present application.

[0237] The processor 1001 is used in the embodiment of the present application to implement Fig. 9 The function of the reference frame determination device is shown.

[0238] Processor 1001 may be a CPU, a general-purpose processor, a DSP, an ASIC, an FPGA or other programmable logic device, a transistor logic device, a hardware component or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 1001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0239] The bus 1002 may include a path to transmit information between the above components. The bus 1002 may be a PCI bus or an EISA bus, etc. The bus 1002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.10 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0240] The memory 1003 may be a ROM or other type of static storage device that can store static information and instructions, a RAM or other type of dynamic storage device that can store information and instructions, or an EEPROM, a CD-ROM or other optical disk storage, an optical disk storage (including a compressed optical disk, a laser disk, an optical disk, a digital versatile disk, a Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.

[0241] The memory 1003 is used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 1001. The processor 1001 is used to execute the application code stored in the memory 1003 to implement Fig. 9 The illustrated embodiment provides actions of a device for determining a reference frame.

[0242] The embodiment of the present application provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that when the computer device executes the computer instructions, the following conditions are achieved:

[0243] Determine the frame type of the image frame where the current block is located; obtain the identifier of the reference frame of at least one adjacent block of the current block; use each single reference frame prediction mode to predict the current block respectively, and obtain the rate-distortion cost corresponding to each single reference frame prediction mode; based on the frame type, the identifier of the reference frame of each adjacent block and the rate-distortion cost corresponding to each single reference frame prediction mode, obtain the target reference frame template corresponding to each combined reference frame prediction mode, wherein each target reference frame template stores the identifier of at least two target single reference frames, and the at least two target single reference frames are used to combine and form a target combined reference frame corresponding to the corresponding combined reference frame prediction mode; based on the target reference frame template corresponding to each combined reference frame prediction mode, determine the corresponding target combined reference frame when using each combined reference frame prediction mode to predict the current block.

[0244] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0245] The above are only some implementation methods of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for determining a reference frame, characterized in that: include: Determine the frame type of the image frame where the current block is located; Obtaining an identifier of a reference frame of at least one adjacent block of the current block; Predicting the current block respectively by using each single reference frame prediction mode, and obtaining a rate distortion cost corresponding to each single reference frame prediction mode; Based on the frame type, the identifiers of the reference frames of each adjacent block and the rate-distortion cost corresponding to each single reference frame prediction mode, a target reference frame template corresponding to each combined reference frame prediction mode is obtained, including: taking the single reference frame corresponding to the minimum rate-distortion cost corresponding to each single reference frame prediction mode as the first optimal single reference frame, and obtaining the identifier of the first optimal single reference frame; storing the identifiers of the reference frames of each adjacent block and the identifier of the first optimal single reference frame in a predetermined form to obtain a first reference frame template; based on the frame type and the rate-distortion cost corresponding to each single reference frame prediction mode, a second reference frame template corresponding to each single reference frame prediction mode is obtained; based on the frame type, a third reference frame template is obtained; based on the first reference frame template, each second reference frame template and the third reference frame template, a target reference frame template corresponding to each combined reference frame mode is obtained; wherein each target reference frame template stores the identifiers of at least two target single reference frames, and the at least two target single reference frames are used to combine and form a target combined reference frame corresponding to the corresponding combined reference frame prediction mode; Based on the target reference frame templates corresponding to the combined reference frame prediction modes, a target combined reference frame corresponding to when the current block is predicted by using the combined reference frame prediction modes is determined.

2. The method according to claim 1, characterized in that The method further comprises: Taking the single reference frame corresponding to the minimum rate-distortion cost corresponding to the specified single reference frame prediction mode as the second optimal single reference frame, and obtaining an identifier of the second optimal single reference frame; The step of storing the identifiers of the reference frames of the adjacent blocks and the identifier of the first optimal single reference frame in a predetermined format to obtain the first reference frame template includes: The identifiers of the reference frames of the adjacent blocks, the identifier of the first optimal single reference frame, and the identifier of the second optimal single reference frame are stored in a predetermined format to obtain the first reference frame template.

3. The method according to claim 1, characterized in that The acquiring, based on the frame type and the rate-distortion cost corresponding to each single reference frame prediction mode, a second reference frame template corresponding to each single reference frame prediction mode comprises: Obtaining a corresponding weight level based on the frame type; Based on the weight level and the minimum rate-distortion cost corresponding to each single reference frame prediction mode, obtaining a corresponding judgment threshold; A single reference frame corresponding to a rate-distortion cost less than the judgment threshold among the rate-distortion costs corresponding to each single reference frame prediction mode is obtained, and an identifier of the single reference frame is stored in a predetermined form to obtain a second reference frame template corresponding to the single reference frame prediction mode.

4. The method according to claim 3, characterized in that The obtaining a corresponding judgment threshold based on the weight level and the minimum rate-distortion cost corresponding to each single reference frame prediction mode includes: Obtaining the parameter value corresponding to the weight level from a preset correspondence table, wherein the preset correspondence table stores the correspondence between each weight level and each parameter value; The parameter value is multiplied by the minimum rate-distortion cost to obtain the judgment threshold.

5. The method according to claim 1, characterized in that The acquiring a third reference frame template based on the frame type includes: Obtaining a corresponding weight level based on the frame type; If the weight level is less than or equal to the preset level, the first number of specified single reference frames are stored in a preset form to obtain the third reference frame template; if the weight level is greater than the preset level, the second number of specified single reference frames are stored in a preset form to obtain the third reference frame template, wherein the first number is greater than the second number.

6. The method according to claim 1, characterized in that The acquiring, based on the first reference frame template, each second reference frame template and the third reference frame template, a target reference frame template corresponding to each combined reference frame mode comprises: The single reference frame prediction mode corresponding to the minimum rate distortion cost is used as the optimal single reference frame mode; For each combined reference frame prediction mode, if the combined reference frame prediction mode is obtained by combining only the optimal single reference frame mode, combining the second reference frame template, the first reference frame template and the third reference frame template corresponding to the optimal single reference frame mode to obtain a target reference frame template corresponding to the combined reference frame prediction mode; and If the combined reference frame prediction mode is not obtained by combining only the optimal single reference frame mode, the second reference frame template corresponding to the single reference frame prediction mode that is used to obtain the combined reference frame prediction mode is combined with the first reference frame template to obtain the target reference frame template corresponding to the combined reference frame prediction mode.

7. The method according to claim 6, characterized in that The target reference frame template is obtained by combining the reference frame templates, including: Obtaining the identifier of the target single reference frame stored in each reference frame template; The identifier of the target single reference frame is stored in a predetermined format to obtain the target reference frame template.

8. The method according to claim 1, characterized in that: The determining, based on the target reference frame templates corresponding to the combined reference frame prediction modes, the target combined reference frame corresponding to when the current block is predicted using the combined reference frame prediction modes, comprises: An identifier set consisting of identifiers of target single reference frames stored in a target reference frame template corresponding to each combined reference frame prediction mode is obtained, and a combined reference frame whose label corresponds to a single reference frame in each combined reference frame belongs to the identifier set is determined as the target combined reference frame corresponding to the combined reference frame prediction mode.

9. A device for determining a reference frame, characterized in that: include: A frame type determination module, used to determine the frame type of the image frame where the current block is located; A module for obtaining an identifier of a reference frame of an adjacent block, used to obtain an identifier of a reference frame of at least one adjacent block of the current block; A single reference frame prediction module, used to predict the current block using each single reference frame prediction mode, and obtain a rate distortion cost corresponding to each single reference frame prediction mode; The target reference frame template acquisition module is used to acquire the target reference frame template corresponding to each combined reference frame prediction mode based on the frame type, the identifier of the reference frame of each adjacent block and the rate-distortion cost corresponding to each single reference frame prediction mode, including: taking the single reference frame corresponding to the minimum rate-distortion cost corresponding to each single reference frame prediction mode as the first optimal single reference frame, and acquiring the identifier of the first optimal single reference frame; storing the identifier of the reference frame of each adjacent block and the identifier of the first optimal single reference frame in a predetermined form to obtain a first reference frame template; acquiring the second reference frame template corresponding to each single reference frame prediction mode based on the frame type and the rate-distortion cost corresponding to each single reference frame prediction mode; acquiring the third reference frame template based on the frame type; acquiring the target reference frame template corresponding to each combined reference frame mode based on the first reference frame template, each second reference frame template and the third reference frame template; wherein each target reference frame template stores the identifiers of at least two target single reference frames, and the at least two target single reference frames are used to combine and form the target combined reference frame corresponding to the corresponding combined reference frame prediction mode; The target combined reference frame determination module is used to determine the target combined reference frame corresponding to each combined reference frame prediction mode when predicting the current block based on the target reference frame template corresponding to each combined reference frame prediction mode.

10. An electronic device, characterized in that: including memory and processor; The memory stores a computer program; The processor is configured to execute the computer program to implement the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images