Video encoding method and apparatus, computer program, and electronic device.

Abstract

A video encoding method performed by an electronic device, wherein N adjacent blocks of the current block in the current video frame have been encoded, and the encoding scheme of each of the N adjacent blocks is different from backward predictive encoding, the first number of adjacent blocks for which forward predictive encoding has been performed within the N adjacent blocks is statistically determined, where N is a positive integer of 2 or more, and the first number is greater than 0, and the forward predictive encoding cost of the current block is determined using a reference frame set, and the forward predictive encoding cost of the current block is less than a predetermined target cost threshold corresponding to the first number, Step (S304) is to determine the coding cost used to perform backward predictive coding on the current block using frames, and to determine the minimum coding cost among the determined coding costs used to perform backward predictive coding as the backward predictive coding cost of the current block, wherein the forward predictive coding cost is the minimum coding cost among the coding costs used to perform forward predictive coding on the current block using each reference frame in the reference frame set, and to determine the coding scheme of the current block based on the relationship between the forward predictive coding cost and the backward predictive coding cost.

Description

TECHNICAL FIELD

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application claims priority to the Chinese patent application filed on October 19, 2023, with application number 202311372122.9 and entitled "Video Coding Method and Apparatus Thereof, Storage Medium, and Electronic Device", the entire content of which is incorporated into this application by reference.

[0002] The present application relates to the field of computers, and specifically, to a video coding method and apparatus thereof, a storage medium, and an electronic device.

BACKGROUND ART

[0003] In the related art, when encoding a video frame, an encoder often needs to sequentially try various predictive coding modes. For example, skip coding is tried first, then merge coding is tried, then inter coding (interframe coding) is tried, and finally intra coding (intraframe predictive coding) is tried. In particular, when the encoder tries inter coding, it often tries forward predictive coding first, then backward predictive coding, and finally bidirectional predictive coding. After the encoder tries to encode the video frame using all predictive coding modes, the coding scheme with the minimum coding cost is selected as the final coding scheme for the video frame.

[0004] In this approach, on the one hand, when the encoder attempts backward predictive coding, it typically needs to scan all available reference frames. For example, the parent block, child blocks, left neighboring blocks, upper neighboring blocks, upper left neighboring blocks, and the backward reference frames used in the current optimal mode (e.g., merge coding or skip coding) are often considered candidate reference frames for backward predictive coding of the current block, which can be time-consuming. On the other hand, when attempting backward predictive coding, existing algorithms have a 54.21% probability of encountering the worst-case scenario, namely scanning all available reference frames, but ultimately selecting forward predictive coding from all predictive coding modes (e.g., forward predictive coding, backward predictive coding, bidirectional predictive coding, etc.) with an 87.27% probability. This means that despite wasting a lot of time attempting backward predictive coding, the encoder ultimately selects forward predictive coding to encode the video frame, resulting in little advantage in compression ratio and low video coding efficiency.

[0005] Currently, no effective solution has been proposed to address the aforementioned problem of inefficient video encoding. [Overview of the Initiative]

[0006] This application provides a video encoding method and apparatus therefor, a storage medium, and electronic equipment.

[0007] According to one aspect of the embodiments of this application, a video encoding method is provided which is performed by an electronic device, and the method is Steps include: 1. In a video frame, where N adjacent blocks of the current block are currently encoded, and the encoding scheme of each of the N adjacent blocks is different from backward predictive encoding, a first number of adjacent blocks in the N adjacent blocks for which forward predictive encoding has been performed, where N is a positive integer of 2 or more; Steps of determining the coding cost used to perform the backward predictive coding on the current block using each reference frame in the reference frame set, where the first quantity is greater than 0, the forward predictive coding cost of the current block is determined using a reference frame set, and the forward predictive coding cost of the current block is less than a predetermined target cost threshold corresponding to the first quantity, and the minimum coding cost among the determined coding costs used to perform the backward predictive coding is determined as the backward predictive coding cost of the current block, wherein the forward predictive coding cost is the minimum coding cost among the coding costs used to perform the forward predictive coding on the current block using each reference frame in the reference frame set, The process includes the step of determining the coding scheme for the current block based on the relationship between the forward predictive coding cost and the backward predictive coding cost.

[0008] According to another aspect of the embodiments of this application, a video encoding apparatus is provided, the apparatus is A forward predictive coding adjacent block statistics unit configured to statistically determine a first number of adjacent blocks in the N adjacent blocks for which forward predictive coding has been performed, where N is a positive integer of 2 or more, in the case where N adjacent blocks of the current block in the current video frame have been coded, and the coding scheme of each of the N adjacent blocks is different from backward predictive coding, the forward predictive coding adjacent block statistics unit, A backward predictive coding cost determination unit configured to determine the coding cost used to perform backward predictive coding on the current block using each reference frame in the reference frame set when the first quantity is greater than 0, the forward predictive coding cost of the current block is determined using a reference frame set, and the forward predictive coding cost of the current block is less than a predetermined target cost threshold corresponding to the first quantity, and to determine the minimum coding cost among the determined coding costs used to perform backward predictive coding as the backward predictive coding cost of the current block, wherein the forward predictive coding cost is the minimum coding cost among the coding costs used to perform forward predictive coding on the current block using each reference frame in the reference frame set, The system includes a first coding scheme determination unit configured to determine the coding scheme for the current block based on the relationship between the forward predictive coding cost and the backward predictive coding cost.

[0009] A video encoding method performed by an electronic device, A step of determining a second number of adjacent blocks in the N adjacent blocks for which backward predictive coding has been performed, where N is a positive integer of 2 or more, given that N adjacent blocks in the current video frame have already been coded, and the coding scheme of each of the N adjacent blocks is different from forward predictive coding. Steps of determining the coding cost used to perform forward predictive coding on the current block using each reference frame in the reference frame set, where the second quantity is greater than 0, the backward predictive coding cost of the current block is determined using a reference frame set, and the backward predictive coding cost of the current block is less than a predetermined target cost threshold corresponding to the second quantity, and determining the minimum coding cost among the determined coding costs used to perform forward predictive coding as the forward predictive coding cost of the current block, wherein the backward predictive coding cost is the minimum coding cost among the coding costs used to perform backward predictive coding on the current block using each reference frame in the reference frame set, The process includes the step of determining the coding scheme for the current block based on the relationship between the backward predictive coding cost and the forward predictive coding cost.

[0010] A video encoding device, the device, A backward predictive coding adjacent block statistics unit configured to determine a second number of adjacent blocks for which backward predictive coding has been performed, in the case where N adjacent blocks of the current block in the current video frame have been coded, and the coding scheme of each of the N adjacent blocks is different from forward predictive coding, wherein N is a positive integer of 2 or more, and A forward predictive coding cost determination unit configured to determine the coding cost used to perform the forward predictive coding on the current block using each reference frame in the reference frame set when the second quantity is greater than 0, the backward predictive coding cost of the current block is determined using a reference frame set, and the backward predictive coding cost of the current block is less than a predetermined target cost threshold corresponding to the second quantity, and to determine the minimum coding cost among the determined coding costs used to perform the forward predictive coding as the forward predictive coding cost of the current block, wherein the backward predictive coding cost is the minimum coding cost among the coding costs used to perform the backward predictive coding on the current block using each reference frame in the reference frame set, The system includes a second coding scheme determination unit configured to determine the coding scheme for the current block based on the relationship between the backward predictive coding cost and the forward predictive coding cost.

[0011] According to yet another aspect of the embodiments of this application, a computer-readable storage medium is further provided, wherein a computer program is stored in the computer-readable storage medium, and the computer program is configured to perform the video encoding method described above at runtime.

[0012] According to yet another aspect of the embodiments of this application, a computer program product or computer program is provided, the computer program product or computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor causes the electronic device to perform the video encoding method described above by executing the computer instructions.

[0013] According to yet another aspect of the embodiments of this application, an electronic device is provided, the electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to perform the video encoding method described above by executing the computer program.

[0014] Details of one or more embodiments of this application are shown in the following drawings and description. Other features, purposes, and advantages of this application will become apparent from the specification, drawings, and claims. [Brief explanation of the drawing]

[0015] [Figure 1] Figure 1 is a schematic diagram illustrating an application scenario of one selectable video encoding method according to an embodiment of this application. [Figure 2] Figure 2 is a schematic diagram illustrating an application scenario of one selectable video encoding method according to an embodiment of this application. [Figure 3] This is a flowchart of one selectable video encoding method according to an embodiment of the present application. [Figure 4] This is a schematic diagram showing the correspondence between one selectable CU size and CU depth according to an embodiment of this application. [Figure 5] This is a schematic diagram showing one selectable encoded information according to an embodiment of the present application. [Figure 6] This is a schematic diagram illustrating the determination of one selectable cost threshold according to an embodiment of the present application. [Figure 7] This is a schematic diagram showing one selectable encoding cost and cumulative number of iterations according to an embodiment of this application. [Figure 8] Figure 1 is a schematic diagram showing the determination and statistics of one selectable encoding information and count set according to an embodiment of this application. [Figure 9] Figure 2 is a schematic diagram showing the determination and statistics of one selectable encoding information and count set according to an embodiment of this application. [Figure 10] This is a schematic diagram showing one selectable flag setting according to an embodiment of this application. [Figure 11] FIG. 2 is a schematic diagram illustrating an optional coding method for an adjacent block according to an embodiment of the present application. [Figure 12] FIG. 3 is a schematic diagram illustrating an optional reference frame set according to an embodiment of the present application. [Figure 13] FIG. 4 is a schematic diagram illustrating an optional determination of a reference frame permitted to be used for performing forward predictive coding on a current block according to an embodiment of the present application. [Figure 14] FIG. 5 is a schematic diagram illustrating an optional determination of a coding method for a current block according to an embodiment of the present application. [Figure 15] FIG. 6 is a schematic diagram illustrating an optional adjacent block of a current block according to an embodiment of the present application. [Figure 16] FIG. 7 is a schematic diagram illustrating an optional determination of a current block according to an embodiment of the present application. [Figure 17] FIG. 8 is a schematic diagram of an optional video coding method according to an embodiment of the present application. [Figure 18] FIG. 9 is a schematic diagram illustrating an optional determination of a forward predictive coding cost and a backward predictive coding cost according to an embodiment of the present application. [Figure 19] FIG. 10 is a schematic diagram illustrating the configuration of an optional video coding apparatus according to an embodiment of the present application. [Figure 20] FIG. 11 is a schematic diagram illustrating the configuration of an optional electronic device according to an embodiment of the present application. [Figure 21] FIG. 12 is a block diagram illustrating the computer system configuration of an optional electronic device according to an embodiment of the present application. [Description of Embodiments]

[0016] To explain the technical solutions in the embodiments of the present application or the prior art more clearly, the drawings used in the description of the embodiments or the prior art have been briefly introduced above. Obviously, the above drawings are only part of the embodiments of the present application, and those skilled in the art can also obtain other drawings based on the published drawings without creative efforts.

[0017] The technical solutions in the embodiments of this application will be described clearly and completely below with reference to the drawings in the embodiments of this application. Obviously, the embodiments described are only a selection of embodiments of this application, not all embodiments. All other embodiments that can be obtained by a person skilled in the art without creative effort based on the embodiments of this application are included in the scope of protection of this application.

[0018] Furthermore, terms such as “First,” “Second,” etc., in the specification, claims, and drawings of this application are used to distinguish similar subjects and do not limit any particular order or sequence. It should be understood that the data used in this manner is interchangeable under appropriate circumstances, thereby allowing the embodiments of this application described herein to be carried out in an order other than those illustrated or described herein. Also, the terms “includes,” “has,” and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus including a series of steps or units is not limited to those explicitly listed steps or units, and may include other steps or units not explicitly listed or specific to those processes, methods, products, or apparatus.

[0019] First, let's explain the terminology used in the embodiments of this application.

[0020] Regarding coding units, parent blocks, child blocks, and coding unit depth (cuDepth): In the H265 standard, a 64x64 CTU (Coding Tree Unit) can be divided into four 32x32 CUs (Coding Units), each 32x32 CU can be divided into four 16x16 CUs, and each 16x16 CU can be divided into four 8x8 CUs. Assuming the current size of a CU is 16x16, its four 8x8 CUs are its child blocks, and for the four 8x8 CUs, the original 16x16 CU is their parent block. CuDepth = 6 - (log2(cu size)), and the cuDepth of the 64x64, 32x32, 16x16, and 8x8 CUs are 0, 1, 2, and 3, respectively. Here, cu can have the same meaning as CU.

[0021] I-frame (also called Intra picture or keyframe): An intra-frame predictive coding frame, consisting only of macroblocks of intra-frame predictions, and coded using only the information of this frame without referencing other frames.

[0022] P-frame (Predictive frame): This frame uses a motion prediction method to perform inter-frame predictive coding, utilizing the previous I-frame or P-frame.

[0023] B-frames (Bi-directional frame interpolated prediction frames): These offer the highest compression ratio and require both the previous image frame (e.g., I-frame or P-frame) and the subsequent image frame (e.g., P-frame), and perform bidirectional predictive coding between frames using a motion prediction scheme.

[0024] According to one aspect of the embodiments of this application, a video encoding method is provided, which, optionally as one possible embodiment, can be applied to the hardware environment shown in Figure 1, but is not limited thereto.

[0025] As shown in Figure 1, the terminal device 102 and the server 104 can communicate via the network, and either the terminal device 102 or the server 104 can encode video frames. Taking the case where the server optionally encodes video frames as an example, an encoder 104-1 is implemented on the server, and if the terminal device 102 needs to encode video frames, it can encode them via the encoder 104-1 implemented on the server 104.

[0026] As shown in Figure 2, the current video may include multiple video frames, each having a frame number, for example, video frames numbered 1 to 64. Here, the current video frame may be a single video frame to be encoded in the current video, for example, the current video frame may be a video frame with frame number 16. The current block in the current video frame may be a block to be encoded in the current video frame, for example, the current block is block 3. The adjacent blocks of block 3 may include, but are not limited to, the adjacent block located to the left of block 3 (i.e., block 4), the adjacent block located above block 3 (i.e., block 2), the adjacent block located to the upper left of block 3 (i.e., block 1), the adjacent block located to the right of block 3, and the adjacent block located to the upper right of block 3. That is, the adjacent blocks of block 3 may include, but are not limited to, block 1, block 2, and block 4. This application allows the current block in the current video frame to be encoded by the following steps. In this application, the orientation of a block, for example, the top, left, right, upper left, lower left, upper right, and lower right of the block, is determined with respect to the block when the block or the video frame in which the block is located is viewed from the front.

[0027] First, assuming that Block 1, Block 2, and Block 4, which are adjacent blocks to Block 3, are already encoded, and that the encoding schemes of Block 1, Block 2, and Block 4 are all different from backward predictive coding, we statistically calculate the first quantity of adjacent blocks among Block 1, Block 2, and Block 4 for which forward predictive coding has been performed. Optionally, in this embodiment, the encoding schemes of Block 1 and Block 2 are forward predictive coding, and the encoding scheme of Block 4 is intraframe predictive coding, so the first quantity is understood to be 2.

[0028] Next, if the first quantity is greater than 0, and the forward predictive coding cost of block 3 is determined using the reference frame set, and the forward predictive coding cost of block 3 is less than a predetermined target cost threshold corresponding to the first quantity, the coding cost used to perform backward predictive coding on block 3 using each reference frame in the reference frame set is determined, and the minimum coding cost among the determined coding costs used to perform backward predictive coding is determined as the backward predictive coding cost of block 3. Here, the forward predictive coding cost is the minimum coding cost among the coding costs used to perform forward predictive coding on block 3 using each reference frame in the reference frame set.

[0029] As one possible example, the reference frame set may include, but is not limited to, frames 11, 12, 13, 14, and 15. If the coding cost used to perform forward predictive coding on block 3 using frame 11 is cost 1, the coding cost used to perform forward predictive coding on block 3 using frame 12 is cost 2, the coding cost used to perform forward predictive coding on block 3 using frame 13 is cost 3, the coding cost used to perform forward predictive coding on block 3 using frame 14 is cost 4, and the coding cost used to perform forward predictive coding on block 3 using frame 15 is cost 5, then the forward predictive coding cost is the smallest of cost 1, cost 2, cost 3, cost 4, and cost 5, which is cost 4. As one possible example, if the coding cost used to perform backward predictive coding on block 3 using frame 11 is cost 6, the coding cost used to perform backward predictive coding on block 3 using frame 12 is cost 7, the coding cost used to perform backward predictive coding on block 3 using frame 13 is cost 8, the coding cost used to perform backward predictive coding on block 3 using frame 14 is cost 9, and the coding cost used to perform backward predictive coding on block 3 using frame 15 is cost 10, then the backward predictive coding cost is the smallest of cost 6, cost 7, cost 8, cost 9, and cost 10, which is cost 10.

[0030] Then, based on cost 4 and cost 10, the encoding scheme for block 3 is determined.

[0031] In this embodiment, the example described included blocks located to the left, upper left, and above block 3 as adjacent blocks to block 3. However, in reality, adjacent blocks to block 3 may include, but are not limited to, blocks located to the left, right, above, below, upper left, upper right, lower left, and lower right of block 3.

[0032] Using the method described above, if there are multiple encoded neighboring blocks in the current block's adjacent blocks, and the encoding scheme of each neighboring block is different from backward predictive encoding, and if there are neighboring blocks in the multiple encoded neighboring blocks of the current block that have undergone forward predictive encoding, and the forward predictive encoding cost of the current block is smaller than the cost threshold corresponding to the number of neighboring blocks in the multiple encoded neighboring blocks that have undergone forward predictive encoding, then the set of reference frames used to determine the forward predictive encoding cost of the current block is used as the set of reference frames to be used for backward predictive encoding of the current block. This method eliminates the need to search all available reference frames when performing backward predictive encoding on the current block. For example, it avoids using the parent block, child blocks, left neighboring blocks, upper neighboring blocks, upper-left neighboring blocks, and the backward reference frames used in the current optimal mode (e.g., merge encoding or skip encoding) as candidate reference frames for backward predictive encoding of the current block, thereby reducing the time required to determine the set of reference frames to be used for backward predictive encoding on the current block.

[0033] Furthermore, the minimum coding cost used to perform backward predictive coding on the current block using each reference frame in the reference frame set can be directly determined as the backward predictive coding cost of the current block. This reduces the time required to determine the backward predictive coding cost of the current block and improves the efficiency of determining the backward predictive coding cost of the current block.

[0034] Furthermore, the coding scheme for the current block can be determined directly based on the forward and backward predictive coding costs of the current block, avoiding the process of determining the coding scheme by sequentially trying various predictive coding schemes. For example, it avoids selecting the scheme with the minimum coding cost as the final predictive coding scheme for the current block after all predictive coding schemes have been tried, such as trying forward predictive coding followed by backward predictive coding.

[0035] This reduces the time required to determine the encoding scheme for the current block, improves the efficiency of determining the encoding scheme for the current block, and thereby achieves the technical effect of improving the efficiency of video encoding, thus solving the technical problem of low video encoding efficiency.

[0036] Optionally, in this embodiment, the terminal device may include, but is not limited to, at least one of the following: mobile phones (Android phones, iOS phones, etc.), notebook computers, tablet computers, personal digital assistants (PDAs), MIDs (Mobile Internet Devices), PADs, desktop computers, and smart televisions. The target client may be a video client, an instant messaging client, a browser client, an educational client, etc. The network may include, but is not limited to, a wired network and a wireless network, wherein the wired network includes a local area network, a metropolitan area network, and a wide area network, and the wireless network includes Bluetooth®, Wi-Fi, and other networks that enable wireless communication. The server may be a single server, a server cluster consisting of multiple servers, or a cloud server. The above is merely an example, and this embodiment is not limited thereto.

[0037] Optionally, the video encoding methods in each embodiment of this application may be performed by a terminal device or a server, but are not limited to these. For example, the video encoding methods in each embodiment of this application may be performed by an encoder in a terminal device or an encoder in a server. As one possible example, some steps of the video encoding methods in each embodiment of this application, for example, step S302 in Figure 3, or steps S302, S304, may be performed by a server (e.g., an encoder in the server), and other steps of the video encoding methods in each embodiment of this application, for example, steps S304, S306 in Figure 3, or step S306, may be performed by a terminal device (e.g., an encoder in the terminal device), but are not limited to these.

[0038] As an optional, possible embodiment, Figure 3 is a flowchart of one possible video encoding method according to an embodiment of the present application. As shown in Figure 3, the flow of the video encoding method may include the following steps:

[0039] In step S302, if N adjacent blocks of the current block in the current video frame have been encoded, and the encoding scheme of each of the N adjacent blocks is different from backward predictive encoding, then a first number of adjacent blocks in the N adjacent blocks for which forward predictive encoding has been performed is statistically determined, where N is a positive integer of 2 or greater.

[0040] Optionally, encoding can be performed by encoding video frames one by one according to the encoding order of the video frames. The video frame currently being encoded is called the current video frame. When encoding the current video frame, the current video frame is divided into multiple blocks, and each block is encoded one by one. The block currently being encoded is called the current block. Here, the encoding order of the video frames can be determined using the timestamp or frame number of the video frame. The order in which blocks are encoded within a single video frame can be determined based on the position of the block within that video frame. Specifically, the encoding can start from the block at the corner of the video frame, scan each block row by row and column by column, and encode each scanned block.

[0041] Optionally, in this embodiment, a block within a video frame may be, but is not limited to, a divided encoding unit within the video frame. Some blocks in the current video frame may already be encoded, and the current block may be, but is not limited to, a block that has not yet been encoded within the current video frame. Adjacent blocks may include, but are not limited to, blocks adjacent to the current block's position in multiple directions within the current video frame. Multiple directions may include, but are not limited to, up, down, left, right, upper left, upper right, lower left, lower right, etc. For example, an adjacent block may include, but is not limited to, a block located to the left of the current block within the current video frame, or a block located above the current block within the current video frame, or a block located to the upper left of the current block within the current video frame. N adjacent blocks may be, but are not limited to, at least some of the adjacent blocks of the current block. The number of adjacent blocks of different blocks may be the same or different, and the range of the number of adjacent blocks of different blocks may be the same or different, but is not limited to these. For example, the maximum number of adjacent blocks for block a and block b is 3, and the minimum number is 0 for both.

[0042] Optionally, in this embodiment, the block coding scheme may include, but is not limited to, intraframe predictive coding or interpredictive coding. Based on the direction of the reference frame used for coding, interpredictive coding may include, but is not limited to, forward predictive coding, backward predictive coding, and bidirectional predictive coding.

[0043] As an example, interpredictive coding involves using adjacent coded video frames to find the best matching block for the current block, subtracting the pixel value of each pixel point in the current block from the pixel value of each pixel point in the current block to obtain the residual value for each pixel point in the current block, and then determining the coded data to be obtained by performing interpredictive coding on the current block based on the residual value for each pixel point in the current block. For example, transformation and quantization processes are performed on the residual value of each pixel point in the current block to obtain coded data obtained by performing interpredictive coding on the current block.

[0044] For example, when performing forward predictive coding on the current block within the current video frame, the permitted reference frame may include, but is not limited to, one or more video frames coded before the current video frame. Video frames coded before the current video frame may include, but are not limited to, video frames whose frame number is earlier than the frame number of the current video frame, and / or video frames whose frame number is later than the frame number of the current video frame.

[0045] When performing backward predictive coding on the current block within the current video frame, the permitted reference frame may include, but is not limited to, one or more video frames coded before the current video frame. Video frames coded before the current video frame may include, but is not limited to, video frames whose frame number is later than the frame number of the current video frame, and / or video frames whose frame number is earlier than the frame number of the current video frame.

[0046] Currently, when performing bidirectional predictive coding on the current block within a video frame, both the previous and subsequent video frames are required, and bidirectional predictive coding between frames is performed using a motion prediction method. For example, frame B uses the previous I or P frame and the subsequent P frame as reference frames, and determines the residual value of each pixel point of the current block in frame B, and two motion vectors corresponding to the current block (for example, the motion vector between the current block in frame B and the best-matching block in the previous reference frame, and the motion vector between the current block in frame B and the best-matching block in the subsequent reference frame).

[0047] Optionally, in this embodiment, if N adjacent blocks of the current block within the current video frame are already encoded, and the encoding scheme of each of the N adjacent blocks is different from backward predictive encoding, then it can be understood that the encoding scheme of each of the N adjacent blocks of the current block may not be backward predictive encoding, but another encoding scheme, such as forward predictive encoding, bidirectional predictive encoding, or intra-frame predictive encoding. In this case, the number of adjacent blocks among the N adjacent blocks that have undergone forward predictive encoding can be obtained, but is not limited to this.

[0048] In step S304, if the first quantity is greater than 0, and the forward predictive coding cost of the current block is determined using the reference frame set, and the forward predictive coding cost of the current block is less than a predetermined target cost threshold corresponding to the first quantity, the coding cost used to perform backward predictive coding on the current block is determined using each reference frame in the reference frame set, and the minimum coding cost among the determined coding costs used to perform backward predictive coding is determined as the backward predictive coding cost of the current block, where the forward predictive coding cost is the minimum coding cost among the coding costs used to perform forward predictive coding on the current block using each reference frame in the reference frame set.

[0049] Optionally, in this embodiment, the coding cost used to perform forward predictive coding on the current block using each reference frame in the reference frame set may include, but is not limited to, the computational resources and / or computation time spent when performing forward predictive coding on the current block using each reference frame in the reference frame set. The magnitude of the coding cost depends on several factors, including the complexity of the data, the complexity of the coding, the computational resources required for the coding process, and the coding speed requirements. Furthermore, the coding cost is also affected by hardware device constraints and performance requirements.

[0050] In this embodiment, however, one may optionally perform forward predictive coding on the current block using each reference frame in the reference frame set, obtain multiple coding costs (the quantity of coding costs is the same as the number of reference frames in the reference frame set), and determine the smallest coding cost among the multiple coding costs as the forward predictive coding cost of the current block.

[0051] Optionally, in this embodiment, if the first quantity is greater than 0 and the forward predictive coding cost of the current block is smaller than a predetermined target cost threshold corresponding to the first quantity, it indicates that the forward predictive coding cost of the current block is relatively small, and a reference frame in the reference frame set is determined to be the reference frame for performing backward predictive coding on the current block. This means that both the reference frame for performing forward predictive coding and backward predictive coding on the current block are reference frames within the reference frame set. This method eliminates the need to search all available reference frames when performing backward predictive coding, as in related technologies, significantly reducing the time required to determine the reference frame for performing backward predictive coding on the current block and improving the efficiency of determining the reference frame for performing backward predictive coding on the current block.

[0052] Optionally, in this embodiment, the forward predictive coding cost is the minimum coding cost among those used to perform forward predictive coding on the current block using each reference frame in the reference frame set. This method reduces the computational resources consumed when performing forward predictive coding on the current block and improves the utilization of computational resources.

[0053] Optionally, in this embodiment, the smallest coding cost among those used to perform backward predictive coding on the current block using each reference frame in the reference frame set may be obtained and used as the backward predictive coding cost of the current block, but is not limited to this. Specifically, backward predictive coding is performed on the current block using each reference frame in the reference frame set one by one, multiple coding costs (the quantity of coding costs is the same as the number of reference frames) are obtained, and the smallest coding cost among the multiple coding costs is determined as the backward predictive coding cost of the current block. This method reduces the number of reference frames that need to be searched when performing backward predictive coding on the current block, and reduces the computational resources consumed when performing backward predictive coding on the current block, thereby improving the utilization rate of computational resources.

[0054] To better understand the process for determining forward and backward predictive coding costs in the embodiments of this application, the process for determining forward and backward predictive coding costs in the embodiments of this application will be described below with reference to exemplary embodiments, although this is applicable to, but not limited to, the embodiments of this application.

[0055] As shown in Figure 18, the current video may, but is not limited to, video frames having multiple frame numbers. For example, it may include video frames numbered 1 through 64. Here, the current video frame may be one video frame to be encoded in the current video, for example, the video frame with frame number 16 (also called frame 16), but is not limited to that. The current block in the current video frame may be the block to be encoded in the current video frame, for example, the current block may be block 3, but is not limited to that. The adjacent blocks of block 3 may include, but are not limited to, adjacent blocks located to the left of block 3, adjacent blocks located above block 3, adjacent blocks located to the upper left, adjacent blocks located to the right, and adjacent blocks located to the upper right, for example, block 1, block 2, and block 4. The N adjacent blocks of block 3 may include at least two of the following adjacent blocks: the adjacent block located to the left of block 3, the adjacent block located above, the adjacent block located to the upper left, the adjacent block located to the right, and the adjacent block located to the upper right. For example, the N adjacent blocks of block 3 may include, but are not limited to, block 1, block 2, and block 4. Block 3 can be encoded by, but are not limited to, the following steps.

[0056] When the adjacent blocks of block 3, namely block 1, block 2, and block 4, are already encoded, and the encoding schemes of block 1, block 2, and block 4 are all different from backward predictive coding, the first number of adjacent blocks among block 1, block 2, and block 4 that have undergone forward predictive coding is determined. Optionally, in this embodiment, the encoding schemes of block 1 and block 2 are forward predictive coding, and the encoding scheme of block 4 is intraframe predictive coding, so the first number is understood to be 2.

[0057] When the first quantity is greater than 0, and the forward predictive coding cost of block 3 is determined using the reference frame set, and the forward predictive coding cost of block 3 is less than a predetermined target cost threshold corresponding to the first quantity, the coding cost used to perform backward predictive coding on block 3 using each reference frame in the reference frame set is determined, and the minimum coding cost among the determined coding costs used to perform backward predictive coding is determined as the backward predictive coding cost of block 3.

[0058] For example, when performing forward predictive coding on block 3, the permitted reference frame set may include, but is not limited to, video frames numbered 11 through 15 (also referred to as frames 11 through 15). To improve coding efficiency, forward predictive coding can be performed using only some of the reference frames within the permitted reference frame set, for example, using frames 13 through 15 to perform forward predictive coding on block 3. It is understood that the reference frame set includes some of the reference frames within the permitted reference frame set, for example, frames 13 through 15. If the coding cost used to perform forward predictive coding on block 3 using frame 13 is cost 1, the coding cost used to perform forward predictive coding on block 3 using frame 14 is cost 2, and the coding cost used to perform forward predictive coding on block 3 using frame 15 is cost 3, then cost 3 can be determined as the forward predictive coding cost for block 3, but is not limited to this. To make it clear, the reference frame set for performing backward predictive coding on block 3 may include, but is not limited to, frames 13 through 15.

[0059] In this situation, the use of frames 13 to 15 for backward predictive coding of block 3 does not need to be limited. If the coding cost used to perform backward predictive coding of block 3 using frame 13 is cost 4, the coding cost used to perform backward predictive coding of block 3 using frame 14 is cost 5, and the coding cost used to perform backward predictive coding of block 3 using frame 15 is cost 6, and cost 6 is the smallest coding cost among cost 4, cost 5, and cost 6, then cost 6 can be determined as the backward predictive coding cost of block 3, but is not limited to this.

[0060] In this embodiment, the example described was that the adjacent blocks to block 3 include blocks located to the left, upper left, and above block 3. However, in reality, the adjacent blocks to block 3 may include, but are not limited to, blocks located to the left, right, above, below, upper left, upper right, lower left, and lower right of block 3.

[0061] In step S306, the coding scheme for the current block is determined based on the relationship between the forward predictive coding cost and the backward predictive coding cost.

[0062] Optionally, in this embodiment, the encoding scheme for the current block can be determined based on the relationship between the forward predictive coding cost and the backward predictive coding cost, but is not limited to this. This method avoids the need in related technologies to try forward predictive coding, backward predictive coding, and bidirectional predictive coding one by one, and then select the scheme with the minimum coding cost from these three as the encoding scheme for the current block. Instead, by directly determining the encoding scheme for the current block based on the relationship between the forward predictive coding cost and the backward predictive coding cost, the time required to determine the encoding scheme for the current block is significantly reduced, and the efficiency of determining the encoding scheme for the current block is improved.

[0063] In this embodiment, the encoding scheme for the current block can be optionally determined based on the relationship between the forward predictive encoding cost and the backward predictive encoding cost, and the current block can be encoded using the determined encoding scheme, but this is not limited to this. This method significantly reduces the time required to determine the encoding scheme for the current block, as well as the time required for the entire process of encoding the current block, thereby significantly improving the encoding efficiency of the current block.

[0064] Using the method described above, if there are multiple encoded neighboring blocks in the current block's adjacent blocks, and the encoding scheme of each neighboring block is different from backward predictive encoding, and if there are neighboring blocks in the multiple encoded neighboring blocks of the current block that have undergone forward predictive encoding, and the forward predictive encoding cost of the current block is smaller than the cost threshold corresponding to the number of neighboring blocks in the multiple encoded neighboring blocks that have undergone forward predictive encoding, then the set of reference frames used to determine the forward predictive encoding cost of the current block is used as the set of reference frames to be used for backward predictive encoding of the current block. This method eliminates the need to search all available reference frames when performing backward predictive encoding on the current block. For example, it avoids using the parent block, child blocks, left neighboring blocks, upper neighboring blocks, upper-left neighboring blocks, and the backward reference frames used in the current optimal mode (e.g., merge encoding or skip encoding) as candidate reference frames for backward predictive encoding of the current block, thereby reducing the time required to determine the set of reference frames to be used for backward predictive encoding on the current block.

[0065] Furthermore, the minimum coding cost used to perform backward predictive coding on the current block using each reference frame in the reference frame set can be directly determined as the backward predictive coding cost of the current block. This reduces the time required to determine the backward predictive coding cost of the current block and improves the efficiency of determining the backward predictive coding cost of the current block.

[0066] Furthermore, the coding scheme for the current block can be determined directly based on the forward and backward predictive coding costs of the current block, avoiding the process of determining the coding scheme by sequentially trying various predictive coding schemes. For example, it avoids selecting the scheme with the minimum coding cost as the final predictive coding scheme for the current block after all predictive coding schemes have been tried, such as trying forward predictive coding followed by backward predictive coding.

[0067] This reduces the time required to determine the encoding scheme for the current block, improves the efficiency of determining the encoding scheme for the current block, and thereby achieves the technical effect of improving the efficiency of video encoding, thus solving the technical problem of low video encoding efficiency.

[0068] One possible approach is to determine the target cost threshold using the following method, but it is not limited to this.

[0069] In S11, a target cost threshold is determined in a predetermined cost threshold set, corresponding to a first quantity and the size of the current block. Here, each cost threshold in the cost threshold set corresponds to one quantity in the quantity set and one size in the size set, where the quantity set includes N positive integers from 1 to N, the first quantity is the quantity in the quantity set, the size set includes the sizes of different blocks that are permitted to be encoded in the current video frame, and the size of the current block is the size in the size set.

[0070] Optionally, in this embodiment, the quantity set can be used to record the quantities of adjacent blocks in which forward predictive coding has been performed within the adjacent blocks of a block, but is not limited thereto. Each cost threshold in the cost threshold set corresponds to one quantity in the quantity set and one size in the size set. It can be seen that in the cost threshold set, cost threshold 1 corresponding to quantity 1 in the quantity set and size 1 in the size set is different from cost threshold 2 corresponding to quantity 1 in the quantity set and size 2 in the size set. Also, cost threshold 3 corresponding to quantity 2 in the quantity set and size 1 in the size set is different from cost threshold 4 corresponding to quantity 1 in the quantity set and size 1 in the size set.

[0071] Optionally, in this embodiment, the size set may include, but is not limited to, different block sizes that are permitted to be encoded in the current video frame. For example, block sizes may include, but are not limited to, 64x64, 32x32, 16x16, and 8x8. As shown in Figure 4, there may be a correspondence between the permitted block size and the CU depth. For example, if the size of block a that is permitted to be encoded in the current video frame is 128x128, the correspondence between the CU size (also called the block size) and the CU depth may include, but is not limited to, that the CU size is 64x64 when the depth is 0, the CU size is 32x32 when the depth is 1, the CU size is 16x16 when the depth is 2, and the CU size is 8x8 when the depth is 3.

[0072] This method enables the comprehensive determination of the corresponding cost threshold based on the different block sizes currently permitted for encoding in a video frame, and the number of adjacent blocks where forward predictive encoding has been performed within the adjacent blocks of a block, thereby improving the rationality of the determined cost threshold.

[0073] One possible approach is to determine the cost threshold set by the following method, but it is not limited to this.

[0074] In S21, a set of video frames in the sample video is encoded, and the encoding information generated by the encoding of the set of video frames is obtained. Here, the encoding information includes a set of cumulative encoding costs and a set of counts, where each cumulative encoding cost in the set of cumulative encoding costs corresponds to one quantity in the set of quantities and one size in the set of sizes, and each count in the set of counts corresponds to one cumulative encoding cost in the set of cumulative encoding costs. The (i,j)th cumulative encoding cost in the set of cumulative encoding costs corresponds to the i-th quantity in the set of quantities and the j-th size in the set of sizes, and also corresponds to the (i,j)th count in the set of counts. The (i,j)th count represents the number of times the target encoding event occurred in the encoding process of the set of video frames, where i and j are both positive integers greater than or equal to 1. A target coding event refers to a situation in a set of video frames where N adjacent blocks of a single block have already been coded, the coding scheme of each of those N adjacent blocks is different from backward predictive coding, the size of that block is the j-th size, and the number of adjacent blocks that have undergone forward predictive coding among the N adjacent blocks of that block is the i-th quantity. In this case, forward predictive coding is performed on that single block. The (i,j)-th cumulative coding cost is the sum of the forward predictive coding costs of a single block in the (i,j)-th occurrence of target coding events.

[0075] In S22, the cost threshold set is determined based on the encoding information.

[0076] To better understand the method for determining the cost threshold set in the embodiments of this application, the process for determining the cost threshold set in the embodiments of this application will be described below with reference to selectable embodiments.

[0077] As shown in Figure 5, the quantity set may include 1, 2, and 3, and the size set may include, but is not limited to, 64x64, 32x32, 16x16, and 8x8. The coding information may include, but is not limited to, the cumulative coding cost set and the count set. The cumulative coding cost set may include, but is not limited to, g_cost[n=1][cu=64x64] (also called cumulative coding cost), g_cost[n=1][cu=32x32], g_cost[n=1][cu=16x16], g_cost[n=1][cu=8x8], g_cost[n=2][cu=64x64], g_cost[n=2][cu=32x32], g_cost[n=2][cu=16x16], g_cost[n=2][cu=8x8], g_cost[n=3][cu=64x64], g_cost[n=3][cu=32x32], g_cost[n=3][cu=16x16], and g_cost[n=3][cu=8x8].

[0078] Here, g_cost[n=1][cu=64x64] corresponds to the case where the quantity in the quantity set is 1 and the size in the size set is 64x64. g_cost[n=1][cu=32x32] corresponds to the case where the quantity in the quantity set is 1 and the size in the size set is 32x32. g_cost[n=1][cu=16x16] corresponds to the case where the quantity in the quantity set is 1 and the size in the size set is 16x16, and g_cost[n=1][cu=8x8] corresponds to the case where the quantity in the quantity set is 1 and the size in the size set is 8x8. g_cost[n=2][cu=64x64] corresponds to the case where the quantity in the quantity set is 2 and the size in the size set is 64x64. g_cost[n=2][cu=32x32] corresponds to the case where the quantity in the quantity set is 2 and the size in the size set is 32x32. g_cost[n=2][cu=16x16] corresponds to the case where the quantity in the quantity set is 2 and the size in the size set is 16x16, and g_cost[n=2][cu=8x8] corresponds to the case where the quantity in the quantity set is 2 and the size in the size set is 8x8. g_cost[n=3][cu=64x64] corresponds to the case where the quantity in the quantity set is 3 and the size in the size set is 64x64. g_cost[n=3][cu=32x32] corresponds to the case where the quantity in the quantity set is 3 and the size in the size set is 32x32. g_cost[n=3][cu=16x16] corresponds to the case where the quantity in the quantity set is 3 and the size in the size set is 16x16, and g_cost[n=3][cu=8x8] corresponds to the case where the quantity in the quantity set is 3 and the size in the size set is 8x8.

[0079] g_cost[n=1][cu=64x64] corresponds to g_count[n=1][cu=64x64] in the count set, g_cost[n=1][cu=32x32] corresponds to g_count[n=1][cu=32x32] in the count set, g_cost[n=1][cu=16x16] corresponds to g_count[n=1][cu=16x16] in the count set, and g_cost[n=1][cu=8x8] corresponds to g_count[n=1][cu=8x8] in the count set.

[0080] g_cost[n=2][cu=64x64] corresponds to g_count[n=2][cu=64x64] in the count set, g_cost[n=2][cu=32x32] corresponds to g_count[n=2][cu=32x32] in the count set, g_cost[n=2][cu=16x16] corresponds to g_count[n=2][cu=16x16] in the count set, and g_cost[n=2][cu=8x8] corresponds to g_count[n=2][cu=8x8] in the count set.

[0081] g_cost[n=3][cu=64x64] corresponds to g_count[n=3][cu=64x64] in the count set, g_cost[n=3][cu=32x32] corresponds to g_count[n=3][cu=32x32] in the count set, g_cost[n=3][cu=16x16] corresponds to g_count[n=3][cu=16x16] in the count set, and g_cost[n=3][cu=8x8] corresponds to g_count[n=3][cu=8x8] in the count set.

[0082] Here, g_count[n=1][cu=64x64] represents the number of times a target coding event occurred during the coding process of a set of video frames. A target coding event refers to performing forward predictive coding on a block when N adjacent blocks of a block in a set of video frames have already been coded, the coding scheme of each of the N adjacent blocks of that block is different from backward predictive coding, the size of that block is 64x64, and the number of adjacent blocks that have undergone forward predictive coding among the N adjacent blocks of that block is 1.

[0083] g_count[n=1][cu=32x32] represents the number of times a target coding event occurred during the encoding process of a set of video frames. A target coding event occurs when N adjacent blocks of a block in a set of video frames have already been coded, the coding scheme of each of the N adjacent blocks of that block is different from backward predictive coding, the size of that block is 32x32, and the number of adjacent blocks that have undergone forward predictive coding among the N adjacent blocks of that block is 1. In this case, forward predictive coding is performed on that block. g_count[n=1][cu=16x16] represents the number of times a target coding event occurred during the encoding process of a set of video frames. A target coding event refers to performing forward predictive coding on a block when, in a set of video frames, N adjacent blocks of that block have been coded, the coding scheme of each of those N adjacent blocks is different from backward predictive coding, the size of that block is 16x16, and the number of adjacent blocks that have undergone forward predictive coding among the N adjacent blocks of that block is 1. g_count[n=1][cu=8x8] represents the number of times a target coding event occurred in the coding process of a set of video frames. A target coding event refers to performing forward predictive coding on a block when, in a set of video frames, N adjacent blocks of that block have been coded, the coding scheme of each of those N adjacent blocks is different from backward predictive coding, the size of that block is 8x8, and the number of adjacent blocks that have undergone forward predictive coding among the N adjacent blocks of that block is 1.

[0084] g_count[n=2][cu=64x64] represents the number of times a target coding event occurred during the encoding process of a set of video frames. A target coding event occurs when N adjacent blocks of a block in a set of video frames have already been coded, the coding scheme of each of the N adjacent blocks of that block is different from backward predictive coding, the size of that block is 64x64, and the number of adjacent blocks that have undergone forward predictive coding among the N adjacent blocks of that block is 2, in which case forward predictive coding should be performed on that block. g_count[n=2][cu=32x32] represents the number of times a target coding event occurred during the encoding process of a set of video frames. A target coding event refers to performing forward predictive coding on a block when, in a set of video frames, N adjacent blocks of that block have been coded, the coding scheme of each of those N adjacent blocks is different from backward predictive coding, the size of that block is 32x32, and the number of adjacent blocks that have undergone forward predictive coding out of the N adjacent blocks of that block is 2. g_count[n=2][cu=16x16] represents the number of times a target coding event occurred in the coding process of a set of video frames. A target coding event refers to performing forward predictive coding on a block when, in a set of video frames, N adjacent blocks of that block have been coded, the coding scheme of each of those N adjacent blocks is different from backward predictive coding, the size of that block is 16x16, and the number of adjacent blocks that have undergone forward predictive coding out of the N adjacent blocks of that block is 2. g_count[n=2][cu=8x8] represents the number of times the target coding event occurred during the coding process of a set of video frames.A target coding event refers to performing forward predictive coding on a block when, in a set of video frames, N adjacent blocks of a single block have already been coded, the coding scheme of each of those N adjacent blocks is different from backward predictive coding, the size of the block is 8x8, and the number of adjacent blocks that have undergone forward predictive coding among the N adjacent blocks of that block is 2.

[0085] g_count[n=3][cu=64x64] represents the number of times a target coding event occurred during the encoding process of a set of video frames. A target coding event occurs when N adjacent blocks of a block in a set of video frames have already been coded, the coding scheme of each of the N adjacent blocks of that block is different from backward predictive coding, the size of that block is 64x64, and the number of adjacent blocks that have undergone forward predictive coding among the N adjacent blocks of that block is 3, in which case forward predictive coding should be performed on that block. g_count[n=3][cu=32x32] represents the number of times a target coding event occurred during the encoding process of a set of video frames. A target coding event refers to performing forward predictive coding on a block when, in a set of video frames, N adjacent blocks of that block have been coded, the coding scheme of each of those N adjacent blocks is different from backward predictive coding, the size of that block is 32x32, and the number of adjacent blocks that have undergone forward predictive coding out of the N adjacent blocks of that block is 3. g_count[n=3][cu=16x16] represents the number of times a target coding event occurred in the coding process of a set of video frames. A target coding event refers to performing forward predictive coding on a block when, in a set of video frames, N adjacent blocks of that block have been coded, the coding scheme of each of those N adjacent blocks is different from backward predictive coding, the size of that block is 16x16, and the number of adjacent blocks that have undergone forward predictive coding out of the N adjacent blocks of that block is 3. g_count[n=3][cu=8x8] represents the number of times the target coding event occurred during the coding process of a set of video frames.A target coding event refers to performing forward predictive coding on a block when, in a set of video frames, N adjacent blocks of that block have already been coded, the coding scheme of each of those N adjacent blocks is different from backward predictive coding, the size of that block is 8x8, and the number of adjacent blocks that have undergone forward predictive coding among the N adjacent blocks of that block is 3.

[0086] This method allows for the classification of cost thresholds in a cost threshold set according to the number of adjacent blocks that have undergone forward predictive coding within a block's adjacent blocks and the size of the block, thereby improving the correlation between the cost threshold and the block size and the coding scheme of the adjacent blocks.

[0087] One possible approach is to determine the (i,j)th cost threshold in the cost threshold set by the following steps, but not limited to these: where the (i,j)th cost threshold corresponds to the i-th quantity in the quantity set and the j-th size in the size set.

[0088] In S31, the (i,j)th cost threshold is determined to be equal to the value obtained by dividing the (i,j)th cumulative coding cost by the (i,j)th count.

[0089] Optionally, in this embodiment, the (i,j)th cost threshold can be determined to be equal to the value obtained by dividing the (i,j)th cumulative coding cost by the (i,j)th count, but is not limited to this. As shown in Figure 6, g_thr[n=1][cu=32x32] (also called the cost threshold) can be determined to be the value obtained by dividing g_cost[n=1][cu=32x32] by g_count[n=1][cu=32x32], but is not limited to this. Here, g_thr[n=1][cu=32x32] is the cost threshold corresponding to the i-th quantity being 1 (which can be understood as n being equal to 1) and the j-th size being 32x32, and g_thr[n=1][cu=32x32] can be understood to be the (1st,32x32)th cost threshold.

[0090] This method allows for the determination of cost thresholds in a cost threshold set based on the cumulative encoding cost set and the count set in the encoded information, accurately classifying cost thresholds within the cost threshold set and improving the accuracy of cost threshold determination.

[0091] As one possible approach, in the encoding process of a set of video frames in a sample video, the method further includes the following steps:

[0092] In S41, if N adjacent blocks of the k-th block in a set of video frames are encoded, and the encoding scheme of each of the N adjacent blocks of the k-th block is different from backward predictive encoding, and the size of the k-th block is the j-th size, and the number of adjacent blocks of the k-th block that have undergone forward predictive encoding is the i-th quantity, then the forward predictive encoding cost and backward predictive encoding cost of the k-th block are obtained, where k is a positive integer greater than or equal to 1.

[0093] In S42, if it is determined that the coding scheme for the k-th block is forward predictive coding based on the forward predictive coding cost and backward predictive coding cost of the k-th block, the forward predictive coding cost of the k-th block is added to the (i,j)-th cumulative coding cost, and the (i,j)-th count is incremented by 1.

[0094] Optionally, in this embodiment, if the forward predictive coding cost of the k-th block is smaller than the backward predictive coding cost, the coding scheme for the k-th block is determined to be forward predictive coding. This scheme reduces the coding cost used for coding the k-th block and improves the coding efficiency of the k-th block.

[0095] .

[0096] The forward predictive coding cost and backward predictive coding cost of the k-th block are obtained. Based on the forward and backward predictive coding costs of the k-th block, if it is determined that the coding scheme of the k-th block is forward predictive coding, the forward predictive coding cost of the k-th block is accumulated in g_cost[n=2][cu=32x32] (also called the (2,32x32)th cumulative coding cost), and the count of g_count[n=2][cu=32x32] (also called the (2,32x32)th count) is incremented by 1. This method allows for real-time updates of the cumulative coding cost in the cumulative coding cost set and the count in the count set during the video frame coding process, enabling real-time updates of the cost threshold in the cost threshold set and improving the real-time performance of the cost threshold in the cost threshold set.

[0097] To better understand the process of determining the cost threshold set in the embodiments of this application, the process of determining the cost thresholds within the cost threshold set in the embodiments of this application will be described below with reference to select embodiments, which are applicable to but not limited to the embodiments of this application.

[0098] If inter prediction ultimately chooses forward prediction coding despite conventional algorithms in related technologies encountering worst-case scenarios, the embodiments of this application can, but are not limited to, accumulating the coding cost and counts in the count set for forward predictions according to the number of forward predictions n and CU depth used in adjacent blocks. At the end of coding, the average coding cost for different n and CU depths is calculated and used as the cost threshold for the combination of n and CU depth.

[0099] Specifically, as shown in Figure 8, when the encoder is started, the g_cost table (also called the cumulative coding cost set) and the g_count table (also called the count set) are initialized, and the coding costs and counts are accumulated, respectively. Specifically, this may include, but is not limited to, the following steps.

[0100] In step S801, the information is initialized. g_cost[n=4][4]={0} and g_count[n=4][4]={0} are initialized.

[0101] In step S802, the sample sequence is encoded.

[0102] In step S803, statistical information is obtained, and n=0.

[0103] In step S804, it is determined whether n is less than 4. If it is determined that n is less than 4, step S805 is executed. If it is determined that n is 4 or greater, the encoder is terminated.

[0104] In step S805, cuDepth is set to 0.

[0105] In step S806, it is determined whether cuDepth is less than 4. If it is determined that cuDepth is less than 4, step S807 is executed.

[0106] In step S807, the value obtained by g_thr[n=2][cuDepth]=g_cost[n=2][cuDepth] / g_count[n=2][cuDepth] is determined, and cuDepth++ is performed.

[0107] As shown in Figure 9, before trying different prediction methods for inter, the flag is set to 0. The flag is used to indicate whether the conventional algorithm has encountered a worst-case scenario (i.e., none of the adjacent blocks of the current block are using backward predictive coding, and all available reference frames must be scanned).

[0108] As shown in Figure 10, when the conventional algorithm encounters a worst-case scenario, the flag is set to 1, and the following steps may be included, but are not limited to.

[0109] In step S1001, the backreference frame list to be searched is initialized (i.e., list1=0).

[0110] In step S1002, the number of adjacent blocks to be statistically analyzed using forward-referencing frames is initialized (n=0). Specifically, before entering into the relevant fast algorithm, a variable is initialized (i.e., n=0), and this variable is used to indicate the number of adjacent blocks using forward prediction.

[0111] In step S1003, it is determined whether the processing of the parent block has been completed. If it is determined that it has been completed, step S1004 is executed.

[0112] In step S1004, it is determined whether or not a backreference frame was used for the parent block. If it is determined that a backreference frame was used for the parent block, step S1005 is executed.

[0113] In step S1005, the index of the backreference frame used for the parent block is obtained, and list1 = 1 << backreference frame index.

[0114] In step S1006, it is determined whether the processing of the child block is complete. If it is determined to be complete, step S1007 is executed. If it is determined to be incomplete, steps S1008 to S1011 are executed.

[0115] In step S1007, it is determined whether or not an adjacent block exists above. If it is determined that an adjacent block exists above, steps S1012 to S1014 are executed.

[0116] In step S1008, it is determined whether all child blocks have been scanned. If it is determined that all child blocks have been scanned, step S1007 is executed. If not all child blocks have been scanned, step S1009 is executed.

[0117] In step S1009, the next child block is obtained.

[0118] In step S1010, it is determined whether or not a backreference frame was used for the child block. If it is determined that a backreference frame was used for the child block, step S1011 is executed.

[0119] In step S1011, the index of the backreference frame used for the child block is obtained, and list1 = 1 << backreference frame index.

[0120] In step S1012, it is determined whether or not a backreference frame was used for the upper adjacent block. If it is determined that a backreference frame was used for the upper adjacent block, step S1013 is executed. If it is determined that a backreference frame was not used for the upper adjacent block, step S1014 is executed.

[0121] In step S1013, the backreference frame index used for the upper adjacent block is obtained, and list1 = 1 << backreference frame index.

[0122] In step S1014, it is determined whether or not a forward reference frame was used for the upper adjacent block. If it is determined that a forward reference frame was used for the upper adjacent block, n++ is performed.

[0123] In this embodiment, only the processing steps for adjacent blocks located above the current block (also called upper adjacent blocks) (for example, steps S1007 to S1014) have been described. However, in reality, the processing steps for adjacent blocks located to the left of the current block (also called left adjacent blocks) (for example, steps S1015 to S1018 shown in Figure 10) and adjacent blocks located to the upper left of the current block (also called upper left adjacent blocks) (for example, steps S1019 to S1022 shown in Figure 10) are the same as those for upper adjacent blocks. When processing left adjacent blocks, if a left adjacent block exists and forward predictive coding is used for the left adjacent block, n++ is performed. When processing upper left adjacent blocks, if an upper left adjacent block exists and forward predictive coding is used for the upper left adjacent block, n++ is performed. Further explanation is omitted here.

[0124] In step S1023, it is determined whether the current optimal mode uses backreference frames. If it is determined that the current optimal mode uses backreference frames, step S1024 is executed. If it is determined that the current optimal mode does not use backreference frames, step S1025 is executed.

[0125] In step S1024, the index of the backreference frame used by the current optimal mode is obtained, and list1 = 1 << backreference frame index.

[0126] In step S1025, it is determined whether or not any of the blocks use backreference frames. If it is determined that no of the blocks use backreference frames, step S1026 is executed. If it is determined that all of the blocks use backreference frames, the process terminates.

[0127] In step S1026, it is determined whether n != 0. If it is equal to 0, step S1019 is executed; otherwise, step S1020 is executed.

[0128] In step S1027, the flag is set to 1.

[0129] In step S1028, we set List1 = -1 and search for all reference frames backward.

[0130] As shown in Figure 9, forward predictive coding, backward predictive coding, and bidirectional predictive coding are performed on the block. After bidirectional predictive coding is completed, the coding costs are compared to select the optimal prediction mode. For example, based on an evaluation metric, the optimal search method is selected, and if flag is 1 and the optimal prediction mode is forward predictive coding, the coding cost of the optimal prediction mode is accumulated in g_cost[n][cuDepth] (where cuDepth is the depth of the CU), and at the same time, g_count[n][cuDepth] is incremented by 1.

[0131] After the entire sequence has been encoded, the values ​​obtained by g_cost[n][cuDepth] / g_count[n][cuDepth] for different combinations of n and cuDepth are output and used as the cost threshold for each combination of n and cuDepth. This cost threshold is then used as (g_thr[n][cuDepth]) in the optimized algorithm.

[0132] As one possible approach, before determining a first number of adjacent blocks in which forward predictive coding has been performed within N adjacent blocks, the method further includes the following steps:

[0133] In S51, each adjacent block of the N adjacent blocks is coded, where the coding scheme for each adjacent block of the N adjacent blocks is one of forward predictive coding and backward predictive coding, or one of forward predictive coding, backward predictive coding, and bidirectional predictive coding, or one of forward predictive coding, backward predictive coding, bidirectional predictive coding, and intraframe predictive coding.

[0134] As shown in Figure 11, taking the example of a video frame being frame number 16 and the current block being block 3 within that video frame, the adjacent blocks to block 3 include, but are not limited to, blocks 1, 2, and 4. In such a situation, the encoding method for each adjacent block of block 1, block 2, and block 4 may be one of forward predictive coding and backward predictive coding, or one of forward predictive coding, backward predictive coding, and bidirectional predictive coding, or one of forward predictive coding, backward predictive coding, bidirectional predictive coding, and intra-frame predictive coding. For example, the encoding method for block 4 may be backward predictive coding, bidirectional predictive coding, or intra-frame predictive coding.

[0135] This method enables encoding each adjacent block using multiple encoding schemes, and the encoding schemes for each adjacent block may be the same or different. This expands the range of selectable encoding schemes for encoding adjacent blocks and improves the diversity of encoding schemes for each adjacent block.

[0136] One possible approach is to determine the coding cost used to perform backward predictive coding on the current block using each reference frame in the reference frame set, before further including the following steps:

[0137] In S61, if there are M reference frames that are permitted to be used for forward predictive coding of the current block, some of the M reference frames are determined to be a set of reference frames, where M is a positive integer greater than or equal to 2.

[0138] To reduce coding time and improve coding efficiency, a set of reference frames may be formed by selecting some reference frames from those permitted to be used for forward predictive coding of the current block. Optionally, in this embodiment, the reference frames permitted to be used for forward predictive coding of the current block relate to an encoder. The encoder may include, but is not limited to, an H265 encoder, an H264 encoder, an AV1 encoder, or an H266 encoder.

[0139] As shown in Figure 12, the reference frames permitted for use in performing forward predictive coding on block 3 may include, but are not limited to, frames 8, 9, 10, 11, 12, 13, 14, and 15. In this situation, frames 11, 12, 13, 14, and 15 may be determined as the reference frame set, but are not limited to them, in order to reduce coding time and improve coding efficiency. This approach reduces the number of reference frames in the reference frame set, shortens the coding time, and thereby improves coding efficiency.

[0140] This method allows for the selection of reference frames from those permitted to be used for forward predictive coding of the current block, thereby creating a set of reference frames. This reduces the number of reference frames used when performing forward predictive coding on the current block, shortening the time required for forward predictive coding of the current block and improving its efficiency.

[0141] One possible approach is to determine a subset of the M reference frames as a set of reference frames using the following method, but this is not limited to it.

[0142] In S71, if the current video frame is a video frame transmitted in real time, and there are M reference frames permitted to be used for forward predictive coding of the current block, then some of the M reference frames are determined to be a set of reference frames.

[0143] Currently, when video frames are being transmitted in real time, for example, in screen sharing scenes of real-time online conferencing applications or real-time online communication applications, the similarity between preceding and succeeding video frames is very high. To ensure the real-time nature and smoothness of the video frames and to avoid problems such as lag, a faster encoding speed is usually required. In such situations, a subset of reference frames can be determined as a reference frame set from the reference frames permitted for use in forward predictive encoding of the current block. This method reduces the number of reference frames used when encoding the current block, improves the speed of real-time predictive encoding of video, and ensures the smoothness of video frames transmitted in real time.

[0144] One possible approach is that, before determining a subset of the M reference frames as a set of reference frames, the method further includes S81 or S82 below.

[0145] In S81, the M encoded video frames are determined to be M reference frames, where the frame numbers of the M video frames are smaller than the frame numbers of the current video frames.

[0146] In S82, P encoded video frames and Q encoded video frames are determined as M reference frames, where M is equal to the sum of P and Q, the frame numbers of the P video frames are smaller than the frame number of the current video frame, the frame numbers of the Q video frames are larger than the frame number of the current video frame, and P and Q are positive integers greater than or equal to 1.

[0147] Optionally, in this embodiment, the frame numbers of the reference frames permitted to be used for forward predictive coding of the current block may be, but are not limited to, smaller than the frame number of the current video frame. It can be understood that the reference frames permitted to be used for forward predictive coding of the current block are video frames coded before the current video frame. Alternatively, the frame numbers of the reference frames permitted to be used for forward predictive coding of the current block may be, but are not limited to, smaller than or larger than the frame number of the current video frame. It can be understood that the reference frames permitted to be used for forward predictive coding of the current block are video frames coded before the current video frame and video frames coded after the current video frame.

[0148] As shown in Figure 13, if the current video frame number is 16 (also known as frame 16), the reference frames permitted to be used for forward predictive coding of the current block within the current video frame may include, but are not limited to, the video frame number 8 (also known as frame 8), the video frame number 9 (also known as frame 9), the video frame number 10 (also known as frame 10), the video frame number 11 (also known as frame 11), the video frame number 12 (also known as frame 12), the video frame number 13 (also known as frame 13), the video frame number 14 (also known as frame 14), and the video frame number 15 (also known as frame 15). Here, frames 8, 9, 10, 11, 12, 13, 14, and 15 are all video frames coded before frame 16. Alternatively, the reference frames permitted for use in performing forward predictive coding on the current block within the current video frame may include, but are not limited to, frames 11, 12, 13, 14, 15, and the video frame number 18 (also known as frame 18), the video frame number 19 (also known as frame 19), and the video frame number 23 (also known as frame 23), where frames 11, 12, 13, 14, and 15 are video frames coded before frame 16, and frames 18, 19, and 23 are video frames coded after frame 16.

[0149] This method enables the determination of reference frames permitted to be used for forward predictive coding of the current block from encoded video frames located before and after the current video frame, thereby expanding the range of permitted reference frames and improving the coding quality in predictive coding of the current block.

[0150] One possible approach is to determine the encoding scheme for the current block using the following method, and then encode the current block using the determined encoding scheme, but this is not limited to this approach.

[0151] In S91, if the forward predictive coding cost is less than the backward predictive coding cost, the coding scheme for the current block is determined to be forward predictive coding; if the forward predictive coding cost is greater than the backward predictive coding cost, the coding scheme for the current block is determined to be backward predictive coding. In S92, if the forward predictive coding cost is less than the backward predictive coding cost and the bidirectional predictive coding cost of the current block, the coding scheme for the current block is determined to be forward predictive coding, where the bidirectional predictive coding cost is the minimum coding cost among those used to perform bidirectional predictive coding on the current block using each reference frame in the reference frame set. If the backward predictive coding cost is less than the forward predictive coding cost and the bidirectional predictive coding cost, the coding scheme for the current block is determined to be backward predictive coding, and if the bidirectional predictive coding cost is less than the forward predictive coding cost and the backward predictive coding cost, the coding scheme for the current block is determined to be bidirectional predictive coding.

[0152] To reduce coding costs, the coding method with the lower cost among forward predictive coding costs and backward predictive coding costs can be selected, but is not limited to this. As shown in Figure 14, the forward predictive coding cost of the current block may be, but is not limited to, cost 4, and the backward predictive coding cost of the current block may be, but is not limited to, cost 10. If cost 4 is less than cost 10, the coding method for the current block is determined to be forward predictive coding, and forward predictive coding is performed on the current block. If cost 4 is greater than cost 10, the coding method for the current block is determined to be backward predictive coding, and backward predictive coding is performed on the current block. This method significantly reduces the coding cost in coding the current block, and eliminates the need to scan through various coding methods such as forward predictive coding and backward predictive coding in related technologies to select the method with the lowest cost, thereby shortening the time required to determine the coding method for the current block and improving the efficiency of determining the coding method for the current block.

[0153] To improve the rationality of the encoding scheme for the current block, the encoding scheme for the current block can be determined from multiple encoding schemes, but is not limited to this. For example, if the forward predictive coding cost is smaller than the backward predictive coding cost and the bidirectional predictive coding cost of the current block, it indicates that the cost of performing forward predictive coding on the current block is minimized, and in such a situation, the encoding scheme for the current block is determined as forward predictive coding. If the bidirectional predictive coding cost is smaller than the forward predictive coding cost and the backward predictive coding cost, it indicates that the cost of performing bidirectional predictive coding on the current block is minimized, and in such a situation, the encoding scheme for the current block is determined as bidirectional predictive coding. This method improves the rationality of determining the encoding scheme for the current block by allowing it to be determined from a wider range of encoding schemes, and improves the efficiency of determining the encoding scheme for the current block by reducing the time required to determine it.

[0154] One possible approach is to determine the encoding scheme for the current block based on the relationship between the forward predictive coding cost and the backward predictive coding cost, and then encode the current block using the determined encoding scheme.

[0155] In S101, if the encoding scheme for the current block is determined to be forward predictive coding, forward predictive coding is performed on the current block. In S102, if the encoding scheme for the current block is determined to be backward predictive coding, backward predictive coding is performed on the current block. In S103, if the coding scheme for the current block is determined to be bidirectional predictive coding, bidirectional predictive coding is performed on the current block.

[0156] Optionally, in this embodiment, in order to improve coding efficiency, the coding scheme for the current block is determined based on the relationship between the forward predictive coding cost and the backward predictive coding cost, and then the current block is coded using the determined coding scheme. For example, if the forward predictive coding cost is smaller than the backward predictive coding cost, it indicates that the cost of performing forward predictive coding on the current block is minimized. In such a situation, the coding scheme for the current block is determined to be forward predictive coding, and forward predictive coding is performed on the current block. If the backward predictive coding cost is smaller than the forward predictive coding cost, it indicates that the cost of performing backward predictive coding on the current block is minimized. In such a situation, the coding scheme for the current block is determined to be backward predictive coding, and backward predictive coding is performed on the current block.

[0157] Alternatively, if the forward predictive coding cost is smaller than the backward predictive coding cost and the bidirectional predictive coding cost of the current block, it indicates that the cost of performing forward predictive coding on the current block is minimized. In this situation, the coding scheme for the current block is determined to be forward predictive coding, and forward predictive coding is performed on the current block. If the bidirectional predictive coding cost is smaller than the forward predictive coding cost and the backward predictive coding cost, it indicates that the cost of performing bidirectional predictive coding on the current block is minimized. In this situation, the coding scheme for the current block is determined to be bidirectional predictive coding, and bidirectional predictive coding is performed on the current block.

[0158] This method significantly reduces the coding cost used to encode the current block by encoding it using a coding scheme with lower coding costs, thereby reducing the computational resources required for encoding the current block and improving the utilization of those resources.

[0159] As one possible solution, the method further includes the following steps:

[0160] In S111, the N adjacent blocks include at least two of the following: an adjacent block located to the left of the current block in the current video frame, an adjacent block located above the current block in the current video frame, an adjacent block located to the upper left of the current block in the current video frame, an adjacent block located to the right of the current block in the current video frame, and an adjacent block located to the upper right of the current block in the current video frame.

[0161] As shown in Figure 15, the current video frame can be any video frame to be encoded in the current video, for example, the video frame number 16 (also called frame 16). The current block in the current video frame can be any block to be encoded in the current video frame, for example, the current block in frame 16 can be block 3. The adjacent blocks of block 3 may include adjacent blocks located to the left of block 3, adjacent blocks located above, adjacent blocks located to the upper left, adjacent blocks located to the right, adjacent blocks located to the upper right, and so on. For example, block 1 is an adjacent block located to the upper left of block 3 in frame 16, block 2 is an adjacent block located above block 3 in frame 16, block 4 is an adjacent block located to the left of block 3 in frame 16, block 9 is an adjacent block located to the upper right of block 3 in frame 16, and block 10 is an adjacent block located to the right of block 3 in frame 16. In this situation, the N adjacent blocks include at least two of the following: adjacent blocks located to the left of block 3, adjacent blocks located above, adjacent blocks located to the upper left, adjacent blocks located to the right, and adjacent blocks located to the upper right. For example, the N adjacent blocks include at least two of the following: blocks 1, 2, 4, 9, and 10. For example, the N adjacent blocks may include blocks 1 and 4, or blocks 1, 9, and 10, etc.

[0162] This method enables the determination of N adjacent blocks from blocks adjacent to the current block in the current video frame, such as to the left, right, upper left, and upper right. This expands the range of adjacent blocks used to determine N adjacent blocks, improving the efficiency of determining N adjacent blocks.

[0163] As one possible solution, the method further includes the following steps:

[0164] In S121, if N adjacent blocks of the current block in the current video frame have been encoded, and the encoding scheme of each of the N adjacent blocks is different from forward predictive encoding, then a second number of adjacent blocks in the N adjacent blocks for which backward predictive encoding has been performed is determined, where N is a positive integer greater than or equal to 2.

[0165] In S122, if the second quantity is greater than 0, and the backward predictive coding cost of the current block is determined using the reference frame set, and the backward predictive coding cost of the current block is less than a predetermined target cost threshold corresponding to the second quantity, the coding cost used to perform forward predictive coding on the current block using each reference frame in the reference frame set is determined, and the minimum coding cost among the determined coding costs used to perform forward predictive coding is determined as the forward predictive coding cost of the current block, where the backward predictive coding cost is the minimum coding cost among the coding costs used to perform backward predictive coding on the current block using each reference frame in the reference frame set.

[0166] In S123, the coding scheme for the current block is determined based on the relationship between the backward predictive coding cost and the forward predictive coding cost.

[0167] An encoder that adaptively selects whether to perform forward predictive coding or backward predictive coding first, or an encoder that fixedly performs backward predictive coding first and then forward predictive coding, can guide the coding scheme of the current block based on the adjacent blocks in the current block where backward predictive coding has been performed. As shown in Figure 16, the current video may be, but is not limited to, video frames containing multiple frame numbers (e.g., video frames numbered 1 to 64). The current video frame may also be a frame awaiting coding, for example, the video frame with frame number 16 (or also called frame 16), but is not limited to this. The current block may also be a block to be coded, for example, the current block may be block 3, but is not limited to this. The adjacent blocks of block 3 may include adjacent blocks located to the left of block 3, adjacent blocks located above, adjacent blocks located to the upper left, adjacent blocks located to the right, and adjacent blocks located to the upper right. For example, the adjacent blocks of block 3 may include, but are not limited to, blocks 1, 2, and 4. The N adjacent blocks of block 3 may include at least two of the following adjacent blocks: the adjacent block located to the left of block 3, the adjacent block located above, the adjacent block located to the upper left, the adjacent block located to the right, and the adjacent block located to the upper right. For example, the N adjacent blocks of block 3 may include, but are not limited to, block 1, block 2, and block 4. Block 3 can be encoded by, but are not limited to, the following steps.

[0168] In step S1601, if blocks 1, 2, and 4 are encoded, and the encoding schemes of blocks 1, 2, and 4 are all different from forward predictive coding, then the second number of adjacent blocks among blocks 1, 2, and 4 that have undergone backward predictive coding is determined. Optionally, in this embodiment, it can be understood that the encoding scheme of block 1 is backward predictive coding, the encoding scheme of block 2 is bidirectional predictive coding, the encoding scheme of block 4 is intraframe predictive coding, and the second number is 1.

[0169] In step S1602, if the second quantity is greater than 0, and the backward predictive coding cost of block 3 is determined using the reference frame set, and the backward predictive coding cost of block 3 is less than a predetermined target cost threshold corresponding to the second quantity, the coding cost used to perform forward predictive coding on block 3 using each reference frame in the reference frame set is determined, and the minimum coding cost among the determined coding costs used to perform forward predictive coding is determined as the forward predictive coding cost of block 3, where the backward predictive coding cost is the minimum coding cost among the coding costs used to perform backward predictive coding on block 3 using each reference frame in the reference frame set.

[0170] Optionally, this embodiment may include, but is not limited to, frames 17, 19, 20, 24, and 25. If the cost used when performing backward predictive coding on block 3 using frame 17 is cost 1', the cost used when performing backward predictive coding on block 3 using frame 19 is cost 2', the cost used when performing backward predictive coding on block 3 using frame 20 is cost 3', the cost used when performing backward predictive coding on block 3 using frame 24 is cost 4', and the cost used when performing backward predictive coding on block 3 using frame 25 is cost 5', then the backward predictive coding cost is the smallest of costs 1', 2', 3', 4', and 5', cost 5'.

[0171] Optionally, in this embodiment, if the cost used when performing forward predictive coding on block 3 using frame 17 is set to cost 6', the cost used when performing forward predictive coding on block 3 using frame 19 is set to cost 7', the cost used when performing forward predictive coding on block 3 using frame 20 is set to cost 8', the cost used when performing forward predictive coding on block 3 using frame 24 is set to cost 9', and the cost used when performing forward predictive coding on block 3 using frame 25 is set to cost 10', then the forward predictive coding cost is the smallest of costs 6', 7', 8', 9', and 10', which is cost 9'.

[0172] In step S1604, the encoding scheme for block 3 is determined based on costs 5' and 9', and block 3 is encoded using the determined encoding scheme.

[0173] The steps described above allow us to guide the coding scheme of the current block not only based on adjacent blocks where forward predictive coding has been performed, but also based on adjacent blocks where backward predictive coding has been performed, thereby increasing the flexibility in determining the coding scheme of the current block.

[0174] To better understand the video encoding method in the embodiments of this application, the video encoding method in the embodiments of this application will be described below with reference to selectable embodiments, which are applicable to but not limited to the embodiments of this application.

[0175] As shown in Figure 17, this application primarily optimizes the trigger conditions, threshold table generation, and how to select reference frames after triggering for the relevant high-speed algorithm. Here, the optimizations regarding the trigger conditions and the selection of reference frames after triggering correspond to the content shown in the dashed box in Figure 17. The main idea is to select candidate reference frames for backward predictive coding while simultaneously recording the use of forward predictive coding in the left adjacent block of the current block (also called the adjacent block located to the left of the current block in the current video frame), the upper adjacent block (also called the adjacent block located above the current block in the current video frame), and the upper-left adjacent block (also called the adjacent block located to the upper-left of the current block in the current video frame). If the existing algorithm encounters a worst-case scenario, and forward predictive coding is used for the adjacent block, and the forward predictive coding cost of the current block is less than the corresponding cost threshold, then backward predictive coding will use the same candidate reference frames as forward predictive coding. The video coding method in the embodiment of this application can be implemented by the following steps, but is not limited thereto.

[0176] In step S1701, the backreference frame list to be searched is initialized (list1=0).

[0177] In step S1702, the number of adjacent blocks to be statistically analyzed using forward-referencing frames is initialized (n=0). Specifically, before entering into the relevant fast algorithm, a variable is initialized (i.e., n=0), and this variable is used to indicate the number of adjacent blocks using forward prediction.

[0178] In step S1703, it is determined whether the processing of the parent block has been completed. If it is determined that it has been completed, step S1704 is executed.

[0179] In step S1704, it is determined whether or not a backreference frame was used for the parent block. If it is determined that a backreference frame was used for the parent block, step S1705 is executed.

[0180] In step S1705, the index of the backreference frame used for the parent block is obtained, and list1 = 1 << backreference frame index.

[0181] In step S1706, it is determined whether the processing of the child block is complete. If it is determined to be complete, step S1707 is executed. If it is determined to be incomplete, steps S1708 to S1711 are executed.

[0182] In step S1707, it is determined whether or not an adjacent block exists above. If it is determined that an adjacent block exists above, steps S1712 to S1714 are executed.

[0183] In step S1708, it is determined whether all child blocks have been scanned. If it is determined that all child blocks have been scanned, step S1707 is executed. If not all child blocks have been scanned, step S1709 is executed.

[0184] In step S1709, the next child block is obtained.

[0185] In step S1710, it is determined whether or not a backreference frame was used for the child block. If it is determined that a backreference frame was used for the child block, step S1711 is executed.

[0186] In step S1711, the index of the backreference frame used for the child block is obtained, and list1 = 1 << backreference frame index.

[0187] In step S1712, it is determined whether or not a backreference frame was used for the upper adjacent block. If it is determined that a backreference frame was used for the upper adjacent block, step S1713 is executed. If it is determined that a backreference frame was not used for the upper adjacent block, step S1714 is executed.

[0188] In step S1713, the backreference frame index used for the upper adjacent block is obtained, and list1 = 1 << backreference frame index.

[0189] If it is determined in step S1714 that a forward reference frame was used for the upper adjacent block, then n++ is performed.

[0190] In this embodiment, only the processing steps for adjacent blocks located above the current block (also called upper adjacent blocks) (for example, steps S1707 to S1714) have been described. However, in reality, the processing steps for adjacent blocks located to the left of the current block (also called left adjacent blocks) (for example, steps S1715 to S1718 shown in Figure 17) and adjacent blocks located to the upper left of the current block (also called upper left adjacent blocks) (for example, steps S1719 to S1722 shown in Figure 17) are the same as those for upper adjacent blocks. When processing left adjacent blocks, if a left adjacent block exists and forward predictive coding is used for the left adjacent block, n++ is performed. When processing upper left adjacent blocks, if an upper left adjacent block exists and forward predictive coding is used for the upper left adjacent block, n++ is performed. Further explanation is omitted here.

[0191] In step S1723, it is determined whether the current optimal mode uses backreference frames. If it is determined that the current optimal mode uses backreference frames, step S1724 is executed. If it is determined that the current optimal mode does not use backreference frames, step S1725 is executed.

[0192] In step S1724, the index of the backreference frame used by the current optimal mode is obtained, and list1 = 1 << backreference frame index.

[0193] In step S1725, it is determined whether or not any of the blocks use backreference frames. If it is determined that no of the blocks use backreference frames, step S1726 is executed. If it is determined that any of the blocks use backreference frames, the process terminates.

[0194] In step S1726, it is determined whether n!=0 and whether the forward search cost of the current block is less than the threshold (g_thr[n][cuDepth]). If it is determined to be "yes", step S1727 is executed; otherwise, step S1728 is executed.

[0195] In step S1727, during the backward search, only reference frames that have already been searched in the forward search are searched.

[0196] In step S1728, we set List1 = -1 and search for all backreference frames.

[0197] Furthermore, if the associated high-speed algorithm encounters a worst-case scenario (i.e., none of the adjacent blocks use backward predictive coding and it is necessary to scan all available reference frames), and there are adjacent blocks that use forward predictive coding (n!=0), and the forward predictive coding cost of the current block is less than the corresponding cost threshold g_thr[n][cuDepth], then candidate reference frames for forward prediction are used during backward prediction; otherwise, all reference frames are tried during backward prediction. To understand this, g_thr[n][cuDepth] is an identifier for the cost threshold of the block size corresponding to the CU depth of the current block, where n is the first quantity, and cuDepth represents the CU depth of the current block. CU sizes are optionally 64x64, 32x32, 16x16, and 8x8. Regarding the relationship between CU size and depth, when the CU depth is 0, the CU size is 64x64; when the CU depth is 1, the CU size is 32x32; when the CU depth is 2, the CU size is 16x16; and when the CU depth is 3, the CU size is 8x8.

[0198] The video coding method in the embodiment of this application speeds up the backward prediction coding process by reducing the number of reference frames tried in backward prediction. As shown in Table 1, explaining the optimization effect based on the algorithm's own metrics, when coding one sequence before and after algorithm optimization, the number of reference frames tried in backward prediction decreased from 4,162,260 to 3,463,050, the probability of algorithm failure (the algorithm reaching its worst-case scenario and trying all reference frames) decreased from 54.21% to 36.05%, and the probability of completely wasted processing (all backward reference frames being tried, but ultimately a forward reference frame being selected) when the algorithm fails decreased from 87.27% to 83.65%.

[0199] [Table 1] The video encoding method in the embodiment of this application significantly improved the performance of the algorithm. As shown in Table 2, when describing the performance of the algorithm in terms of the speed-up effect on the entire H265 low-latency encoder, after optimization, the overall processing speed of the encoder improved by 3.45%, the PSNR BD-rate loss was 0.30%, and the speed-up rate was 11.68.

[0200] [Table 2] For the sake of simplicity, the embodiments of each of the methods described above are all expressed as a combination of a series of operations. However, those skilled in the art should know that, according to this application, some steps may be performed in a different order or simultaneously, and therefore this application is not limited to the order of operations described. Furthermore, those skilled in the art should know that all embodiments described herein are preferred embodiments, and the relevant operations and modules are not necessarily required for this application.

[0201] According to another aspect of the embodiments of this application, an apparatus for performing the above-described video coding is further provided. As shown in Figure 19, the apparatus comprises a forward predictive coding adjacent block statistics unit 1902, a backward predictive coding cost determination unit 1904, and a first coding scheme determination unit 1906.

[0202] The forward predictive coding adjacent block statistics unit 1902 is configured to statistically determine a first number of adjacent blocks in the N adjacent blocks for which forward predictive coding has been performed, when the N adjacent blocks of the current block in the current video frame have been coded, and the coding scheme of each of the N adjacent blocks is different from the backward predictive coding, where N is a positive integer of 2 or greater.

[0203] The backward predictive coding cost determination unit 1904 is configured to determine the coding cost used to perform backward predictive coding on the current block using each reference frame in the reference frame set when the first quantity is greater than 0, the forward predictive coding cost of the current block is determined using the reference frame set, and the forward predictive coding cost of the current block is less than a predetermined target cost threshold corresponding to the first quantity, and to determine the minimum coding cost among the determined coding costs used to perform backward predictive coding as the backward predictive coding cost of the current block, where the forward predictive coding cost is the minimum coding cost among the coding costs used to perform forward predictive coding on the current block using each reference frame in the reference frame set.

[0204] The first coding scheme determination unit 1906 is configured to determine the coding scheme for the current block based on the relationship between the forward prediction coding cost and the backward prediction coding cost.

[0205] According to the embodiment of this application, when there are multiple encoded adjacent blocks in the current block's adjacent blocks, and the encoding scheme of each adjacent block is different from backward predictive encoding, if there are adjacent blocks in the multiple encoded adjacent blocks of the current block that have undergone forward predictive encoding, and the forward predictive encoding cost of the current block is smaller than a cost threshold corresponding to the number of adjacent blocks in the multiple encoded adjacent blocks that have undergone forward predictive encoding, then the set of reference frames used to determine the forward predictive encoding cost of the current block is used as the set of reference frames to be used for backward predictive encoding of the current block. This method eliminates the need to search all available reference frames when performing backward predictive encoding on the current block. For example, it avoids using the parent block, child blocks, left adjacent block, upper adjacent block, upper-left adjacent block, and the backward reference frames used in the current optimal mode (e.g., merge encoding or skip encoding) as candidate reference frames for backward predictive encoding of the current block, thereby reducing the time required to determine the set of reference frames to be used for backward predictive encoding on the current block.

[0206] Furthermore, the minimum coding cost used to perform backward predictive coding on the current block using each reference frame in the reference frame set can be directly determined as the backward predictive coding cost of the current block. This reduces the time required to determine the backward predictive coding cost of the current block and improves the efficiency of determining the backward predictive coding cost of the current block.

[0207] Furthermore, the coding scheme for the current block can be determined directly based on the forward and backward predictive coding costs of the current block, avoiding the process of determining the coding scheme by sequentially trying various predictive coding schemes. For example, it avoids selecting the scheme with the minimum coding cost as the final predictive coding scheme for the current block after all predictive coding schemes have been tried, such as trying forward predictive coding followed by backward predictive coding.

[0208] This reduces the time required to determine the encoding scheme for the current block, improves the efficiency of determining the encoding scheme for the current block, and thereby achieves the technical effect of improving the efficiency of video encoding, thus solving the technical problem of low video encoding efficiency.

[0209] As one possible solution, the device further includes a target cost threshold determination unit, The target cost threshold determination unit is configured to determine a target cost threshold in a given cost threshold set corresponding to a first quantity and the size of the current block, before determining the coding cost used to perform backward predictive coding on the current block using each reference frame in the reference frame set, where each cost threshold in the cost threshold set corresponds to one quantity in the quantity set and one size in the size set, where the quantity set includes N positive integers from 1 to N, the first quantity is the quantity in the quantity set, and the size set includes the sizes of different blocks that are allowed to be coded in the current video frame, the size of the current block is the size in the size set.

[0210] As one possible solution, the apparatus further comprises a processing unit and a cost threshold set determination unit, The processing unit is configured to encode a pair of video frames in a sample video and obtain encoding information generated by encoding the pair of video frames, before determining a target cost threshold corresponding to a first quantity and the size of the current block in a predetermined cost threshold set, where the encoding information includes a cumulative encoding cost set and a count set, where each cumulative encoding cost in the cumulative encoding cost set corresponds to one quantity in the quantity set and one size in the size set, and each count in the count set corresponds to one cumulative encoding cost in the cumulative encoding cost set. The (i,j)th cumulative encoding cost in the cumulative encoding cost set corresponds to the i-th quantity in the quantity set and the j-th size in the size set, and also corresponds to the (i,j)th count in the count set. The (i,j)th count represents the number of times the target encoding event occurred in the encoding process of the pair of video frames, where i and j are both positive integers greater than or equal to 1. A target coding event refers to a situation in a set of video frames where N adjacent blocks of a single block have already been coded, the coding scheme of each of those N adjacent blocks is different from backward predictive coding, the size of that block is the j-th size, and the number of adjacent blocks that have undergone forward predictive coding among the N adjacent blocks of that block is the i-th quantity. In this case, forward predictive coding is performed on that single block. The (i,j)-th cumulative coding cost is the sum of the forward predictive coding costs of a single block in the (i,j)-th occurrence of target coding events.

[0211] The cost threshold set determination unit is configured to determine the cost threshold set based on the encoded information.

[0212] One possible solution is for the cost threshold set decision unit to: The following steps are configured to determine the (i,j)th cost threshold in the cost threshold set, where the (i,j)th cost threshold corresponds to the i-th quantity in the quantity set and the j-th size in the size set. The aforementioned step includes determining the (i,j)th cost threshold to be equal to the value obtained by dividing the (i,j)th cumulative coding cost by the (i,j)th number of times.

[0213] As one possible solution, the device further comprises an acquisition unit and a cumulative unit, The acquisition unit is configured to acquire the forward predictive coding cost and the backward predictive coding cost of the k-th block in the encoding process of a set of video frames in a sample video, where k is a positive integer greater than or equal to 1, if N adjacent blocks of the k-th block in the set of video frames have been encoded, and the encoding scheme of each of the N adjacent blocks of the k-th block is different from backward predictive coding, and the size of the k-th block is the j-th size, and the number of adjacent blocks of the k-th block that have undergone forward predictive coding is the i-th quantity, where k is a positive integer greater than or equal to 1.

[0214] The cumulative unit is configured such that, based on the forward predictive coding cost and backward predictive coding cost of the k-th block, if it determines that the coding scheme of the k-th block is forward predictive coding, it adds the forward predictive coding cost of the k-th block to the (i,j)-th cumulative coding cost and increments the (i,j)-th count by 1.

[0215] As one possible solution, the device further comprises an encoding module configured to encode each adjacent block of N adjacent blocks before determining a first number of adjacent blocks in which forward predictive coding has been performed, wherein the encoding scheme for each adjacent block of N adjacent blocks is one of forward predictive coding and backward predictive coding, or one of forward predictive coding, backward predictive coding, and bidirectional predictive coding, or one of forward predictive coding, backward predictive coding, bidirectional predictive coding, and intraframe predictive coding.

[0216] As one possible solution, the device further comprises a reference frame set determination unit, The reference frame set determination unit is configured to determine a subset of reference frames as a reference frame set if there are M reference frames permitted to perform forward predictive coding on the current block, before determining the coding cost used to perform backward predictive coding on the current block using each reference frame in the reference frame set, where M is a positive integer greater than or equal to 2.

[0217] As one possible solution, the reference frame set determination unit is further configured to determine a subset of the M reference frames as a reference frame set if the current video frame is a video frame in real-time transmission and there are M reference frames permitted to be used for forward predictive coding of the current block.

[0218] As one possible solution, the device further, A first reference frame determination unit configured to determine M encoded video frames as M reference frames before determining some of the M reference frames as a set of reference frames, wherein the frame numbers of the M video frames are smaller than the frame numbers of the current video frames, or A second reference frame determination unit is configured to determine P encoded video frames and Q encoded video frames as M reference frames, where M is equal to the sum of P and Q, the frame numbers of the P video frames are less than the frame number of the current video frame, the frame numbers of the Q video frames are greater than the frame number of the current video frame, and P and Q are positive integers greater than or equal to 1.

[0219] As one possible solution, the first coding scheme determination unit is further configured to determine the current block's coding scheme as a forward predictive coding scheme if the forward predictive coding cost is less than the backward predictive coding cost, and to determine the current block's coding scheme as a backward predictive coding scheme if the forward predictive coding cost is greater than the backward predictive coding cost, or The first coding scheme determination unit is further configured to determine the current block's coding scheme as forward predictive coding if the forward predictive coding cost is less than the backward predictive coding cost and the bidirectional predictive coding cost of the current block, where the bidirectional predictive coding cost is the minimum coding cost among those used to perform bidirectional predictive coding on the current block using each reference frame in the reference frame set; to determine the current block's coding scheme as backward predictive coding if the backward predictive coding cost is less than the forward predictive coding cost and the bidirectional predictive coding cost; and to determine the current block's coding scheme as bidirectional predictive coding if the bidirectional predictive coding cost is less than the forward predictive coding cost and the backward predictive coding cost.

[0220] As one possible solution, the device further, An encoding module configured to perform forward predictive coding on the current block after determining the encoding scheme for the current block based on the relationship between the forward predictive coding cost and the backward predictive coding cost, or If the coding scheme for the current block is determined to be backward predictive coding, then a coding module configured to perform backward predictive coding on the current block, or If the encoding scheme for the current block is determined to be bidirectional predictive coding, the system includes an encoding module configured to perform bidirectional predictive coding on the current block.

[0221] One possible solution is for each of the N adjacent blocks to include at least two of the following: an adjacent block located to the left of the current block in the current video frame, an adjacent block located above the current block in the current video frame, an adjacent block located to the upper left of the current block in the current video frame, an adjacent block located to the right of the current block in the current video frame, and an adjacent block located to the upper right of the current block in the current video frame.

[0222] According to another aspect of the embodiments of this application, another apparatus for performing the above-described video coding is further provided, the apparatus comprising a backward predictive coding adjacent block statistics unit, a forward predictive coding cost determination unit, and a second coding scheme determination unit.

[0223] The backward predictive coding adjacent block statistics unit is configured to determine a second number of adjacent blocks in the N adjacent blocks that have undergone backward predictive coding, where N is a positive integer greater than or equal to 2, given that N adjacent blocks in the current video frame have been coded, and the coding scheme of each of the N adjacent blocks is different from that of forward predictive coding.

[0224] The forward predictive coding cost determination unit is configured to determine the coding cost used to perform forward predictive coding on the current block using each reference frame in the reference frame set when the second quantity is greater than 0, the backward predictive coding cost of the current block is determined using the reference frame set, and the backward predictive coding cost of the current block is less than a predetermined target cost threshold corresponding to the second quantity, and to determine the minimum coding cost among the determined coding costs used to perform forward predictive coding as the forward predictive coding cost of the current block, where the backward predictive coding cost is the minimum coding cost among the coding costs used to perform backward predictive coding on the current block using each reference frame in the reference frame set.

[0225] The second coding scheme determination unit is configured to determine the coding scheme for the current block based on the relationship between the backward predictive coding cost and the forward predictive coding cost.

[0226] According to yet another aspect of the embodiments of the present invention, an electronic device for carrying out the video encoding method described above is further provided. The electronic device may be a terminal device or a server as shown in Figure 1. This embodiment describes the case where the electronic device is a server as an example. As shown in Figure 20, the electronic device comprises a memory 2002 and a processor 2004, the memory 2002 storing a computer program, and the processor 2004 is configured to perform the steps in any of the above-described embodiments of the method by the computer program.

[0227] Optionally, in this embodiment, the above-mentioned electronic device may be placed in at least one of the multiple network devices of the computer network.

[0228] Optionally, in this embodiment, the processor may be configured to perform the following steps by a computer program.

[0229] In S1, if the N adjacent blocks of the current block in the current video frame are already encoded, and the encoding scheme of each of the N adjacent blocks is different from backward predictive encoding, then the first number of adjacent blocks for which forward predictive encoding has been performed is statistically determined, where N is a positive integer greater than or equal to 2.

[0230] In S2, if the first quantity is greater than 0, the forward predictive coding cost of the current block is determined using the reference frame set, and the forward predictive coding cost of the current block is less than a predetermined target cost threshold corresponding to the first quantity, the coding cost used to perform backward predictive coding on the current block is determined using each reference frame in the reference frame set, and the minimum coding cost among the determined coding costs used to perform backward predictive coding is determined as the backward predictive coding cost of the current block, where the forward predictive coding cost is the minimum coding cost among the coding costs used to perform forward predictive coding on the current block using each reference frame in the reference frame set.

[0231] In S3, the encoding scheme for the current block is determined based on the relationship between the forward predictive coding cost and the backward predictive coding cost.

[0232] Optionally, a person skilled in the art will understand that the structure shown in Figure 20 is merely illustrative, and that the electronic device may be a smartphone (e.g., an Android phone, an iOS® phone, etc.), a tablet computer, a PDA, and a terminal device such as a Mobile Internet Device (MID), or a PAD. Figure 20 is not limiting to the configuration of the electronic device. For example, the electronic device may include more or fewer components (e.g., network interfaces) than those shown in Figure 20, or may have a different configuration than that shown in Figure 20.

[0233] Here, memory 2002 may be used to store software programs and modules, for example, program instructions / modules corresponding to the video encoding method and apparatus in embodiments of the present invention. The processor 2004 executes various functional applications and data processing by executing the software programs and modules stored in memory 2002, i.e., realizing the video encoding method described above. Memory 2002 may include high-speed random memory, or one or more non-volatile memories such as magnetic memory, flash memory, or other non-volatile solid-state memory. In some examples, memory 2002 may further include memory located remotely from the processor 2004, and the remote memory may be connected to a terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, intranet, local area network, mobile communication network, and combinations thereof. Here, memory 2002 may, but are not limited to, be used to store information such as sample features of articles and target virtual resource accounts. As an example, as shown in Figure 20, the memory 2002 may include, but is not limited to, the forward predictive coding adjacent block statistics unit 1902, the backward predictive coding cost determination unit 1904, and the first coding scheme determination unit 1906 in the video coding device. Furthermore, it may include other module units in the video coding device, but further explanation is omitted in this example.

[0234] Optionally, the transmission device 2006 described above is configured to receive or transmit data using a single network. Specific examples of the network include wired and wireless networks. In one example, the transmission device 2006 includes a Network Interface Controller (NIC). The NIC may be connected to other network devices and routers via network cables to communicate with the Internet or a local area network. In one example, the transmission device 2006 may be a radio frequency (RF) module used to communicate with the Internet wirelessly.

[0235] The electronic device further includes a display device 2008 for displaying the order information to be processed, and a connection bus 2010 for connecting each module component within the electronic device.

[0236] In another embodiment, the above-mentioned terminal device or server may be a node in a distributed system, where the distributed system may be a blockchain system, and the blockchain system may be a distributed system formed by connecting multiple nodes via network communication. Here, a peer-to-peer (P2P) network may be formed between the nodes, and any type of computing device, such as electronic devices like servers and terminals, can become a node in the blockchain system by participating in the peer-to-peer network.

[0237] According to one aspect of this application, a computer program product is provided, the computer program product comprising a computer program / instruction, the computer program / instruction comprising program code for executing a method shown in a flowchart. In such an embodiment, the computer program may be downloaded and installed from a network via a communication unit 2109 and / or installed from removable media 2111. When the computer program is executed by the central processing unit 2101, the various functions provided in the embodiments of this application are executed. The numbering of embodiments of the present invention described above is for illustrative purposes only and does not indicate any order of superiority or inferiority of the embodiments.

[0238] Figure 21 illustrates a block diagram of a computer system configuration for an electronic device to realize an embodiment of this application. Note that the computer system 2100 of the electronic device shown in Figure 21 is merely an example and should not impose any limitations on the functions and scope of use of the embodiment of this application. As shown in Figure 21, the computer system 2100 includes a central processing unit (CPU) 2101, which can perform various appropriate operations and processes based on programs stored in read-only memory (ROM) 2102 or programs loaded from storage unit 2108 into random access memory (RAM) 2103. Various programs and data necessary for system operation are also stored in the random access memory 2103. The central processing unit 2101, read-only memory 2102, and random access memory 2103 are connected to each other via a bus 2104. An input / output interface 2105 (I / O interface) is also connected to the bus.

[0239] The input / output interface 2105 is connected to an input unit 2106 including a keyboard and mouse, an output unit 2107 including a CRT (Cathode Ray Tube), LCD (Liquid Crystal Display), and speakers, a storage unit 2108 including a hard disk, and a communication unit 2109 including a network interface card such as a LAN card and modem. The communication unit 2109 performs communication processing via a network such as the Internet. A drive 2110 is also connected to the input / output interface 2105 as needed. Removable media 2111, such as disks, optical disks, magneto-optical disks, and semiconductor memory, are loaded into the drive 2110 as needed, and computer programs read from the removable media 2111 are installed into the storage unit 2108 as needed.

[0240] In particular, according to embodiments of this application, the processes described in the flowcharts of each method can be implemented as computer software programs. For example, embodiments of this application provide a computer program product which includes a computer program stored on a computer-readable medium which includes program code for performing the methods shown in the flowcharts. In such embodiments, the computer program may be downloaded and installed from a network via a communication unit 2109 and / or installed from removable media 2111. When the computer program is executed by the central processing unit 2101, various functions defined in the system of this application are performed.

[0241] According to one aspect of this application, a computer-readable storage medium is provided, and the processor of an electronic device reads computer instructions from the computer-readable storage medium and executes the computer instructions, thereby causing the electronic device to perform the methods provided in the various selectable embodiments described above.

[0242] Optionally, in this embodiment, the computer-readable storage medium may be configured to store a computer program for performing the following steps.

[0243] In S1, if N adjacent blocks of the current block in the current video frame have been encoded, and the encoding scheme of each of the N adjacent blocks is different from backward predictive encoding, then a first number of adjacent blocks in the N adjacent blocks for which forward predictive encoding has been performed is statistically determined, where N is a positive integer of 2 or greater.

[0244] In S2, if the first quantity is greater than 0, the forward predictive coding cost of the current block is determined using the reference frame set, and the forward predictive coding cost of the current block is less than a predetermined target cost threshold corresponding to the first quantity, the coding cost used to perform the backward predictive coding on the current block using each reference frame in the reference frame set is determined, and the minimum coding cost among the determined coding costs used to perform the backward predictive coding is determined as the backward predictive coding cost of the current block, where the forward predictive coding cost is the minimum coding cost among the coding costs used to perform the forward predictive coding on the current block using each reference frame in the reference frame set.

[0245] In S3, the encoding scheme for the current block is determined based on the relationship between the forward predictive coding cost and the backward predictive coding cost.

[0246] Optionally, in this embodiment, those skilled in the art will understand that all or part of the steps in each of the above embodiments can be completed by a program that instructs hardware related to a terminal device, the program may be stored in a computer-readable storage medium, the storage medium may include a flash disk, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0247] The integrated unit in the above embodiments may be stored on a computer-readable storage medium if it is implemented in the form of a software function module and sold or used as an independent product. Based on this understanding, an essential part of the technical solution of the embodiments of the present invention, or a part that contributes to the prior art, or all or part of the technical solution, may be embodied in the form of a computer software product, which is stored on a storage medium and includes several instructions for causing one or more computer devices (which may be personal computers, servers, or network devices, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.

[0248] In the above embodiments of the present invention, the description of each embodiment is emphasized in itself, and for parts not described in detail in one embodiment, the relevant description in another embodiment may be referenced.

[0249] In some embodiments provided in this application, the disclosed clients may be implemented in other ways. Herein, the embodiments of the apparatus described above are illustrative only, and for example, the division of the units is only a division of logical functions, and in actual implementation, other methods of division may be possible, for example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not implemented. Furthermore, the mutual coupling, direct coupling or communication connection indicated or discussed may be implemented using several communication interfaces, and indirect coupling or communication connection between devices or units may be electrical, mechanical or other in form.

[0250] The units described as separate components may or may not be physically separated, and the components referred to as units may or may not be physical units, and may be located in one place or distributed across multiple network units. Depending on the actual needs, some or all of these units can be selected to achieve the objectives of the technical solution in this embodiment.

[0251] Furthermore, each functional unit in each embodiment of the present invention may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The integrated unit may be implemented in hardware form or in the form of a software functional unit.

[0252] Each of the technical features of the above embodiments can be combined in any way, and for the sake of brevity, not all possible combinations of each of the technical features in the above embodiments are described. However, all such combinations of technical features should be considered to be included within the scope of this specification, as long as they do not contradict each other.

[0253] The embodiments described above represent only a few embodiments of this application, and although they are described in more specific and detailed terms, they are not intended to limit the scope of protection of this application. It should be noted that a person skilled in the art could make several modifications and improvements without departing from the spirit of this application, and these modifications and improvements are included within the scope of protection of this application. Therefore, the scope of protection of this application shall be subject to the scope of protection of the claims.

Claims

1. A video encoding method performed by an electronic device, Steps include:

1. In a video frame, where N adjacent blocks of the current block have been encoded, and the encoding scheme of each of the N adjacent blocks is different from backward predictive encoding, a first number of adjacent blocks in the N adjacent blocks for which forward predictive encoding has been performed, where N is a positive integer of 2 or more; Steps of determining the coding cost used to perform the backward predictive coding on the current block using each reference frame in the reference frame set, where the first quantity is greater than 0, the forward predictive coding cost of the current block is determined using a reference frame set, and the forward predictive coding cost of the current block is determined as the backward predictive coding cost of the current block, wherein the forward predictive coding cost is the minimum coding cost used to perform the forward predictive coding on the current block using each reference frame in the reference frame set, A video coding method comprising the step of determining the coding scheme for the current block based on the relationship between the forward predictive coding cost and the backward predictive coding cost.

2. The aforementioned video encoding method is The process further includes determining a target cost threshold in a predetermined cost threshold set, where each cost threshold in the cost threshold set corresponds to a quantity in a quantity set and a size in a size set, where the quantity set comprises N positive integers from 1 to N, the first quantity is a quantity in the quantity set, and the size set comprises different block sizes that are permitted to be encoded in the current video frame, where the current block size is a size in the size set. The video encoding method according to claim 1.

3. In the predetermined set of cost thresholds, before determining the target cost threshold corresponding to the first quantity and the size of the current block, the video encoding method: The steps of encoding a set of video frames in a sample video and obtaining encoding information generated by encoding the set of video frames, wherein the encoding information includes a set of cumulative encoding costs and a set of counts, each cumulative encoding cost in the set of cumulative encoding costs corresponds to one quantity in the set of quantities and one size in the set of sizes, each count in the set of counts corresponds to one cumulative encoding cost in the set of cumulative encoding costs, the (i, j)th cumulative encoding cost in the set of cumulative encoding costs corresponds to the i-th quantity in the set of quantities and the j-th size in the set of sizes, and also corresponds to the (i, j)th count in the set of counts, and the (i, j)th count is in the encoding process of the set of video frames Step(i,j) represents the number of times the target coding event has occurred, where i and j are both positive integers greater than or equal to 1, and the target coding event refers to performing the forward predictive coding on a block when N adjacent blocks of a block in the set of video frames have been coded, the coding scheme of each of the N adjacent blocks of the block is different from the backward predictive coding, the size of the block is the j-th size, and the number of adjacent blocks of the N adjacent blocks of the block on which the forward predictive coding has been performed is the i-th quantity, and the (i,j)-th cumulative coding cost is the sum of the forward predictive coding costs of the block in the target coding event that has occurred the (i,j)-th time, and The further step includes determining the cost threshold set based on the encoded information, The video encoding method according to claim 2.

4. The step of determining the cost threshold set based on the encoded information is: The step includes determining the (i, j)th cost threshold in the cost threshold set to be equal to the value obtained by dividing the (i, j)th cumulative coding cost by the (i, j)th number of times, wherein the (i, j)th cost threshold corresponds to the i-th quantity in the quantity set and the j-th size in the size set. The video encoding method according to claim 3.

5. In the encoding process of a set of video frames in a sample video, the video encoding method is: Steps to obtain the forward predictive coding cost and backward predictive coding cost of the k-th block, where k is a positive integer of 1 or more, given that N adjacent blocks of the k-th block in the set of video frames are encoded, and the encoding scheme of each of the N adjacent blocks of the k-th block is different from the backward predictive coding, and the size of the k-th block is the j-th size, and the number of adjacent blocks of the k-th block that have undergone the forward predictive coding is the i-th quantity, wherein k is a positive integer of 1 or more. If, based on the forward predictive coding cost and backward predictive coding cost of the k-th block, it is determined that the coding scheme of the k-th block is the forward predictive coding, the forward predictive coding cost of the k-th block is added to the (i, j)-th cumulative coding cost, and the (i, j)-th count is incremented by 1, further comprising the steps of: The video encoding method according to claim 3.

6. The aforementioned video encoding method is The process further includes the step of encoding each adjacent block of the N adjacent blocks, wherein the encoding scheme for each adjacent block of the N adjacent blocks is one of the forward predictive encoding and the backward predictive encoding, or one of the forward predictive encoding, the backward predictive encoding, and the bidirectional predictive encoding, or one of the forward predictive encoding, the backward predictive encoding, the bidirectional predictive encoding, and the intraframe predictive encoding. The video encoding method according to claim 1.

7. The aforementioned video encoding method is If there are M reference frames permitted to be used to perform the forward predictive coding on the current block, the further step includes determining some of the M reference frames as the set of reference frames, where M is a positive integer of 2 or more. The video encoding method according to claim 1.

8. If the number of reference frames permitted to be used for performing the forward predictive coding on the current block is M, the step of determining a subset of the M reference frames as the set of reference frames is: If the current video frame is a video frame for real-time transmission and there are M reference frames permitted to be used for performing the forward predictive coding on the current block, the step includes determining a subset of the M reference frames as the set of reference frames. The video encoding method according to claim 7.

9. The aforementioned video encoding method is The process further includes determining M encoded video frames as the M reference frames, wherein the frame numbers of the M video frames are smaller than the frame numbers of the current video frame. The video encoding method according to claim 7.

10. The aforementioned video encoding method is The process further includes determining P encoded video frames and Q encoded video frames as the M reference frames, where M is equal to the sum of P and Q, the frame numbers of the P video frames are less than the frame number of the current video frame, the frame numbers of the Q video frames are greater than the frame number of the current video frame, and P and Q are positive integers greater than or equal to 1. The video encoding method according to claim 7.

11. The step of determining the coding scheme for the current block based on the relationship between the forward predictive coding cost and the backward predictive coding cost is: If the forward predictive coding cost is smaller than the backward predictive coding cost, the coding scheme for the current block is determined to be the forward predictive coding scheme. The video encoding method according to claim 1.

12. The step of determining the coding scheme for the current block based on the relationship between the forward predictive coding cost and the backward predictive coding cost is: If the forward predictive coding cost is greater than the backward predictive coding cost, the coding scheme for the current block is determined to be the backward predictive coding scheme. The video encoding method according to claim 1.

13. The step of determining the coding scheme for the current block based on the relationship between the forward predictive coding cost and the backward predictive coding cost is: If the forward predictive coding cost is less than the backward predictive coding cost and the bidirectional predictive coding cost of the current block, the coding scheme of the current block is determined to be the forward predictive coding, wherein the bidirectional predictive coding cost is the minimum coding cost among the coding costs used to perform bidirectional predictive coding on the current block using each reference frame in the reference frame set. The video encoding method according to claim 1.

14. The step of determining the coding scheme for the current block based on the relationship between the forward predictive coding cost and the backward predictive coding cost is: If the backward predictive coding cost is less than the forward predictive coding cost and the bidirectional predictive coding cost of the current block, the coding scheme of the current block is determined to be the backward predictive coding, wherein the bidirectional predictive coding cost is the minimum coding cost among the coding costs used to perform bidirectional predictive coding on the current block using each reference frame in the reference frame set. The video encoding method according to claim 1.

15. The step of determining the coding scheme for the current block based on the relationship between the forward predictive coding cost and the backward predictive coding cost is: The coding scheme for the current block is determined to be the bidirectional predictive coding if the bidirectional predictive coding cost of the current block is smaller than the forward predictive coding cost and the backward predictive coding cost, wherein the bidirectional predictive coding cost is the minimum coding cost among the coding costs used to perform bidirectional predictive coding on the current block using each reference frame in the reference frame set. The video encoding method according to claim 1.

16. The aforementioned video encoding method is If the encoding scheme for the current block is determined to be the forward predictive encoding, the further step includes performing the forward predictive encoding on the current block. The video encoding method according to claim 1.

17. The video encoding method further includes the step of performing the backward predictive encoding on the current block if the encoding scheme for the current block is determined to be the backward predictive encoding. The video encoding method according to claim 1.

18. The aforementioned video encoding method is If the coding scheme for the current block is determined to be bidirectional predictive coding, the further step includes performing the bidirectional predictive coding on the current block. The video encoding method according to claim 1.

19. The N adjacent blocks include at least two of the following: an adjacent block located to the left of the current block in the current video frame; an adjacent block located above the current block in the current video frame; an adjacent block located to the upper left of the current block in the current video frame; an adjacent block located to the right of the current block in the current video frame; and an adjacent block located to the upper right of the current block in the current video frame. The video encoding method according to claim 1.

20. A video encoding method performed by an electronic device, A step of determining a second number of adjacent blocks in the N adjacent blocks for which backward predictive coding has been performed, where N is a positive integer of 2 or more, given that N adjacent blocks in the current video frame have already been coded, and the coding scheme of each of the N adjacent blocks is different from forward predictive coding. Steps of determining the coding cost used to perform the forward predictive coding on the current block using each reference frame in the reference frame set, where the second quantity is greater than 0, the backward predictive coding cost of the current block is determined using a reference frame set, and the backward predictive coding cost of the current block is less than a predetermined target cost threshold corresponding to the second quantity, wherein the backward predictive coding cost is the minimum coding cost used to perform the backward predictive coding on the current block using each reference frame in the reference frame set, and the minimum coding cost among the determined coding costs used to perform the forward predictive coding is determined as the forward predictive coding cost of the current block, wherein the backward predictive coding cost is the minimum coding cost among the coding costs used to perform the backward predictive coding on the current block using each reference frame in the reference frame set, A video coding method comprising the step of determining the coding scheme for the current block based on the relationship between the backward predictive coding cost and the forward predictive coding cost.

21. A video encoding device, A forward predictive coding adjacent block statistics unit configured to statistically calculate a first number of adjacent blocks in the N adjacent blocks for which forward predictive coding has been performed, where N is a positive integer of 2 or more, in the case where N adjacent blocks of the current block in the current video frame have been coded, and the coding scheme of each of the N adjacent blocks is different from backward predictive coding, the forward predictive coding adjacent block statistics unit, A backward predictive coding cost determination unit configured to determine the coding cost used to perform backward predictive coding on the current block using each reference frame in the reference frame set when the first quantity is greater than 0, the forward predictive coding cost of the current block is determined using a reference frame set, and the forward predictive coding cost of the current block is less than a predetermined target cost threshold corresponding to the first quantity, and to determine the minimum coding cost among the determined coding costs used to perform backward predictive coding as the backward predictive coding cost of the current block, wherein the forward predictive coding cost is the minimum coding cost among the coding costs used to perform forward predictive coding on the current block using each reference frame in the reference frame set, A video coding apparatus comprising: a first coding scheme determination unit configured to determine the coding scheme of the current block based on the relationship between the forward predictive coding cost and the backward predictive coding cost.

22. A video encoding device, A backward predictive coding adjacent block statistics unit configured to determine a second number of adjacent blocks for which backward predictive coding has been performed, in the case where N adjacent blocks of the current block in the current video frame have been coded, and the coding scheme of each of the N adjacent blocks is different from forward predictive coding, wherein N is a positive integer of 2 or more, and A forward predictive coding cost determination unit configured to determine the coding cost used to perform the forward predictive coding on the current block using each reference frame in the reference frame set when the second quantity is greater than 0, the backward predictive coding cost of the current block is determined using a reference frame set, and the backward predictive coding cost of the current block is less than a predetermined target cost threshold corresponding to the second quantity, and to determine the minimum coding cost among the determined coding costs used to perform the forward predictive coding as the forward predictive coding cost of the current block, wherein the backward predictive coding cost is the minimum coding cost among the coding costs used to perform the backward predictive coding on the current block using each reference frame in the reference frame set, A video coding apparatus comprising: a second coding scheme determination unit configured to determine the coding scheme of the current block based on the relationship between the backward predictive coding cost and the forward predictive coding cost.

23. A computer program for causing a processor to perform a step of the method according to any one of claims 1 to 20.

24. Electronic device comprising memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to perform the method according to any one of claims 1 to 20 by executing the computer program.

Images