Video encoding method, device, computer equipment and storage medium

Abstract

The embodiments of the present application disclose a video encoding method, apparatus, computer equipment, and storage medium. The method includes: performing a first encoding process on each video frame in a processed video to obtain multiple encoding blocks included in each video frame and a reference frame for each encoding block; when a target video frame in the processed video is encoded for the Nth time, determining a reference encoding block that overlaps with the target encoding block from the multiple encoding blocks included in the target video frame during the target encoding process; the target encoding process is an encoding process between the first encoding process and the N-1th encoding process; based on the reference frames of each reference encoding block in the target encoding process, determining a set of candidate reference frames for the target encoding block; determining a target reference frame for the target encoding block from the set of candidate reference frames, and performing encoding prediction on the target encoding block based on the target reference frame. The present application can improve encoding efficiency.

Description

Technical Field

[0001] The present application relates to the field of coding, and in particular to a video coding method, apparatus, computer equipment, and storage medium. Background Art

[0002] With the continuous development of computer technology, digital multimedia technology, and coding and decoding technology, videos and images are becoming increasingly popular in various fields and people's daily lives. Typically, a video needs to be encoded multiple times in business scenarios such as video streams requiring different clarity and bit rate control algorithms. In any of the multiple encodings of a video, for each coding block to be encoded in the video, the encoder often needs to perform a motion search on all possible reference frames to determine which frame and which area to use for prediction; that is, it needs to select a final reference frame from all possible reference frames to predict the coding block to be encoded.

[0003] Currently, motion search is usually performed on each frame in the preset reference frame set corresponding to the coding block to determine the final reference frame. It can be seen that this method of determining the reference frame requires traversing each frame in the set, which requires a large amount of calculation and may lead to low coding efficiency. Summary of the Invention

[0004] The embodiments of the present application provide a video encoding method, apparatus, computer device, and storage medium, which can improve encoding efficiency.

[0005] In a first aspect, an embodiment of the present application provides a video encoding method, including:

[0006] Obtaining a video to be processed, and performing a first encoding process on each video frame in the video to be processed to obtain a plurality of encoding blocks included in each video frame and a reference frame for each encoding block; the plurality of encoding blocks included in a video frame are obtained by dividing the video frame into blocks, and the reference frame for a encoding block is a video frame used in encoding prediction for the encoding block in the video to be processed;

[0007] When performing an Nth encoding process on a target video frame of the video to be processed, determining one or more reference encoding blocks that overlap with the target encoding block from a plurality of encoding blocks included in the target video frame during the target encoding process; the target video frame is a video frame containing the target encoding block, the target encoding process is one or more encoding processes between a first encoding process and an N-1th encoding process, where N is a positive integer greater than 1;

[0008] Determining a candidate reference frame set for the target coding block based on a coding mode of each reference coding block and a reference frame of each reference coding block in the target secondary coding process; the coding mode includes an inter-frame coding mode or an intra-frame coding mode;

[0009] A target reference frame of the target coding block is determined from the candidate reference frame set, and coding prediction is performed on the target coding block based on the target reference frame.

[0010] In a second aspect, an embodiment of the present application provides a video encoding device, including:

[0011] a first encoding unit, configured to obtain a video to be processed and perform a first encoding process on each video frame in the video to be processed to obtain a plurality of encoding blocks included in each video frame and a reference frame for each encoding block; the plurality of encoding blocks included in a video frame are obtained by dividing the video frame into blocks, and the reference frame for a encoding block is a video frame used in encoding prediction for the encoding block in the video to be processed;

[0012] a first determining unit configured to, when performing an Nth encoding process on a target video frame of the to-be-processed video, determine, from a plurality of encoding blocks included in the target video frame during the target encoding process, one or more reference encoding blocks that overlap with the target encoding block; the target video frame being the video frame containing the target encoding block, the target encoding process being one or more encoding processes between the first encoding process and the N-1th encoding process, where N is a positive integer greater than 1;

[0013] a second determining unit, configured to determine a candidate reference frame set for the target coding block based on a coding mode of each reference coding block and a reference frame of each reference coding block in the target secondary encoding process; the coding mode includes an inter-frame coding mode or an intra-frame coding mode;

[0014] The second encoding unit is configured to determine a target reference frame of the target coding block from the candidate reference frame set, and perform encoding prediction on the target coding block based on the target reference frame.

[0015] In a third aspect, an embodiment of the present application provides a computer device, comprising: a processor and a memory, wherein the processor is configured to execute the method described in the first aspect above.

[0016] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, in which program instructions are stored. When the program instructions are executed, the method described in the first aspect above is implemented.

[0017] In a fifth aspect, an embodiment of the present application further provides a computer program product or a computer program, which includes program instructions, and when the program instructions are executed by a processor, the method described in the first aspect above is implemented.

[0018] In an embodiment of the present application, a first encoding process can be performed on each video frame in the video to be processed to obtain multiple coding blocks included in each video frame and a reference frame for each coding block; wherein the multiple coding blocks included in a video frame are obtained by dividing the video frame into blocks, and the reference frame of a coding block is the video frame used when performing coding prediction on the coding block in the video to be processed. When the target video frame of the video to be processed is encoded for the Nth time, one or more reference coding blocks that overlap with the target coding block can be determined from the multiple coding blocks included in the target video frame during the target encoding process; the target video frame can be the video frame where the target coding block is located, and the target encoding process can be one or more encoding processes between the first encoding process and the N-1th encoding process, where N is a positive integer greater than 1. Then, based on the encoding mode of each reference coding block and the reference frame of each reference coding block in the target encoding process, a set of candidate reference frames for the target coding block can be determined; the encoding mode can include an inter-frame coding mode or an intra-frame coding mode. Furthermore, a target reference frame for a target coding block can be determined from a set of candidate reference frames, and coding prediction can be performed on the target coding block based on the target reference frame. By implementing the above method, in a scenario where multiple encodings are performed, when encoding the target coding block in the Nth encoding process, the reference frame selection results of each coding block in the historical encoding process can be reused to reduce the computational complexity of selecting a reference frame, save computing resources, and reduce the selection time for selecting a reference frame, thereby achieving rapid reference frame selection and improving coding efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 This is a schematic diagram of the architecture of a video encoding system provided by an embodiment of the present application;

[0021] Figure 2 This is a flow chart of a video encoding method provided by an embodiment of the present application;

[0022] Figure 3a This is a flowchart of determining the presence of multiple encoding requirements provided by an embodiment of the present application;

[0023] Figure 3b 1 is a schematic diagram showing a block division result of a video frame provided by an embodiment of the present application;

[0024] Figure 3c This is a schematic diagram showing a method of determining a reference coding block according to an embodiment of the present application;

[0025] Figure 4 This is a flowchart of another video encoding method provided by an embodiment of the present application;

[0026] Figure 5a This is a schematic diagram showing a one-way prediction provided by an embodiment of the present application;

[0027] Figure 5b This is a schematic diagram of a bidirectional prediction display provided by an embodiment of the present application;

[0028] Figure 5c This is a flowchart of another video encoding method provided by an embodiment of the present application;

[0029] Figure 6 is a structural diagram of a video encoding device provided in an embodiment of the present application;

[0030] Figure 7 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application.

[0032] Cloud technology refers to a hosting technology that unifies hardware, software, network and other resources within a wide area network or local area network to achieve data computing, storage, processing and sharing.

[0033] Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, and application technology based on the cloud computing business model. It can form a resource pool for on-demand, flexible and convenient use. Cloud computing technology will become a key support. Backend services of technical network systems, such as video websites, image websites, and more portals, require extensive computing and storage resources. With the rapid development and application of the internet industry, every item will likely have its own unique identification mark, which will need to be transmitted to backend systems for logical processing. Different levels of data will be processed separately. All types of industry data require strong system support, which can only be achieved through cloud computing.

[0034] Cloud computing is a computing model that distributes computing tasks across a resource pool consisting of a large number of computers, enabling various application systems to access computing power, storage space, and information services as needed. The network that provides these resources is called the "cloud." To users, these resources appear infinitely scalable and can be accessed at any time, used on demand, expanded at any time, and paid for on a per-use basis.

[0035] This application stores the data required for video encoding in the "cloud", and the data in the cloud can be obtained and expanded at any time according to needs. For example, when the encoder encodes a video multiple times, the multiple encoding blocks included in each video frame obtained by the first encoding process and the reference frame of each encoding block can be stored in the "cloud". If the encoder encodes the video for the second time and subsequently, the relevant data can be obtained from the "cloud".

[0036] In one implementation, an embodiment of the present application proposes a video encoding scheme. Specifically, the scheme operates as follows: First, a video to be processed may be obtained, and each video frame in the video to be processed may be subjected to a first encoding process to obtain multiple encoding blocks included in each video frame and a reference frame for each encoding block. When the video to be processed is subjected to an Nth encoding process, data obtained from the target encoding process may be used to assist in the Nth encoding process of the video to be processed. The target encoding process may be one or more encoding processes between the first encoding process and the N-1th encoding process.

[0037] Optionally, when the target video frame of the video to be processed is subjected to the Nth encoding process, one or more reference encoding blocks that overlap with the target encoding block can be determined from the multiple encoding blocks included in the target video frame during the target encoding process; the target video frame is the video frame where the target encoding block is located. After obtaining one or more reference encoding blocks, the Nth encoding process of the video to be processed can be performed based on the reference frames of each reference encoding block in the target encoding process. In one embodiment, first, a set of candidate reference frames for the target encoding block can be determined based on the reference frames of each reference encoding block in the target encoding process; for example, a set of candidate reference frames for the target encoding block can be determined based on the encoding mode of each reference encoding block and the reference frame of each reference encoding block in the target encoding process; the encoding mode can include an inter-frame encoding mode or an intra-frame encoding mode. Further, a target reference frame of the target encoding block can be determined from the candidate reference frame set, and encoding prediction can be performed on the target encoding block based on the target reference frame.

[0038] By implementing the above scheme, in a scenario of multiple encodings, and when encoding the target coding block in the Nth encoding process, the reference frame selection results of each coding block in the historical encoding process (such as the first encoding process) can be reused to achieve a reduction in the initial reference frame set corresponding to the target coding block, thereby reducing the computational complexity of selecting the reference frame, saving computing resources, and reducing the selection time for selecting the reference frame, thereby improving the encoding efficiency.

[0039] In a specific implementation, the video encoding scheme mentioned above can be executed by a computer device, such as an encoder in a computer device. The computer device can be a terminal or a server; the terminals mentioned here can include but are not limited to: smartphones, tablets, laptops, desktop computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, aircraft, etc.; various clients (applications, APPs) can run in the terminal, such as game clients, multimedia playback clients, social clients, browser clients, information flow clients, educational clients, etc. The server mentioned here can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and basic cloud computing services such as big data and artificial intelligence platforms, etc.

[0040] In a possible implementation, when the computer device is a server, the embodiment of the present application provides a video encoding system, such as Figure 1 As shown, the video encoding system includes at least one terminal and at least one server; the terminal can obtain the video to be processed and upload the obtained video to be processed to the server (i.e., a computer device) so that the computer device can obtain the video to be processed and perform video encoding on the video to be processed.

[0041] In one implementation, the video encoding scheme proposed in the embodiments of the present application can be applied to scenarios such as video on demand and bitrate control. For example, in a video on demand scenario, it is usually necessary to transcode the original video provided by the content manufacturer multiple times (i.e., multiple encodings) according to business needs to output video streams of different resolutions; for another example, in a bitrate control scenario, the bitrate control algorithm usually uses multiple encodings to obtain higher bitrate accuracy and better coding efficiency.

[0042] Based on the above description of the video encoding scheme, the present embodiment proposes a video encoding method. The present embodiment is mainly described with a computer device as the execution subject; see Figure 2 , the video encoding method may include the following steps S201-S204:

[0043] S201 , obtaining a video to be processed, and performing a first encoding process on each video frame in the video to be processed to obtain a plurality of encoding blocks included in each video frame and a reference frame of each encoding block.

[0044] The video to be processed may be any type of video, such as sports video, education video, film and television video, etc.

[0045] In one implementation, when there are multiple encoding requirements for the video to be processed, the video to be processed is obtained.

[0046] Optionally, when the computer device receives a multiple encoding request, it may be determined that there is a multiple encoding requirement for the video to be processed. For example, a technician may send a multiple encoding request for the video to be processed to the computer device, so that the computer device receives the multiple encoding request. After the computer device receives the multiple encoding request, it is determined that there is a multiple encoding requirement for the video to be processed. In one possible implementation, when the technician needs to encode a certain video multiple times, the technician may perform relevant operations through the user operation interface output by the terminal to send the multiple encoding requirements for the video to be processed to the computer device.

[0047] See for example Figure 3a As shown: the terminal used by the technician may display a user operation interface on the terminal screen, and the user operation interface may include at least a video setting area marked by 301 and a confirmation control marked by 302. If the technician wants to encode a certain video multiple times, the technician can enter relevant information of the video to be processed (such as the video or the storage address corresponding to the video) in the video setting area 301, and then perform a trigger operation (such as a click operation, a press operation, etc.) on the confirmation control 302, thereby triggering the terminal used by the technician to send multiple encoding requirements for the video to be processed to the computer device based on the relevant information in the video setting area 301.

[0048] Optionally, the need for multiple encodings can also be triggered by a scheduled encoding task. For example, a scheduled encoding task can be set that indicates the triggering conditions for multiple encodings of a video. For example, the triggering conditions could be when the current time reaches a preset archiving time, when the remaining storage space for storing videos reaches a preset remaining storage space, or when a new video is added to the storage space for storing videos.

[0049] In one implementation, after obtaining a video to be processed, the video to be processed may be encoded multiple times. For example, the number of encodings may be N, where N is a positive integer greater than 1. For example, N may be 2, 4, or other values. For example, if N is 2, then the video to be processed needs to be encoded twice; if N is 4, then the video to be processed needs to be encoded four times. The following first describes the first encoding process of the video to be processed.

[0050] In a specific implementation, the computer device may perform a first encoding process on each video frame in the video to be processed to obtain the video to be processed after the first encoding process. During the first encoding process, multiple encoding blocks included in each video frame and a reference frame corresponding to each encoding block may be obtained.

[0051] Among them, the multiple coding blocks included in a video frame can be obtained by dividing the video frame into blocks. In different coding standards, the names of coding blocks may be different. For example, in the H.264 coding standard, coding blocks can be called sub-macroblocks. In the H.265 / H.266 coding standard, coding blocks can be called CUs (Coding Units). In the embodiment of the present application, sub-macroblocks and CUs can be collectively referred to as coding blocks. The number of coding blocks corresponding to each video frame can be the same or different, and the block division method of each video frame can be the same or different. For example, Figure 3b The boxes 31 and 32 in the figure respectively represent the block division results of a video frame, wherein any small box in the boxes 31 and 32 represents a coding block, for example, the box 33 can be understood as a coding block.

[0052] The reference frame of a coding block may be a video frame used when performing coding prediction on the coding block in the video to be processed.

[0053] S202 , when performing an Nth encoding process on a target video frame of a to-be-processed video, determining one or more reference encoding blocks overlapping with the target encoding block from a plurality of encoding blocks included in the target video frame during the target encoding process.

[0054] The target video frame is the video frame where the target coding block is located; the target video frame can be any video frame in the video to be processed, and the target coding block can be any coding block in the target video frame.

[0055] The target encoding process is one or more encoding processes between the first encoding process and the N-1th encoding process. For example, the target encoding process can be the first encoding process, or can be the first encoding process and the second encoding process, or can be the second encoding process and the third encoding process, and so on. Optionally, the process of each encoding process between the second encoding process and the N-1th encoding process can be the same as the process of the first encoding process; that is, for any encoding process, it is not necessary to use the reference frame selection result of the encoding block in the historical encoding process to encode the encoding block in any encoding process. Optionally, the process of each encoding process between the second encoding process and the N-1th encoding process can be the same as the process for the Nth encoding process (i.e., steps S202-S204) proposed in the embodiment of the present application; that is, for any encoding process, the reference frame selection result of the encoding block in the historical encoding process is used to encode the encoding block in any encoding process. Optionally, the process of the encoding process between the second encoding process and the N-1th encoding process can be the same as the process for the first encoding process and the process for the Nth encoding process.

[0056] It can be understood that when the Nth encoding process is performed on the video to be processed, each video frame in the video to be processed also needs to be encoded. The following description will be made using a certain video frame in the video to be processed as an example. For example, the target video frame can be used for explanation. For example, the target video frame can be a video frame that needs to be encoded for the Nth time at the current moment. In one implementation, when the Nth encoding process is performed on the target video frame of the video to be processed, the target video frame first needs to be divided into blocks or coding blocks to obtain basic coding blocks for encoding, and then each coding block in the multiple coding blocks obtained by the block division is subsequently encoded. Considering that the process of encoding each coding block is the same, the following description will be made using a coding block in the target video frame as an example. For example, the target coding block can be used for explanation. For example, the target coding block can be a coding block that needs to be encoded for the Nth time at the current moment.

[0057] In one implementation, determining a reference coding block corresponding to a target coding block can be accomplished by first obtaining the regional position of the target coding block in a target video frame during the Nth encoding process. Furthermore, obtaining the regional position of each coding block in the target video frame corresponding to the target video frame during the target encoding process can be performed. Furthermore, based on the regional position of each coding block in the target video frame, a coding block having an overlapping region with the regional position of the target coding block can be determined from the multiple coding blocks, and the determined coding block can be used as a reference coding block having an overlapping relationship with the target coding block.

[0058] In one embodiment, if the target sub-encoding process is an encoding process between the first encoding process and the N-1th encoding process, illustratively, the target sub-encoding process is the first encoding process, then the regional position of each encoding block in the target video frame corresponding to the target video frame during the first encoding process can be obtained. In this case, based on the regional position of each encoding block in the target video frame during the first encoding process, an encoding block having an overlapping region with the regional position of the target encoding block can be determined from the multiple encoding blocks during the first encoding process, and the determined encoding block is used as a reference encoding block having an overlapping relationship with the target encoding block.

[0059] For example, see Figure 3c As shown, Figure 3c Box 34 in the figure represents a video frame. The dashed box in box 34 represents the target coding block, and the solid box represents the coding block corresponding to the target secondary encoding process. It can be seen that coding blocks 1, 2, and 3 overlap with the target coding block. Therefore, coding blocks 1, 2, and 3 can be used as reference coding blocks corresponding to the target coding block.

[0060] In one embodiment, if the target sub-encoding process is a plurality of encoding processes between a first encoding process and an N-1th encoding process, such as a first encoding process and a second encoding process, the regional position of each encoding block in the target video frame of the plurality of encoding blocks corresponding to the target video frame during the first encoding process and the regional position of each encoding block in the target video frame of the plurality of encoding blocks corresponding to the target video frame during the second encoding process can be obtained. In this case, based on the regional position of each encoding block in the target video frame during the first encoding process and the second encoding process, an encoding block having an overlapping region with the regional position of the target encoding block can be determined from the plurality of encoding blocks in the first encoding process and the plurality of encoding blocks in the second encoding process, and the determined encoding block can be used as a reference encoding block having an overlapping relationship with the target encoding block.

[0061] S203 : Determine a candidate reference frame set for the target coding block based on the coding mode of each reference coding block and the reference frame of each reference coding block in the target sub-coding process.

[0062] The coding mode may include an inter-frame coding mode or an intra-frame coding mode. The inter-frame coding mode or the intra-frame coding mode may be a coding prediction technology. If intra-frame prediction is used during coding, it indicates that the coding of the target video frame does not require reference to information of other video frames; if inter-frame prediction is used during coding, it indicates that information of adjacent video frames is required to predict the target video frame.

[0063] In one implementation, if no reference coding blocks utilize intra-frame coding mode, a candidate reference frame set for the target coding block may be determined based on the reference frames of the reference coding blocks in the target secondary encoding process. For example, if the target secondary encoding process is the first encoding process, a candidate reference frame set for the target coding block may be determined based on the reference frames of the reference coding blocks in the first encoding process. The reference frames in the candidate reference frame set may be understood as reference frames required for subsequent prediction.

[0064] If any of the reference coding blocks uses intra-frame coding mode, it indicates that the temporal correlation of the image region corresponding to the target video frame is weak, and the certainty of the reference frame during prediction is weak. In this case, all video frames in the initial reference frame set corresponding to the reference coding block can be set as available, that is, the initial reference frame set corresponding to the reference coding block can be used as the candidate reference frame set for the target coding block. The reference frame of any of the one or more reference coding blocks exists in the initial reference frame set corresponding to any of the reference coding blocks.

[0065] It should be understood that in a coding standard that encodes in a block-dividing manner, when the encoder in the coding system performs prediction (such as inter-frame prediction) in units of coding blocks, the encoder usually constructs a reference frame set (reference frame list) for the target video frame, so that the encoder can select a video frame from the reference frame set as a reference frame for prediction. For the convenience of description, the reference frame set here can be referred to as the initial reference frame set. As mentioned above, the encoder can construct an initial reference frame set for the target video frame, that is, each video frame in the video to be processed can correspond to an initial reference frame set; then the initial reference frame sets corresponding to different coding blocks in the same video frame are the same, that is, different coding blocks in the same video frame are predicted using a video frame in the same reference frame set as a reference frame. In other words, the initial reference frame set for the target coding block in the above-mentioned target video frame and any reference coding block is the same.

[0066] The initial reference frame set for each video frame can be pre-set. The initial reference frame set corresponding to a video frame includes one or more video frames in the video to be processed. For example, the video frames in the initial reference frame set corresponding to a video frame can be continuous with the video frame in the video to be processed, or can be discontinuous with the video frame. The number and specific video frames in the initial reference frame set are not limited.

[0067] Optionally, the initial reference frame set of a video frame can be divided into a forward reference frame set (or referred to as a forward reference frame list) and a backward reference frame set (or referred to as a backward reference frame list) according to the encoding order and the playback order. Among them, the forward reference frame set can include several video frames whose encoding order and playback order are both before the target video frame. The backward reference frame set can include video frames whose encoding order is before the target video frame but whose playback order is after the target video frame. That is, in an embodiment of the present application, the initial reference frame set of a video frame can include one or more of the forward reference frame set (or referred to as a forward reference frame list) and the backward reference frame set.

[0068] S204 : Determine a target reference frame of a target coding block from the candidate reference frame set, and perform coding prediction on the target coding block based on the target reference frame.

[0069] In one implementation, a target reference frame for a target coding block can be determined from a set of candidate reference frames based on a motion search. For example, a motion search process can be performed on each reference frame in the set of candidate reference frames to obtain a corresponding target reference frame. After obtaining the target reference frame, coding prediction can be performed on the target coding block based on the target reference frame to obtain an encoded target coding block.

[0070] As previously mentioned, the candidate reference frame set can include one or more of a forward reference frame set and a backward reference frame set. When performing coding prediction for a target coding block, the encoder can select a frame from either the forward or backward candidate reference frame set as a reference frame. This coding prediction can be understood as unidirectional prediction. Alternatively, the encoder can select a frame from each of the two candidate reference frame sets (i.e., two frames in total) as a reference frame. This can be understood as bidirectional prediction. The coding standard does not specify the reference mode (unidirectional or bidirectional prediction) or the specific frame selected as the reference frame. Each frame selected from the candidate reference frame set can be encoded into a standard-compliant bitstream, but different selections may result in different coding efficiencies. To achieve optimal coding efficiency, the encoder typically performs a single pass through all possible reference frame selection schemes to search for the optimal reference frame (i.e., the target reference frame). Each encoding pass may include a highly complex motion search. This means that a motion search can be performed on each reference frame in the candidate reference frame set to obtain the corresponding target reference frame.

[0071] It's understandable that for a block to be encoded in a video, the encoder typically selects a region with the most similar pixel distribution within an already encoded video frame for prediction. This region can be called a similar region. The encoder then only needs to encode the location of this similar region and the pixel difference between the block to be encoded and the similar region. Generally speaking, the smaller the pixel difference, the fewer bytes required to be transmitted, and the higher the coding efficiency. However, if the encoder ultimately selects a region that isn't the most reasonable, but uses it for prediction, it can still generate a standard-compliant bitstream, but the coding efficiency will be compromised. Finding this most reasonable region is a computationally complex process, often achieved through pixel-by-pixel comparison, a process known as motion search. However, coding prediction (such as inter-frame prediction) is far more complex because there are often more than one encoded frame for motion search. The encoder often needs to perform motion search on all possible reference frames to determine which frame and region to use for prediction.

[0072] Based on this, it can be seen that if the number of video frames in the candidate reference frame set of the current encoder is large, the corresponding target reference frame selection process for the current encoder is relatively complex. If motion search is required for each video frame in the candidate reference frame set, the corresponding encoding complexity will also be high. High encoding complexity means that encoding requires more computing resources and longer encoding time, which makes the encoding efficiency lower. In the embodiment of the present application, the reference frame selection result of the target secondary encoding process (such as the first encoding process) can be used to accelerate the reference frame selection process of subsequent encoding. Because compared to selecting reference frames from the initial reference frame set corresponding to the target coding block, using a candidate reference frame set with a smaller number of frames can effectively accelerate the reference frame selection process of subsequent encoding; thereby, it can reduce computational complexity and save computing resources while basically not damaging the encoding efficiency. Compared with the original commonly used reference frame selection scheme, the embodiment of the present application can effectively improve the encoding efficiency.

[0073] In an embodiment of the present application, in a scenario where a video to be processed is encoded multiple times, and when a target coding block is encoded in the Nth encoding process, the reference frame selection results of each coding block in the target encoding process can be reused to achieve a reduction in the initial reference frame set corresponding to the target coding block, thereby reducing the computational complexity of selecting reference frames, saving computing resources, and reducing the selection time for selecting reference frames, thereby improving encoding efficiency.

[0074] See Figure 4 , is a flow chart of another video encoding method provided by an embodiment of the present application. This embodiment of the present application is mainly described with a computer device as the execution subject; see Figure 4 , the video encoding method may include the following steps S401-S405:

[0075] S401 : Obtain a video to be processed, and perform a first encoding process on each video frame in the video to be processed to obtain a plurality of encoding blocks included in each video frame and a reference frame of each encoding block.

[0076] S402 , when performing an Nth encoding process on a target video frame of a to-be-processed video, determining one or more reference encoding blocks overlapping with the target encoding block from a plurality of encoding blocks included in the target video frame during the target encoding process.

[0077] The specific implementation of steps S401 and S402 may refer to the specific implementation of steps S201 and S202 above, which will not be repeated here.

[0078] The target sub-encoding process is one or more encoding processes between the first encoding process and the N-1th encoding process. In one implementation, the target sub-encoding process can be any encoding process or any number of encoding processes randomly selected from between the first encoding process and the N-1th encoding process. In another implementation, in order to speed up the reference frame selection process for each encoding process other than the first encoding process, the target sub-encoding process can be preferentially set to the first encoding process, that is, the reference frame selection result of the first encoding process can be used in the second encoding process and subsequent encoding processes to assist in the selection of the reference frame, thereby improving the reference frame selection efficiency in each encoding process other than the first encoding process, thereby effectively improving the video encoding efficiency.

[0079] S403 : If there is no reference coding block using intra coding mode among the reference coding blocks, determine a candidate reference frame set of the target coding block based on the reference frames of the reference coding blocks in the target sub-coding process.

[0080] It is understood that the smaller the quantizer parameter (QP) of a video frame, the more detailed the motion search for that video frame. Therefore, the difference between the quantizer parameter of the current coded frame at the Nth encoding process and the quantizer parameter of the current coded frame at the target encoding process can be considered to determine the set of candidate reference frames for the target coding block. For ease of description, the quantizer parameter of the target video frame at the Nth encoding process and the quantizer parameter at the target encoding process can be referred to as the target quantizer parameter and the reference quantizer parameter, respectively.

[0081] Based on this, it can be seen that the specific implementation of determining the candidate reference frame set for the target coding block can be: first, the target quantization parameter of the target video frame in the Nth encoding process can be obtained, and the reference quantization parameter of the target video frame in the target encoding process can be obtained. After obtaining these target quantization parameters and reference quantization parameters, the target quantization parameters can be compared with the reference quantization parameters; then, based on the comparison results and the reference frames of each reference coding block in the target encoding process, the candidate reference frame set for the target coding block can be determined.

[0082] The comparison result may include: the target quantization parameter is greater than or equal to the reference quantization parameter, or the target quantization parameter is less than the reference quantization parameter. Different comparison results lead to different methods for determining the candidate reference frame set. The following describes the methods for determining the candidate reference frame set based on these two different comparison results.

[0083] (1) If the comparison result shows that the target quantization parameter is greater than or equal to the reference quantization parameter, the specific method of determining the candidate reference frame set is as follows:

[0084] The reference frames of each reference coding block in the target sub-encoding process can be added to the candidate reference frame set of the target coding block, that is, the candidate reference frame set of the target coding block can be constructed by the reference frames of each reference coding block in the target sub-encoding process.

[0085] It is understandable that, based on the principle of inter-frame prediction, when a video is encoded multiple times, although the quantization parameters of the video frames in the video change, the motion information of the video content is generally the same. Moreover, when the quantization parameter used in the subsequent N-th encoding of the video frame is larger than the quantization parameter used in the target encoding, the motion search result for the video frame may be coarser. Therefore, reusing the reference frame used in the target encoding will basically not cause a loss in encoding efficiency. Based on this, it can be known that the candidate reference frame set of the target coding block can be constructed from the reference frames of each reference coding block in the target encoding process, so that the reference frame corresponding to the target coding block (i.e., the target reference frame) can be directly determined from this candidate reference frame set.

[0086] It should be understood that the reference frame for a reference coding block in the target sub-encoding process is determined from the initial reference frame set, i.e., the candidate reference frame set is a subset of the initial reference frame set. This approach effectively reduces the initial reference frame set corresponding to the target coding block based on the reference frames of each reference coding block in the target sub-encoding process, allowing the target reference frame to be selected based on the candidate reference frame set with a smaller number of frames. This effectively reduces computational complexity, conserves computing resources, and substantially maintains coding efficiency.

[0087] In one embodiment, as mentioned above, the initial reference frame set of a video frame may include one or more of a forward reference frame set and a backward reference frame set.

[0088] When the initial reference frame set of a video frame includes a forward reference frame set or a backward reference frame set, for a reference coding block, the initial reference frame set corresponding to the reference coding block includes a forward reference frame set or a backward reference frame set, and the reference frame corresponding to the reference coding block is selected from the forward reference frame set or the backward reference frame set. However, if the initial reference frame set for each reference coding block in the same video frame is the same, that is, the reference frame of each reference coding block is selected from the forward reference frame set or the backward reference frame set, then the candidate reference frame set corresponding to the target coding block may include the forward candidate reference frame set or the backward candidate reference frame set.

[0089] For example, if the initial reference frame set of the target video frame is a forward reference frame set, then the reference frames corresponding to each reference coding block are selected from the forward reference frame set; further, it can be seen that the candidate reference frame set constructed by the reference frames corresponding to each reference coding block is a forward candidate reference frame set. For another example, if the initial reference frame set of the target video frame is a backward reference frame set, then the reference frames corresponding to each reference coding block are selected from the backward reference frame set; further, it can be seen that the candidate reference frame set constructed by the reference frames corresponding to each reference coding block is a backward candidate reference frame set.

[0090] When the initial reference frame set of a video frame includes a forward reference frame set and a backward reference frame set, for a reference coding block, the initial reference frame set corresponding to the reference coding block includes a forward reference frame set and a backward reference frame set, then the reference frame corresponding to the reference coding block is selected from the forward reference frame set and the backward reference frame set. For example, a video frame can be selected from the forward reference frame set and the backward reference frame set as a reference frame respectively, then the candidate reference frame set corresponding to the target coding block can include a forward candidate reference frame set and a backward candidate reference frame set.

[0091] For example, if the initial reference frame set of the target video frame is a forward reference frame set and a backward reference frame set, then the reference frames corresponding to each reference coding block are selected from the forward reference frame set and the backward reference frame set, such that the reference frames corresponding to each reference coding block include a video frame selected from the forward reference frame set and a video frame selected from the backward reference frame set. Furthermore, it can be seen that the reference frames selected from the forward reference frame set by each reference coding block can be constructed as a forward candidate reference frame set, and the reference frames selected from the backward reference frame set by each reference coding block can be constructed as a backward candidate reference frame set.

[0092] In one embodiment, after the computer device performs the first encoding process on each video frame in the video to be processed in step S401 to obtain multiple encoding blocks included in each video frame and the reference frame of each encoding block, the computer device may also save the reference frame information of the reference frame corresponding to each encoding block. Optionally, when performing each encoding process between the first encoding process and the N-1th encoding process, the reference frame information of the reference frame corresponding to each encoding block during each encoding process may be saved, so that when performing the Nth encoding process, the reference frame information of the reference frame corresponding to each encoding block during one or more encoding processes between the first encoding process and the N-1th encoding process may be used to determine the corresponding candidate reference frame set.

[0093] Among them, the reference frame information of the reference frame corresponding to a coding block may include the index of the reference frame in the initial reference frame set, and the index may refer to the number of the reference frame in the initial reference frame set, so that the corresponding reference frame can be directly determined based on the index. Among them, the number can be a digital number, an alphabetical number, etc., and is not specifically limited. For example, if the reference frame information of the reference frame corresponding to a coding block is 2, it indicates that the reference frame of the coding block is the second video frame in the corresponding initial reference frame set; if the reference frame information of the reference frame corresponding to a coding block is 5, it indicates that the reference frame of the coding block is the fifth video frame in the corresponding initial reference frame set.

[0094] Optionally, if the coding prediction mode is bidirectional prediction, the initial reference frame set includes a forward reference frame set and a backward reference frame set, that is, the reference frame information of the reference frame corresponding to a coding block includes the index of the reference frame in the forward reference frame set and the backward reference frame set. If the coding prediction mode is unidirectional prediction, the initial reference frame set includes a forward reference frame set or a backward reference frame set, that is, the index of a certain reference frame set will not exist. For example, if the initial reference frame set includes a forward reference frame set, the reference frame information of the reference frame corresponding to a coding block includes the index of the reference frame in the forward reference frame set. For another example, if the initial reference frame set includes a backward reference frame set, the reference frame information of the reference frame corresponding to a coding block includes the index of the reference frame in the backward reference frame set.

[0095] Based on this, it can be seen that the above-mentioned determination of the candidate reference frame set of the target coding block can be implemented directly based on the reference frame information (i.e., index). In a specific implementation, the reference frame information of the reference frames of each reference coding block can be obtained first. After obtaining the reference frame information, the initial reference frame set corresponding to the target coding block is filtered based on the reference frame information. The initial reference frame set obtained by filtering is the candidate reference frame set. Optionally, the specific implementation of the filtering operation can be: for the initial reference frame set corresponding to the target coding block, retain the video frames with the index indicated by the reference frame information corresponding to each reference coding block, and filter out the video frames with the index not indicated by the reference frame information corresponding to each reference coding block.

[0096] For example, taking the example that the initial reference frame set includes either a forward reference frame set or a backward reference frame set, assuming that the initial reference frame set corresponding to the target coding block is a forward reference frame set, and the forward reference frame set is {video frame 1 video frame 2 video frame 3 video frame 4}; the reference coding blocks corresponding to the target coding block include reference coding block 1 and reference coding block 2, the index of the reference frame corresponding to reference coding block 1 is 2, and the index of the reference frame corresponding to reference coding block 2 is 3. Then the candidate reference frame set of the target coding block is {video frame 2 video frame 3}, and the candidate reference frame set may refer to the forward candidate reference frame set.

[0097] For another example, taking the example that the initial reference frame set includes a forward reference frame set and a backward reference frame set, assuming that the forward reference frame set corresponding to the target coding block is {video frame 11, video frame 12, video frame 13, video frame 14}, and the backward reference frame set is {video frame 21, video frame 22, video frame 23, video frame 24}; the reference coding blocks corresponding to the target coding block include reference coding block 1 and reference coding block 2, the reference frame corresponding to reference coding block 1 has an index of 12 in the forward reference frame set and an index of 21 in the backward reference frame set; the reference frame corresponding to reference coding block 2 has an index of 14 in the forward reference frame set and an index of 23 in the backward reference frame set. Then, the candidate reference frame set of the target coding block includes a forward candidate reference frame set, which is {video frame 12, video frame 14}, and a backward candidate reference frame set, which is {video frame 21, video frame 23}.

[0098] (2) If the comparison result shows that the target quantization parameter is less than the reference quantization parameter, the specific method of determining the candidate reference frame set is as follows:

[0099] The video frame with the shortest playback distance to the target video frame in the playback of the video to be processed can be obtained from the initial reference frame set corresponding to the target coding block, and the obtained video frame and the reference frame of each reference coding block in the target sub-encoding process are added to the candidate reference frame set of the target coding block. Optionally, if the initial reference frame set includes a forward reference frame set or a backward reference frame set, a video frame with the shortest playback distance to the target video frame can be selected from the forward reference frame set or the backward reference frame set respectively. If the initial reference frame set includes a forward reference frame set and a backward reference frame set, a video frame with the shortest playback distance to the target video frame can be selected from the forward reference frame set and the backward reference frame set respectively.

[0100] It's understandable that when the quantization parameter is smaller, the motion search is more refined, while when the quantization parameter is larger, the search results are more likely to be suboptimal. The two frames closest in playback distance typically have the greatest correlation with the target video frame's image content, making them most likely to be ultimately selected. Therefore, it's possible to consider making them available to mitigate the loss in coding efficiency caused by disabling the optimal reference frame. To further mitigate this loss in coding efficiency, the video frames in the initial reference frame set closest to the target video frame can be made available, even if they aren't in the set consisting of reference frames used by one or more reference coding blocks corresponding to the target sub-encoding process.

[0101] In summary, it can be seen that when the comparison result is that the target quantization parameter is smaller than the reference quantization parameter, the initial reference frame set corresponding to the target coding block can also be reduced to obtain the corresponding candidate reference frame set, thereby reducing the computational complexity and saving computing resources of computer equipment without substantially damaging the coding efficiency.

[0102] It should be noted that, when the comparison result is that the target quantization parameter is less than the reference quantization parameter, the candidate reference frame set of the target coding block may also involve one or more of the forward candidate reference frame set and the backward candidate reference frame set. The relevant understanding can refer to the situation in (1) above, which will not be repeated here.

[0103] It should be noted that, considering that the target sub-encoding process may be one or more encoding processes between the first encoding process and the N-1th encoding process, and when the target sub-encoding process is one encoding process between the first encoding process and the N-1th encoding process, the reference quantization parameter is a parameter, and when the quantization parameters are compared, a comparison result can be obtained (the target quantization parameter is greater than or equal to the reference quantization parameter, or the target quantization parameter is less than the reference quantization parameter). In this case, the method for determining the candidate reference frame set can be as described in (1) or (2) above.

[0104] If the target encoding process is a plurality of encoding processes between the first encoding process and the N-1th encoding process, the reference quantization parameter is a plurality of parameters. When comparing the quantization parameters, each reference quantization parameter needs to be compared with the target quantization parameter. The comparison result may include one or more parameters, such as the target quantization parameter being greater than or equal to each reference quantization parameter, or the target quantization parameter being less than each reference quantization parameter, or the target quantization parameter being greater than or equal to a portion of the reference quantization parameters and the target quantization parameter being less than another portion of the reference quantization parameters. In this case, if the target quantization parameter is greater than or equal to each reference quantization parameter, the method for determining the candidate reference frame set may be as described in (1) above; if the target quantization parameter is less than each reference quantization parameter, the method for determining the candidate reference frame set may be as described in (2) above; if the target quantization parameter is greater than or equal to a portion of the reference quantization parameters and the target quantization parameter is less than another portion of the reference quantization parameters, the method for determining the candidate reference frame set may be combined with the descriptions in (1) and (2) above.

[0105] S404: If there is a reference coding block using an intra-frame coding mode among the reference coding blocks, the initial reference frame set corresponding to the reference coding block is used as a candidate reference frame set of the target coding block.

[0106] S405 , determining a target reference frame of a target coding block from the candidate reference frame set, and performing coding prediction on the target coding block based on the target reference frame.

[0107] In one implementation, as described above, if the coding prediction mode of the reference coding block in the target secondary coding process is a unidirectional prediction mode, the candidate reference frame set may include any one of a forward candidate reference frame set or a backward candidate reference frame set; then, the specific implementation of determining the target reference frame of the target coding block from the candidate reference frame set in step S405 may be: first, a motion search process may be performed on each reference frame in the forward candidate reference frame set or the backward candidate reference frame set; and the target reference frame may be determined from the forward reference frame set or the backward reference frame set based on the motion search process. The target reference frame may include a reference frame in the forward candidate reference frame set or the backward candidate reference frame set.

[0108] In one implementation, if the coding prediction mode of the reference coding block in the target secondary coding process is a bidirectional prediction mode, the candidate reference frame set includes a forward candidate reference frame set and a backward candidate reference frame set; then, the specific implementation of determining the target reference frame of the target coding block from the candidate reference frame set in step S405 may be: first, a motion search process may be performed on each reference frame in the forward candidate reference frame set and the backward candidate reference frame set; and based on the motion search process, a target reference frame is determined from the forward candidate reference frame set and the backward candidate reference frame set, respectively. The target reference frame may include one reference frame in the forward candidate reference frame set and one reference frame in the backward candidate reference frame set.

[0109] In one implementation, after determining the target reference frame from the candidate reference frame set, the target reference frame can be searched for an area with the most similar pixel distribution to the target coding block based on motion search, and the searched area can be used to perform coding prediction on the target coding block to obtain the encoded target coding block. For example, see Figure 5a As shown, if the candidate reference frame set includes a forward candidate reference frame set or a backward candidate reference frame set, a motion search can be performed on the target reference frame for the candidate reference frame set or the backward candidate reference frame set to search for an area with the most similar pixel distribution to the target coding block, and the searched area is used for coding prediction. For another example, see Figure 5b As shown, if the candidate reference frame set includes a forward candidate reference frame set and a backward candidate reference frame set, motion search can be performed on the target reference frames corresponding to the candidate reference frame set and the backward candidate reference frame set, respectively, to search for the area with the most similar pixel distribution to the target coding block, and use the searched area for coding prediction.

[0110] To better understand the video encoding method proposed in the embodiment of the present application, the following Figure 5c The video encoding is further described, wherein the target encoding process is taken as the first encoding process as an example. For example, see Figure 5c As shown, the target coding block in the current video to be processed in the Nth encoding process is used as an example for the relevant description. When the target coding block needs to be encoded, the reference frame information of the reference frame corresponding to the reference coding block can be obtained, where the reference coding block refers to the coding block that has an overlapping relationship with the target coding block in the coding block division result of the target video frame in the first encoding process.

[0111] After obtaining the reference frame information of the reference coding block, it is possible to further determine whether there is a coding block using an intra-frame prediction mode in the reference coding block. If there is no coding block using an intra-frame prediction mode, the reference frame corresponding to the reference coding block can be used to reduce the initial reference frame set of the target coding block to obtain a candidate reference frame set. As mentioned above, the reference frame information may include the index of the reference frame in the initial reference frame set, that is, the initial reference frame set of the target coding block is reduced based on the video frame indicated by the index in the reference frame information. If there is a coding block using an intra-frame prediction mode, the initial reference frame set corresponding to the reference coding block can be used as a candidate reference frame set.

[0112] Optionally, in the case where there is no coding block using an intra-frame prediction mode as mentioned above, the candidate reference frame set of the target coding block can be further determined based on the size relationship between the target quantization parameter of the target video frame in the Nth encoding process and the reference quantization parameter of the target video frame in the first encoding process. For example, if the target quantization parameter is greater than or equal to the reference quantization parameter, the reference frames of each reference coding block in the first encoding process can be added to the candidate reference frame set of the target coding block. If the target quantization parameter is less than the reference quantization parameter, the video frame in the initial reference frame set corresponding to the target coding block that is closest to the target video frame in playback can be listed as an available candidate reference frame, that is, the video frame with the shortest playback distance to the target video frame in playback of the to-be-processed video can be obtained from the initial reference frame set corresponding to the target coding block, and the obtained video frame and the reference frame of each reference coding block in the first encoding process can be added to the candidate reference frame set of the target coding block.

[0113] After determining the candidate reference frame set, each video frame in the candidate reference frame can be encoded to determine the reference mode (coding prediction mode) and the target reference frame from the candidate reference frame set. In a specific implementation, motion search processing can be performed on each video frame in the candidate reference frame to determine the reference mode and the target reference frame. The reference mode can be determined based on the specific set included in the candidate reference frame set. If the candidate reference frame set includes a forward candidate reference frame set or a backward reference frame set, the reference mode is a unidirectional prediction mode. If the candidate reference frame set includes a forward candidate reference frame set and a backward reference frame set, the reference mode is a bidirectional prediction mode.

[0114] In an embodiment of the present application, in a scenario where a video to be processed is encoded multiple times, when encoding a target coding block in the Nth encoding process (such as the second encoding process and subsequent encoding processes), the reference frame selection results of each coding block in the target encoding process can be reused to reduce the initial reference frame set corresponding to the target coding block, thereby reducing the computational complexity of selecting the reference frame, thereby reducing the bitstream encoding time and computing resource overhead, thereby improving coding efficiency and saving computing resources; and can reduce the selection time for selecting the reference frame, and can also improve coding efficiency.

[0115] See also Figure 6 , is a schematic diagram of the structure of a video encoding device provided in an embodiment of the present application. The video encoding device described in this embodiment includes:

[0116] The first encoding unit 601 is configured to obtain a video to be processed and perform a first encoding process on each video frame in the video to be processed to obtain multiple coding blocks included in each video frame and a reference frame for each coding block. The multiple coding blocks included in a video frame are obtained by dividing the video frame into blocks, and the reference frame for a coding block is a video frame used in encoding prediction for the coding block in the video to be processed.

[0117] A first determining unit 602 is configured to, when performing an Nth encoding process on a target video frame of the video to be processed, determine, from a plurality of encoding blocks included in the target video frame during the target encoding process, one or more reference encoding blocks that overlap with the target encoding block; the target video frame is the video frame where the target encoding block is located, the target encoding process is one or more encoding processes between the first encoding process and the N-1th encoding process, where N is a positive integer greater than 1;

[0118] A second determining unit 603 is configured to determine a candidate reference frame set for the target coding block based on a coding mode of each reference coding block and a reference frame of each reference coding block in the target secondary encoding process; the coding mode includes an inter-frame coding mode or an intra-frame coding mode;

[0119] The second encoding unit 604 is configured to determine a target reference frame of the target coding block from the candidate reference frame set, and perform encoding prediction on the target coding block based on the target reference frame.

[0120] In one implementation, the second determining unit 603 is specifically configured to:

[0121] If there is no reference coding block using the intra coding mode among the reference coding blocks, determining a candidate reference frame set for the target coding block based on reference frames of the reference coding blocks in the target sub-encoding process;

[0122] If there is a reference coding block using the intra-frame coding mode among the reference coding blocks, the initial reference frame set corresponding to the reference coding block is used as the candidate reference frame set of the target coding block; the reference frame of any reference coding block among the one or more reference coding blocks exists in the initial reference frame set corresponding to the any reference coding block; the initial reference frame sets corresponding to different coding blocks in the same video frame are the same.

[0123] In one implementation, the second determining unit 603 is specifically configured to:

[0124] Obtaining a target quantization parameter of a target video frame during the Nth encoding process, and obtaining a reference quantization parameter of the target video frame during the target encoding process;

[0125] Comparing the target quantization parameter with the reference quantization parameter;

[0126] Based on the comparison result and the reference frames of each reference coding block in the target sub-encoding process, a candidate reference frame set of the target coding block is determined.

[0127] In one implementation, the second determining unit 603 is specifically configured to:

[0128] If the comparison result shows that the target quantization parameter is greater than or equal to the reference quantization parameter, adding the reference frame of each reference coding block in the target sub-encoding process to the candidate reference frame set of the target coding block;

[0129] If the comparison result is that the target quantization parameter is less than the reference quantization parameter, the video frame with the shortest playback distance to the target video frame in the playback of the video to be processed is obtained from the initial reference frame set corresponding to the target coding block, and the obtained video frame and the reference frames of each reference coding block in the target sub-encoding processing are added to the candidate reference frame set of the target coding block.

[0130] In one implementation, the first determining unit 602 is specifically configured to:

[0131] Obtaining a regional position of the target coding block in the target video frame during the Nth encoding process;

[0132] Obtaining a regional position of each coding block in the target video frame among a plurality of coding blocks corresponding to the target video frame during the target secondary encoding process;

[0133] According to the regional position of each coding block in the target video frame, a coding block having an overlapping area with the regional position of the target coding block is determined from the multiple coding blocks, and the determined coding block is used as a reference coding block having an overlapping relationship with the target coding block.

[0134] In one implementation, the coding prediction mode of the reference coding block in the target secondary encoding process is a unidirectional prediction mode, and the candidate reference frame set includes any one of a forward candidate reference frame set or a backward candidate reference frame set; the second coding unit 604 is specifically configured to:

[0135] Performing motion search processing on each reference frame in the forward candidate reference frame set or the backward candidate reference frame set;

[0136] A target reference frame is determined from the forward reference frame set or the backward reference frame set based on the motion search process; the target reference frame includes a reference frame from the forward candidate reference frame set or the backward candidate reference frame set.

[0137] In one implementation, the coding prediction mode of the reference coding block in the target secondary encoding process is a bidirectional prediction mode, and the candidate reference frame set includes a forward candidate reference frame set and a backward candidate reference frame set; the second coding unit 604 is specifically configured to:

[0138] Performing motion search processing on each reference frame in the forward candidate reference frame set and the backward candidate reference frame set;

[0139] Based on the motion search process, a target reference frame is determined from the forward candidate reference frame set and the backward candidate reference frame set respectively; the target reference frame includes a reference frame in the forward candidate reference frame set and a reference frame in the backward candidate reference frame set.

[0140] It is understood that the division of units in the embodiments of the present application is schematic and is merely a logical functional division. In actual implementation, other division methods may be used. The functional units in the embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0141] See also Figure 7, is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. The computer device described in this embodiment includes: a processor 701, a memory 702, and a network interface 703. The processor 701, the memory 702, and the network interface 703 can exchange data.

[0142] The processor 701 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or any conventional processor.

[0143] The memory 702 may include a read-only memory and a random access memory, and provide program instructions and data to the processor 701. A portion of the memory 702 may also include a non-volatile random access memory. When the processor 701 calls the program instructions, it is used to execute:

[0144] Obtaining a video to be processed, and performing a first encoding process on each video frame in the video to be processed to obtain a plurality of encoding blocks included in each video frame and a reference frame for each encoding block; the plurality of encoding blocks included in a video frame are obtained by dividing the video frame into blocks, and the reference frame for a encoding block is a video frame used in encoding prediction for the encoding block in the video to be processed;

[0145] When performing an Nth encoding process on a target video frame of the video to be processed, determining one or more reference encoding blocks that overlap with the target encoding block from a plurality of encoding blocks included in the target video frame during the target encoding process; the target video frame is a video frame containing the target encoding block, the target encoding process is one or more encoding processes between a first encoding process and an N-1th encoding process, where N is a positive integer greater than 1;

[0146] Determining a candidate reference frame set for the target coding block based on a coding mode of each reference coding block and a reference frame of each reference coding block in the target secondary coding process; the coding mode includes an inter-frame coding mode or an intra-frame coding mode;

[0147] A target reference frame of the target coding block is determined from the candidate reference frame set, and coding prediction is performed on the target coding block based on the target reference frame.

[0148] In one implementation, the processor 701 is specifically configured to:

[0149] If there is no reference coding block using the intra coding mode among the reference coding blocks, determining a candidate reference frame set for the target coding block based on reference frames of the reference coding blocks in the target sub-encoding process;

[0150] If there is a reference coding block using the intra-frame coding mode among the reference coding blocks, the initial reference frame set corresponding to the reference coding block is used as the candidate reference frame set of the target coding block; the reference frame of any reference coding block among the one or more reference coding blocks exists in the initial reference frame set corresponding to the any reference coding block; the initial reference frame sets corresponding to different coding blocks in the same video frame are the same.

[0151] In one implementation, the processor 701 is specifically configured to:

[0152] Obtaining a target quantization parameter of a target video frame during the Nth encoding process, and obtaining a reference quantization parameter of the target video frame during the target encoding process;

[0153] Comparing the target quantization parameter with the reference quantization parameter;

[0154] Based on the comparison result and the reference frames of each reference coding block in the target sub-encoding process, a candidate reference frame set of the target coding block is determined.

[0155] In one implementation, the processor 701 is specifically configured to:

[0156] If the comparison result shows that the target quantization parameter is greater than or equal to the reference quantization parameter, adding the reference frame of each reference coding block in the target sub-encoding process to the candidate reference frame set of the target coding block;

[0157] If the comparison result is that the target quantization parameter is less than the reference quantization parameter, the video frame with the shortest playback distance to the target video frame in the playback of the video to be processed is obtained from the initial reference frame set corresponding to the target coding block, and the obtained video frame and the reference frames of each reference coding block in the target sub-encoding processing are added to the candidate reference frame set of the target coding block.

[0158] In one implementation, the processor 701 is specifically configured to:

[0159] Obtaining a regional position of the target coding block in the target video frame during the Nth encoding process;

[0160] Obtaining a regional position of each coding block in the target video frame among a plurality of coding blocks corresponding to the target video frame during the target secondary encoding process;

[0161] According to the regional position of each coding block in the target video frame, a coding block having an overlapping area with the regional position of the target coding block is determined from the multiple coding blocks, and the determined coding block is used as a reference coding block having an overlapping relationship with the target coding block.

[0162] In one implementation, the coding prediction mode of the reference coding block in the target secondary encoding process is a unidirectional prediction mode, and the candidate reference frame set includes any one of a forward candidate reference frame set or a backward candidate reference frame set; the processor 701 is specifically configured to:

[0163] Performing motion search processing on each reference frame in the forward candidate reference frame set or the backward candidate reference frame set;

[0164] A target reference frame is determined from the forward reference frame set or the backward reference frame set based on the motion search process; the target reference frame includes a reference frame from the forward candidate reference frame set or the backward candidate reference frame set.

[0165] In one implementation, the coding prediction mode of the reference coding block in the target secondary coding process is a bidirectional prediction mode, and the candidate reference frame set includes a forward candidate reference frame set and a backward candidate reference frame set; the processor 701 is specifically configured to:

[0166] Performing motion search processing on each reference frame in the forward candidate reference frame set and the backward candidate reference frame set;

[0167] Based on the motion search process, a target reference frame is determined from the forward candidate reference frame set and the backward candidate reference frame set respectively; the target reference frame includes a reference frame in the forward candidate reference frame set and a reference frame in the backward candidate reference frame set.

[0168] It should be noted that for the aforementioned various method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0169] An embodiment of the present application further provides a computer-readable storage medium, in which program instructions are stored. When the program instructions are executed, some or all of the steps in the above method are implemented, which are not repeated here.

[0170] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0171] The present application also provides a computer program product or computer program, which includes program instructions that, when executed by a processor, implement some or all of the steps in the above-described method. Optionally, the program instructions are stored in a computer-readable storage medium. A processor of a computer device reads the program instructions from the computer-readable storage medium and executes the program instructions, causing the computer device to perform the steps performed in the above-described method embodiments.

[0172] The above is a detailed introduction to a video encoding method, device, computer equipment and storage medium provided in the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present application.

Claims

1. A video encoding method, characterized in that: The method comprises: Obtaining a video to be processed, and performing a first encoding process on each video frame in the video to be processed to obtain a plurality of encoding blocks included in each video frame and a reference frame for each encoding block; the plurality of encoding blocks included in a video frame are obtained by dividing the video frame into blocks, and the reference frame for a encoding block is a video frame used in encoding prediction for the encoding block in the video to be processed; When performing an Nth encoding process on a target video frame of the video to be processed, determining one or more reference encoding blocks that overlap with the target encoding block from a plurality of encoding blocks included in the target video frame during the target encoding process; the target video frame is a video frame containing the target encoding block, the target encoding process is one or more encoding processes between a first encoding process and an N-1th encoding process, where N is a positive integer greater than 1; If no reference coding block using the intra coding mode exists among the one or more reference coding blocks, determining a candidate reference frame set for the target coding block based on reference frames of the one or more reference coding blocks in the target sub-encoding process; If a reference coding block using an intra-frame coding mode exists among the one or more reference coding blocks, an initial reference frame set corresponding to the reference coding block is used as a candidate reference frame set for the target coding block; a reference frame of any reference coding block among the one or more reference coding blocks exists in the initial reference frame set corresponding to the any reference coding block; and different coding blocks in the same video frame correspond to the same initial reference frame set; A target reference frame of the target coding block is determined from the candidate reference frame set, and coding prediction is performed on the target coding block based on the target reference frame.

2. The method according to claim 1, characterized in that The determining, based on the reference frames of the one or more reference coding blocks in the target sub-encoding process, a candidate reference frame set for the target coding block includes: Obtaining a target quantization parameter of a target video frame during the Nth encoding process, and obtaining a reference quantization parameter of the target video frame during the target encoding process; Comparing the target quantization parameter with the reference quantization parameter; Based on the comparison result and the reference frames of the one or more reference coding blocks in the target sub-encoding process, a candidate reference frame set for the target coding block is determined.

3. The method according to claim 2, characterized in that The determining, based on the comparison result and the reference frames of the one or more reference coding blocks in the target sub-encoding process, a candidate reference frame set for the target coding block includes: If the comparison result shows that the target quantization parameter is greater than or equal to the reference quantization parameter, adding the reference frames of the one or more reference coding blocks in the target sub-encoding process to the candidate reference frame set of the target coding block; If the comparison result is that the target quantization parameter is less than the reference quantization parameter, then the video frame with the shortest playback distance to the target video frame in the playback of the video to be processed is obtained from the initial reference frame set corresponding to the target coding block, and the obtained video frame and the reference frames of the one or more reference coding blocks in the target sub-encoding processing are added to the candidate reference frame set of the target coding block.

4. The method according to claim 1, wherein The step of determining, from a plurality of coding blocks corresponding to the target video frame during the target sub-encoding process, one or more reference coding blocks overlapping with the target coding block comprises: Obtaining a regional position of the target coding block in the target video frame during the Nth encoding process; Obtaining a regional position of each coding block in the target video frame among a plurality of coding blocks corresponding to the target video frame during the target secondary encoding process; According to the regional position of each coding block in the target video frame, a coding block having an overlapping area with the regional position of the target coding block is determined from the multiple coding blocks, and the determined coding block is used as a reference coding block having an overlapping relationship with the target coding block.

5. The method according to claim 1, wherein The coding prediction mode of the reference coding block in the target sub-coding process is a unidirectional prediction mode, and the candidate reference frame set includes any one of a forward candidate reference frame set and a backward candidate reference frame set; The determining a target reference frame of the target coding block from the candidate reference frame set includes: Performing motion search processing on each reference frame in the forward candidate reference frame set or the backward candidate reference frame set; A target reference frame is determined from the forward reference frame set or the backward reference frame set based on the motion search process; the target reference frame includes a reference frame from the forward candidate reference frame set or the backward candidate reference frame set.

6. The method according to claim 1, characterized in that The coding prediction mode of the reference coding block in the target secondary coding process is a bidirectional prediction mode, and the candidate reference frame set includes a forward candidate reference frame set and a backward candidate reference frame set; The determining a target reference frame of the target coding block from the candidate reference frame set includes: Performing motion search processing on each reference frame in the forward candidate reference frame set and the backward candidate reference frame set; Based on the motion search process, a target reference frame is determined from the forward candidate reference frame set and the backward candidate reference frame set respectively; the target reference frame includes a reference frame in the forward candidate reference frame set and a reference frame in the backward candidate reference frame set.

7. A video encoding device, characterized in that include: a first encoding unit, configured to obtain a video to be processed and perform a first encoding process on each video frame in the video to be processed to obtain a plurality of encoding blocks included in each video frame and a reference frame for each encoding block; the plurality of encoding blocks included in a video frame are obtained by dividing the video frame into blocks, and the reference frame for a encoding block is a video frame used in encoding prediction for the encoding block in the video to be processed; a first determining unit configured to, when performing an Nth encoding process on a target video frame of the to-be-processed video, determine, from a plurality of encoding blocks included in the target video frame during the target encoding process, one or more reference encoding blocks that overlap with the target encoding block; the target video frame being the video frame containing the target encoding block, the target encoding process being one or more encoding processes between the first encoding process and the N-1th encoding process, where N is a positive integer greater than 1; a second determining unit, configured to determine, if no reference coding block using an intra coding mode exists among the one or more reference coding blocks, a candidate reference frame set for the target coding block based on reference frames of the one or more reference coding blocks in the target sub-encoding process; If a reference coding block using an intra-frame coding mode exists among the one or more reference coding blocks, an initial reference frame set corresponding to the reference coding block is used as a candidate reference frame set for the target coding block; a reference frame of any reference coding block among the one or more reference coding blocks exists in the initial reference frame set corresponding to the any reference coding block; Different coding blocks in the same video frame correspond to the same set of initial reference frames; The second encoding unit is configured to determine a target reference frame of the target coding block from the candidate reference frame set, and perform encoding prediction on the target coding block based on the target reference frame.

8. A computer device, characterized in that: The method comprises a processor and a memory, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores program instructions, which, when executed by a processor, are used to implement the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The computer program product comprises program instructions, and when the program instructions are executed by a processor, they are used to implement the method according to any one of claims 1 to 6.

Images