Video encoding method, apparatus, computer device and storage medium

Abstract

The application relates to a video coding method, device, computer equipment and storage medium. The method comprises the following steps: acquiring a source code stream to be coded, and determining the arrangement structure of video frames in the source code stream; according to the arrangement structure, performing extraction processing on the plurality of video frames in the source code stream to obtain a plurality of candidate code streams with different code stream structures; identifying the reference video frames in each candidate code stream, and determining the target video frames and the regular video frames respectively associated with each reference video frame; respectively coding the target video frames associated with each reference video frame according to each reference video frame to obtain the initial coding result of the candidate code stream of each code stream structure; and for the candidate code stream of each code stream structure, obtaining the target code stream corresponding to the candidate code stream of the corresponding code stream structure according to the initial coding result corresponding to the candidate code stream and the regular video frames. The method can improve the flexibility of video coding.

Description

Technical Field

[0001] This application relates to the field of video coding, and in particular to a video coding method, apparatus, computer equipment, and storage medium. Background Technology

[0002] With the explosive growth of consumer electronics, the demand for multi-screen interaction has become increasingly urgent, necessitating real-time video transmission between different terminals. Currently, during video transmission, due to varying network bandwidth conditions, terminal processing capabilities, and user quality requirements across different scenarios, servers need to encode the video sent by the current terminal multiple times to ensure that other terminals can successfully receive and parse the video.

[0003] However, encoding the same video multiple times reduces the efficiency of video encoding and transmission, and also reduces the flexibility of different terminals accurately receiving the video. Therefore, how to generate videos suitable for different network bandwidths and other scenarios through a single encoding is the problem that this application aims to solve. Summary of the Invention

[0004] Therefore, it is necessary to provide a video coding method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can improve the practicality of video coding in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a video encoding method. The method includes:

[0006] Obtain the source stream to be encoded and determine the arrangement structure of the video frames in the source stream;

[0007] Based on the arrangement structure, multiple video frames in the source stream are extracted and processed to obtain multiple candidate streams with different stream structures.

[0008] Identify reference video frames in each of the candidate bitstreams, and determine the target video frame and regular video frame associated with each of the reference video frames;

[0009] Each of the reference video frames is used to encode the associated target video frame to obtain the initial encoding result of the candidate bitstream for each bitstream structure;

[0010] For each of the multiple candidate bitstreams with different bitstream structures, the target bitstream corresponding to the candidate bitstream structure is obtained based on the initial encoding result and regular video frames corresponding to the candidate bitstream.

[0011] In one embodiment, determining the arrangement structure of video frames in the source stream includes: performing field encoding on the source stream to obtain a plurality of sequentially arranged video frames; the frame number of each video frame is n, where n is a positive integer; determining video frames with special frame numbers from the plurality of video frames; the special frame numbers include at least n+2; and obtaining the arrangement structure of video frames in the source stream based on the video frames with special frame numbers.

[0012] In one embodiment, obtaining the arrangement structure of video frames in the source stream based on the video frames with the special frame number includes: obtaining video frames with special identifiers and reference relationships between each video frame with a special identifier in response to an association operation on the video frames with the special frame number; obtaining video frames with regular identifiers in response to a configuration operation on video frames with other frame numbers among the plurality of video frames; and obtaining the arrangement structure of video frames in the source stream by combining the special identifiers and regular identifiers of the video frames and each of the reference relationships.

[0013] In one embodiment, each reference video frame is used to encode an associated target video frame to obtain an initial encoding result for a candidate bitstream of each bitstream structure. This includes: determining the current reference video frame in the candidate bitstream and determining the current target video frame associated with the current reference video frame; performing inter-frame prediction on the current reference video frame to obtain a current prediction result; superimposing the current prediction result with the current target video frame to obtain a first encoding result corresponding to the current target video frame; using the first encoding result as a new current reference video frame and returning to the step of determining the current target video frame associated with the current reference video frame to continue execution until the first encoding result of the target video frame associated with the last reference video frame in the candidate bitstream is obtained; and using the first encoding result of the target video frame associated with the last reference video frame as the initial encoding result of the candidate bitstream.

[0014] In one embodiment, the bitstream structure includes a full bitstream structure; obtaining the target bitstream corresponding to the candidate bitstream of the corresponding bitstream structure based on the initial encoding result corresponding to the candidate bitstream and regular video frames includes: determining a current reference video frame associated with a regular video frame in the candidate bitstream of the full bitstream structure; if the current reference video frame is the first reference video frame, then superimposing each regular video frame in the candidate bitstream of the full bitstream structure with the current reference video frame to obtain a second encoding result corresponding to each regular video frame; if the current reference video frame is not the first reference video frame, then determining a first encoding result of the current reference video frame, and superimposing each regular video frame in the candidate bitstream of the full bitstream structure with the first encoding result to obtain a second encoding result corresponding to each regular video frame; combining each second encoding result and the initial encoding result of the candidate bitstream of the full bitstream structure to obtain the target bitstream corresponding to the candidate bitstream of the full bitstream structure.

[0015] In one embodiment, the bitstream structure includes a half-bitstream structure; obtaining the target bitstream corresponding to the candidate bitstream of the corresponding bitstream structure based on the initial encoding result corresponding to the candidate bitstream and regular video frames includes: encoding and padding each regular video frame in the candidate bitstream of the half-bitstream structure to obtain a second encoding result corresponding to each regular video frame; and combining each second encoding result and the initial encoding result of the candidate bitstream of the half-bitstream structure to obtain the target bitstream corresponding to the candidate bitstream of the half-bitstream structure.

[0016] Secondly, this application also provides a video encoding apparatus. The apparatus includes:

[0017] The candidate bitstream determination module is used to acquire the source bitstream to be encoded and determine the arrangement structure of the video frames in the source bitstream; according to the arrangement structure, it extracts and processes multiple video frames in the source bitstream to obtain multiple candidate bitstreams with different bitstream structures.

[0018] The video frame recognition module is used to identify reference video frames in each of the candidate bitstreams and determine the target video frame and regular video frame associated with each of the reference video frames.

[0019] The target bitstream determination module is used to encode the associated target video frame according to each of the reference video frames to obtain the initial encoding result of the candidate bitstream for each bitstream structure; for each of the multiple candidate bitstreams with different bitstream structures, the target bitstream corresponding to the candidate bitstream for the corresponding bitstream structure is obtained according to the initial encoding result of the candidate bitstream and the regular video frame.

[0020] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0021] Obtain the source stream to be encoded and determine the arrangement structure of the video frames in the source stream;

[0022] Based on the arrangement structure, multiple video frames in the source stream are extracted and processed to obtain multiple candidate streams with different stream structures.

[0023] Identify reference video frames in each of the candidate bitstreams, and determine the target video frame and regular video frame associated with each of the reference video frames;

[0024] Each of the reference video frames is used to encode the associated target video frame to obtain the initial encoding result of the candidate bitstream for each bitstream structure;

[0025] For each of the multiple candidate bitstreams with different bitstream structures, the target bitstream corresponding to the candidate bitstream structure is obtained based on the initial encoding result and regular video frames corresponding to the candidate bitstream.

[0026] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, performs the following steps:

[0027] Obtain the source stream to be encoded and determine the arrangement structure of the video frames in the source stream;

[0028] Based on the arrangement structure, multiple video frames in the source stream are extracted and processed to obtain multiple candidate streams with different stream structures.

[0029] Identify reference video frames in each of the candidate bitstreams, and determine the target video frame and regular video frame associated with each of the reference video frames;

[0030] Each of the reference video frames is used to encode the associated target video frame to obtain the initial encoding result of the candidate bitstream for each bitstream structure;

[0031] For each of the multiple candidate bitstreams with different bitstream structures, the target bitstream corresponding to the candidate bitstream structure is obtained based on the initial encoding result and regular video frames corresponding to the candidate bitstream.

[0032] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0033] Obtain the source stream to be encoded and determine the arrangement structure of the video frames in the source stream;

[0034] Based on the arrangement structure, multiple video frames in the source stream are extracted and processed to obtain multiple candidate streams with different stream structures.

[0035] Identify reference video frames in each of the candidate bitstreams, and determine the target video frame and regular video frame associated with each of the reference video frames;

[0036] Each of the reference video frames is used to encode the associated target video frame to obtain the initial encoding result of the candidate bitstream for each bitstream structure;

[0037] For each of the multiple candidate bitstreams with different bitstream structures, the target bitstream corresponding to the candidate bitstream structure is obtained based on the initial encoding result and regular video frames corresponding to the candidate bitstream.

[0038] The aforementioned video encoding method, apparatus, computer equipment, storage medium, and computer program product acquire a source bitstream to be encoded and determine the arrangement structure of video frames in the source bitstream. Based on this arrangement structure, multiple video frames in the source bitstream are extracted and processed to obtain multiple candidate bitstreams with different bitstream structures. By identifying reference video frames in each candidate bitstream and determining the associated target video frames and regular video frames for each reference video frame, the associated target video frames can be encoded according to each reference video frame, thereby obtaining the initial encoding result of the candidate bitstream for each bitstream structure. For each candidate bitstream with different bitstream structures, the target bitstream corresponding to the candidate bitstream structure can be obtained based on the initial encoding result and regular video frames. Because this application can directly obtain multiple candidate bitstreams with different bitstream structures based on the arrangement structure, compared to the traditional process of encoding the same video multiple times, this application can obtain the target bitstream corresponding to each candidate bitstream structure, ensuring the flexibility of generating target bitstreams adapted to different network bandwidth scenarios. At the same time, since the target video frame is encoded directly based on the reference video frame, the accuracy of the initial encoding result of the candidate bitstream is improved. Thus, when the target bitstream corresponding to the candidate bitstream is obtained through the initial encoding result and regular video frames, the efficiency of video encoding of the source bitstream can be improved. Attached Figure Description

[0039] Figure 1 This is a diagram illustrating the application environment of a video encoding method in one embodiment;

[0040] Figure 2 This is a flowchart illustrating a video encoding method in one embodiment;

[0041] Figure 3 This is a flowchart illustrating the process of determining the arrangement structure of the source code stream in one embodiment;

[0042] Figure 4 This is a schematic diagram of the arrangement structure in one embodiment;

[0043] Figure 5 This is a schematic diagram of the structure of the first bitstream in one embodiment of the full bitstream structure;

[0044] Figure 6 This is a schematic diagram of the structure of the second bitstream in one embodiment of the half-bitstream structure;

[0045] Figure 7 This is a structural block diagram of a video encoding device in one embodiment;

[0046] Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0048] The video encoding method provided in this application can be applied to, for example... Figure 1The application environment shown includes a first terminal 102, a server 104, and a second terminal 106. The first terminal 102 communicates with the server 104 via a network, and the second terminal 106 also communicates with the server 104 via a network. The first terminal 102 sends the source code stream to be encoded to the server 104. Server 104 is used to determine the arrangement structure of video frames in the source stream, and extract and process multiple video frames in the source stream according to the arrangement structure to obtain multiple candidate bitstreams with different bitstream structures. Server 104 is also used to identify reference video frames in each candidate bitstream and determine the target video frame and regular video frame associated with each reference video frame. Based on each reference video frame, the associated target video frame is encoded to obtain the initial encoding result of the candidate bitstream for each bitstream structure. For each candidate bitstream with different bitstream structures, based on the initial encoding result and regular video frame corresponding to the candidate bitstream, the target bitstream corresponding to the candidate bitstream structure is obtained. Server 104 sends the target bitstream to the second terminal 106 so that the second terminal 106 can obtain the target video by parsing the target bitstream. The first terminal 102 and the second terminal 106 can be, but are not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. Server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0049] In one embodiment, such as Figure 2 As shown, a video encoding method is provided, which is applied to... Figure 1 Taking the server in the example, the following steps are included:

[0050] Step 202: Obtain the source stream to be encoded and determine the arrangement structure of the video frames in the source stream.

[0051] The source stream to be encoded can be the video stream corresponding to the video sent by the first terminal; the arrangement structure can be a GOP (Group of Pictures), used to define the arrangement of each video frame between two first and second video frames. The first video frame can be an I-frame or an IDR frame, and the video frames between two first and second video frames can be P-frames. An I-frame is a data frame obtained after complete image compression, and it occupies a relatively large amount of data information. It performs JPEG compression encoding and transmission of the entire frame image information. During decoding, only the data of the I-frame is used to reconstruct the complete image. An IDR frame is the first I-frame among multiple I-frames. A P-frame is an inter-frame predictive coding frame, which occupies a relatively small amount of data information. It needs to refer to the preceding I-frames or P-frames for encoding, and the P-frame represents the difference between the current frame and the previous frame. During decoding, the difference represented by the current frame needs to be superimposed on the image constructed by the previous frame to generate the final image.

[0052] In one embodiment, such as Figure 3 As shown, determining the arrangement structure of video frames in the source stream includes the following steps:

[0053] Step 302: Perform field encoding on the source stream to obtain multiple video frames arranged in sequence.

[0054] The video frame number is n, where n is a positive integer. Each video frame is presented in a Slice format, which is a region format in the H.264 video coding standard. It consists of a header and data, with the header information being a frame_number field.

[0055] Specifically, the server performs field encoding on the source stream to obtain multiple sequentially arranged video frames, causing the value of the frame_number field for each video frame to increment frame by frame in the order of arrangement. Since the source stream may contain a large number of video frames, the value of the frame_number field usually increments to a certain value and then cycles back to 0 to increment frame by frame again.

[0056] In one embodiment, when a region format is expanded bitwise and the bitstream frame_num field is represented using 4 bytes, the value of the frame_num field ranges from 0 to 15.

[0057] Step 304: Identify the video frame with a special frame number from multiple video frames.

[0058] Among them, special frame numbers include at least n+2, where n is a positive integer. Video frames with special frame numbers can be represented as a long term reference frame (LTR).

[0059] In one embodiment, the special frame number can be obtained by the user in advance. For example, frame number n+3 or n+4 can also be set as the special frame number.

[0060] Step 306: Obtain the arrangement structure of video frames in the source stream based on the video frames with special frame numbers.

[0061] In one embodiment, obtaining the arrangement structure of video frames in the source stream based on video frames with special frame numbers includes: obtaining video frames with special identifiers and reference relationships between each video frame with a special identifier in response to an association operation on video frames with special frame numbers; obtaining video frames with regular identifiers in response to a configuration operation on video frames with other frame numbers among multiple video frames; and obtaining the arrangement structure of video frames in the source stream by combining the special identifiers and regular identifiers of the video frames and each reference relationship.

[0062] Among them, the association of reference relationships represents that by performing inter-frame prediction on video frames with special identifiers of earlier frame numbers, video frames with special identifiers of later frame numbers can be obtained.

[0063] Specifically, the server sets video frames with special frame numbers as video frames with special identifiers; for example, the special identifier is set to 1, which also means the long-term reference frame is set to 1. In response to the user's association operation with video frames with special frame numbers, the server obtains the reference relationships between video frames with each special identifier, and sets video frames with associated reference relationships to 'a', while those without are set to '0'. The server configures video frames with other frame numbers among the multiple video frames, obtaining video frames with regular identifiers; for example, the regular identifier is set to 0. Combining the special and regular identifiers of the video frames, as well as each reference relationship, a sequential arrangement of multiple video frames is obtained.

[0064] like Figure 4 As shown, Figure 4 This is a structural diagram of the arrangement. For example, the special identifiers of frames I0, P2, and P4 are set to 1; the regular identifiers of frames P1, P3, and P5 are set to 0; when I0 and P2 are associated, the reference relationship between I0 and P2 is obtained; when P2 and P4 are associated, the reference relationship between P2 and P4 is obtained, etc. At this time, P2 and P4 are set to 'a', and the video frames with associated reference relationships are represented by solid arrows.

[0065] In one embodiment, the reference relationships corresponding to each specially identified video frame can be cached in a queue sequentially according to the frame number of each specially identified video frame.

[0066] In one embodiment, a video frame with a regular identifier can represent a short-term reference frame (str). When the regular identifier is set to 0, the short-term reference frame is also set to 0. The short-term reference frame with an earlier frame number is typically used for inter-frame prediction of the video frame with a later frame number; that is, there is a correlation between the short-term reference frame and the video frame with the later frame number. Figure 4 As shown, the relationships can be represented by dashed arrows.

[0067] In this embodiment, by pre-determining long-term reference frames with special frame numbers and short-term reference frames with regular identifiers, the reference relationships between video frames with different special identifiers can be associated, thereby accurately obtaining the arrangement structure of video frames in the source stream. Therefore, the candidate streams of various stream structures can be flexibly determined through the arrangement structure.

[0068] In one embodiment, the server can achieve time-scalable encoding through long-term reference frame technology. That is, by controlling the arrangement structure of the source code stream through long-term reference frame technology, multiple target code streams can be split out with a single encoding.

[0069] Step 204: Based on the arrangement structure, extract and process multiple video frames in the source stream to obtain multiple candidate streams with different stream structures.

[0070] The arrangement structure includes special identifiers and regular identifiers; the candidate bitstream includes a first bitstream with a full bitstream structure and a second bitstream with a half bitstream structure.

[0071] In one embodiment, based on the arrangement structure, multiple video frames in the source stream are extracted and processed to obtain multiple candidate bitstreams with different bitstream structures, including: extracting the video frames corresponding to special identifiers and regular identifiers from the source stream to obtain a first bitstream with a full bitstream structure; deleting the video frames corresponding to regular identifiers in the first bitstream, and saving the frame number and regular identifier of the video frames corresponding to regular identifiers to obtain a second bitstream with a half bitstream structure.

[0072] Specifically, the server extracts multiple sequentially arranged video frames from the source stream based on the order of the video frames corresponding to special and regular identifiers, respectively, to obtain the first bitstream with a full bitstream structure, such as... Figure 5 As shown, Figure 5 This is a schematic diagram of the first bitstream in a full-stream structure. The server retains the video frames corresponding to special identifiers in the first bitstream, deletes the video frames corresponding to regular identifiers, and saves the frame number and regular identifier of the video frames corresponding to the regular identifiers, thus obtaining the second bitstream in a half-stream structure. For example... Figure 6 As shown, Figure 6 This is a schematic diagram of the structure of the first bitstream in the half-bitstream structure.

[0073] In one embodiment, the server can determine the bitstream structure of the candidate bitstream based on a special frame number. When the special frame number is n+2, where n is a positive integer, the determined bitstream structure includes a full bitstream structure and a half bitstream structure; when the special frame number is n+3, where n is a positive integer, the determined bitstream structure includes a full bitstream structure and a one-third bitstream structure; when the special frame number is n+4, where n is a positive integer, the determined bitstream structure includes a full bitstream structure and a one-quarter bitstream structure, and so on.

[0074] Step 206: Identify the reference video frames in each candidate bitstream and determine the target video frame and regular video frame associated with each reference video frame.

[0075] Specifically, the server iterates through each video frame in the candidate bitstream for each bitstream structure and determines the identifier type corresponding to the currently iterated video frame. Based on the identifier type, the server determines the type of the current video frame. When the identifier type is a special identifier, the current video frame is determined as a reference video frame; when the identifier type is a regular identifier, the current video frame is determined as a regular video frame.

[0076] In one embodiment, determining the target video frame and regular video frame associated with each reference video frame includes: for each reference video frame in the candidate bitstream, obtaining the target reference relationship corresponding to the current reference video frame; obtaining the current target video frame associated with the current reference video frame based on the target reference relationship; and treating each video frame between the current reference video frame and the current target video frame as a regular video frame associated with the current reference video frame.

[0077] Specifically, the server sequentially determines the current reference video frame according to its frame number and retrieves the target reference relationship corresponding to the current reference video frame from a pre-cached queue of reference relationships. Based on the target reference relationship and the current reference video frame, the server determines the current target video frame and treats every video frame between the current reference video frame and the current target video frame as a regular video frame associated with the current reference video frame. Figure 5 As shown, if the current reference video frame is frame I0, according to the reference relationship represented by the arrow, frame P2 is determined to be the current target video frame associated with frame I0, and frame P1 is the current reference video frame associated with frame I0, which is a regular video frame.

[0078] In one embodiment, a video frame with a regular identifier between the current reference video frame and the current target video frame is considered as a regular video frame associated with the current reference video frame.

[0079] Step 208: Encode the associated target video frame according to each reference video frame to obtain the initial encoding result of the candidate bitstream for each bitstream structure.

[0080] In one embodiment, each reference video frame is used to encode the associated target video frame to obtain the initial encoding result of the candidate bitstream for each bitstream structure. This includes: determining the current reference video frame in the candidate bitstream and determining the current target video frame associated with the current reference video frame; performing inter-frame prediction on the current reference video frame to obtain the current prediction result; superimposing the current prediction result with the current target video frame to obtain the first encoding result corresponding to the current target video frame; using the first encoding result as the new current reference video frame and returning to the step of determining the current target video frame associated with the current reference video frame to continue execution until the first encoding result of the target video frame associated with the last reference video frame in the candidate bitstream is obtained; and using the first encoding result of the target video frame associated with the last reference video frame as the initial encoding result of the candidate bitstream.

[0081] Inter-frame prediction is a compression method based on temporal redundancy. It achieves image compression by utilizing the correlation between video frames, thereby improving the video coding compression rate.

[0082] Specifically, the server determines the current reference video frame sequentially according to frame number, and determines the current target video frame associated with the current reference video frame based on the target reference relationship corresponding to the current reference video frame. The server performs inter-frame prediction on the current reference video frame to obtain the current prediction result, and then superimposes the current prediction result with the current target video frame. The current target video frame is usually a P-frame, which uses motion compensation to transmit the difference and motion vector between itself and the preceding video frames. The motion vector is also called the prediction error. After superimposing the current prediction result with the prediction error, the server can reconstruct the first encoded result corresponding to the current target video frame. At this point, the first encoded result is a complete image.

[0083] The process proceeds to the next iteration, using the first encoding result as the new current reference video frame, and returning to the step of determining the current target video frame associated with the current reference video frame. This continues until the first encoding result of the target video frame associated with the last reference video frame in the candidate bitstream is obtained. Since the process is the same for each iteration, it will not be described in detail here. The server uses the first encoding result of the target video frame associated with the last reference video frame as the initial encoding result of the candidate bitstream. At this point, the initial encoding result is a complete image that has iterated over the first encoding results corresponding to each of the target video frames.

[0084] In one embodiment, if the current reference video frame is the first video frame, that is... Figure 4 When the I0 frame is in the current reference video frame, the complete image can be reconstructed directly.

[0085] In one embodiment, the server performs inter-frame prediction, transform quantization, loop filtering, and entropy coding on the current reference video frame according to the video coding standard, which includes, but is not limited to, the more general and widely used H.264.

[0086] Step 210: For each candidate bitstream among multiple candidate bitstreams with different bitstream structures, obtain the target bitstream corresponding to the candidate bitstream structure based on the initial encoding result and regular video frames corresponding to the candidate bitstream.

[0087] Specifically, when the candidate bitstream is a full bitstream structure, the server determines the association relationship between the regular video frames and, based on the association relationship, determines the reference video frame corresponding to the regular video frame. Then, based on the reference video frame and the regular video frame, the second encoding result corresponding to the regular video frame is obtained. At this time, the second encoding result is a complete image. When the candidate bitstream is a half bitstream structure, the second encoding result corresponding to the regular video frame is determined. Finally, the initial encoding result and the second encoding result corresponding to each regular video frame are combined to obtain the target bitstream corresponding to the candidate bitstream of the corresponding bitstream structure.

[0088] In one embodiment, the server can be a configurable video encoder. The video encoder can be implemented as a software encoder or a hardware encoder. By employing a video encoding standard, the video encoder can sequentially encode each video frame in a composite source stream. When based on the H.264 video encoding standard, the hardware encoder offers greater compatibility; hierarchical encoding can be implemented on the hardware encoder simply by determining the arrangement structure of the source stream.

[0089] The aforementioned video encoding method obtains the source bitstream to be encoded and determines the arrangement structure of the video frames in the source bitstream. Based on this arrangement structure, it extracts and processes multiple video frames from the source bitstream to obtain multiple candidate bitstreams with different bitstream structures. By identifying reference video frames in each candidate bitstream and determining the associated target video frames and regular video frames for each reference video frame, the associated target video frames can be encoded separately based on each reference video frame, thus obtaining the initial encoding result for each candidate bitstream structure. For each candidate bitstream with different bitstream structures, the target bitstream corresponding to the candidate bitstream structure can be obtained based on the initial encoding result and regular video frames. Since this application directly obtains multiple candidate bitstreams with different bitstream structures based on the arrangement structure, compared to the traditional process of encoding the same video multiple times, this application can obtain the target bitstream corresponding to each candidate bitstream structure, ensuring the flexibility of generating target bitstreams adaptable to different network bandwidth scenarios.

[0090] In one embodiment, for each video frame in the source stream, each video frame is encoded sequentially according to its frame number. If the current video frame is a target video frame, it is encoded based on the long-term reference frame corresponding to the target video frame to obtain a first encoding result, wherein the long-term reference frame is obtained through the reference relationship corresponding to the target video frame. If the current video frame is a regular video frame, it is encoded based on the short-term reference frame corresponding to the regular video frame to obtain a second encoding result, wherein the short-term reference frame is obtained through the association relationship corresponding to the regular video frame. The first encoding result corresponding to the first video frame, each target video frame, and each regular video frame corresponding to the second encoding result are combined until the target stream with the full bitstream structure is obtained.

[0091] In one embodiment, if the current video frame is a regular video frame, then all regular video frames are encoded and padded to obtain the second encoding result corresponding to the regular video frame. The first encoding result corresponding to each target video frame, and the second encoding result corresponding to each regular video frame are combined until the target bitstream with a half-bitstream structure is obtained. Since each video frame is encoded sequentially, no time delay is caused in the video encoding.

[0092] In one embodiment, obtaining the target bitstream corresponding to the candidate bitstream of the corresponding bitstream structure based on the initial encoding result corresponding to the candidate bitstream and regular video frames includes: determining the current reference video frame associated with the regular video frames in the candidate bitstream of the full bitstream structure; if the current reference video frame is the first reference video frame, then superimposing each regular video frame in the candidate bitstream of the full bitstream structure with the current reference video frame to obtain a second encoding result corresponding to each regular video frame; if the current reference video frame is not the first reference video frame, then determining the first encoding result of the current reference video frame, and superimposing each regular video frame in the candidate bitstream of the full bitstream structure with the first encoding result to obtain a second encoding result corresponding to each regular video frame; and combining each second encoding result with the initial encoding result of the candidate bitstream of the full bitstream structure to obtain the target bitstream corresponding to the candidate bitstream of the full bitstream structure.

[0093] The bitstream structure includes the full bitstream structure.

[0094] Specifically, since the current reference video frame is the first reference video frame, and the first reference video frame is the first video frame in the source code stream, the complete image corresponding to the current reference video frame can be directly constructed. Therefore, referring to step S208, the specific real-time step of overlaying the current prediction result with the current target video frame, the server overlays each regular video frame on the basis of the current reference video frame to obtain the second encoding result corresponding to each regular video frame. Figure 5 As shown, when the current reference video frame is frame I0, after obtaining the prediction result of frame P1 through frame I0, the prediction result is superimposed with the prediction error in frame P1 to reconstruct the second encoding result corresponding to frame P1. At this time, the second encoding result is a complete image.

[0095] Similarly, when the current reference video frame is not the first reference video frame, it is necessary to determine the first encoding result corresponding to the current reference video frame. The first encoding result corresponding to the current reference video frame represents the first encoding result obtained by encoding the target video frame when the current reference video frame is regarded as the target video frame. For example, when the current reference video frame is frame P1, the first encoding result corresponding to frame P1 is the complete image corresponding to P1. Therefore, based on the first encoding result, the server superimposes each regular video frame to obtain the second encoding result corresponding to each regular video frame. The server arranges each second encoding result according to the frame number order and integrates the preliminary encoding results of the candidate bitstream of the full bitstream structure to obtain the target bitstream corresponding to the candidate bitstream of the full bitstream structure. At this time, the target bitstream can be regarded as a complete video composed of multiple video frames obtained after decoding.

[0096] In one embodiment, the server can be a video decoder. The process of combining each second encoding result and the preliminary encoding result of the candidate bitstream of the full bitstream structure to obtain the target bitstream corresponding to the candidate bitstream of the full bitstream structure can be a video decoding process.

[0097] In this embodiment, for the candidate bitstream of the full bitstream structure, by determining whether the current reference video frame is the first reference video frame, different methods are adopted to determine the second encoding result corresponding to each regular video frame. In this way, the target bitstream corresponding to the candidate bitstream of the full bitstream structure can be obtained efficiently and accurately.

[0098] In one embodiment, obtaining the target bitstream corresponding to the candidate bitstream of the corresponding bitstream structure based on the initial encoding result corresponding to the candidate bitstream and the regular video frames includes: encoding and padding each regular video frame in the candidate bitstream of the half-bitstream structure to obtain the second encoding result corresponding to each regular video frame; and combining each second encoding result with the initial encoding result of the candidate bitstream of the half-bitstream structure to obtain the target bitstream corresponding to the candidate bitstream of the half-bitstream structure.

[0099] The bitstream structure includes a half-bitstream structure.

[0100] Specifically, the server uses a special macroblock to encode and fill each regular video frame in the candidate bitstream of the half-stream structure, obtaining a second encoding result for each regular video frame. This second encoding result contains a smaller amount of data. A regular video frame comprises multiple macroblocks. SKIP MB is a special macroblock in the h264 video coding standard. Its motion vector is predicted from adjacent blocks, and its residual data is all 0. A macroblock in a regular video frame can be considered a copy of a macroblock at the corresponding position in its reference video frame. Therefore, special macroblocks require fewer encoding bits.

[0101] refer to Figure 6 As shown, frames P1, P3, and P5 are all artificially coded frames using special macroblocks with extremely small data volumes. When decoding the target bitstream, for each regular video frame's corresponding second encoding result, the first reference video frame preceding the current regular video frame's frame number sequence is determined, and this reference video frame is copied, i.e., the complete image corresponding to this reference video frame is used as the complete image of the current regular video frame. The server arranges the complete images corresponding to each regular video frame according to the frame number order, and combines the preliminary encoding results of the candidate bitstreams in the half-bitstream structure to obtain the target bitstream corresponding to the candidate bitstreams in the half-bitstream structure.

[0102] In this embodiment, for the candidate bitstream of the half-bitstream structure, special macroblocks are used to encode and fill each regular video frame in the candidate bitstream of the half-bitstream structure. Then, when performing video decoding, the complete image of the corresponding reference video frame can be directly copied, which can efficiently obtain the target bitstream corresponding to the candidate bitstream of the half-bitstream structure and reduce the load on the video decoder.

[0103] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0104] Based on the same inventive concept, this application also provides a video encoding apparatus for implementing the video encoding method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more video encoding apparatus embodiments provided below can be found in the limitations of the video encoding method described above, and will not be repeated here.

[0105] In one embodiment, such as Figure 7 As shown, a video encoding apparatus 700 is provided, including: a candidate bitstream determination module 702, a video frame recognition module 704, and a target bitstream determination module 706, wherein:

[0106] The candidate bitstream determination module 702 is used to acquire the source bitstream to be encoded and determine the arrangement structure of the video frames in the source bitstream; based on the arrangement structure, it extracts and processes multiple video frames in the source bitstream to obtain multiple candidate bitstreams with different bitstream structures.

[0107] The video frame recognition module 704 is used to identify reference video frames in each candidate bitstream and determine the target video frame and regular video frame associated with each reference video frame.

[0108] The target bitstream determination module 706 is used to encode the associated target video frame according to each reference video frame to obtain the initial encoding result of the candidate bitstream for each bitstream structure; for each candidate bitstream with different bitstream structures, the target bitstream corresponding to the candidate bitstream with the corresponding bitstream structure is obtained according to the initial encoding result of the candidate bitstream and the regular video frame.

[0109] In one embodiment, the candidate bitstream determination module 702 further includes an arrangement structure module 7021, which is used to perform field encoding on the source bitstream to obtain a plurality of video frames arranged in sequence; the frame number of the video frame is n, where n is a positive integer; determine the video frame with a special frame number from the plurality of video frames; the special frame number includes at least n+2; and obtain the arrangement structure of the video frames in the source bitstream based on the video frame with the special frame number.

[0110] In one embodiment, the arrangement structure module 7021 is further configured to, in response to the association operation of video frames with special frame numbers, obtain video frames with special identifiers and reference relationships between video frames with special identifiers; in response to the configuration operation of video frames with other frame numbers in a plurality of video frames, obtain video frames with regular identifiers; and combine the special identifiers and regular identifiers of the video frames, as well as each reference relationship, to obtain the arrangement structure of video frames in the source code stream.

[0111] In one embodiment, the candidate bitstream determination module 702 further includes an extraction processing module 7022, which extracts the video frames corresponding to the special identifier and the regular identifier from the source bitstream to obtain a first bitstream with a full bitstream structure; deletes the video frames corresponding to the regular identifier in the first bitstream, and saves the frame number and regular identifier of the video frames corresponding to the regular identifier to obtain a second bitstream with a half bitstream structure.

[0112] In one embodiment, the video frame recognition module 704 is configured to obtain the target reference relationship corresponding to the current reference video frame for each reference video frame in the candidate bitstream; obtain the current target video frame associated with the current reference video frame based on the target reference relationship; and treat each video frame between the current reference video frame and the current target video frame as a regular video frame associated with the current reference video frame.

[0113] In one embodiment, the target bitstream determination module 706 includes an initial encoding result module 7061, configured to determine the current reference video frame in the candidate bitstream and determine the current target video frame associated with the current reference video frame; perform inter-frame prediction on the current reference video frame to obtain a current prediction result; superimpose the current prediction result with the current target video frame to obtain a first encoding result corresponding to the current target video frame; use the first encoding result as the new current reference video frame and return to the step of determining the current target video frame associated with the current reference video frame to continue execution until the first encoding result of the target video frame associated with the last reference video frame in the candidate bitstream is obtained; and use the first encoding result of the target video frame associated with the last reference video frame as the initial encoding result of the candidate bitstream.

[0114] In one embodiment, the target bitstream determination module 706 includes a video frame overlay module 7062, used to determine the current reference video frame associated with the regular video frames in the candidate bitstream of the full bitstream structure. If the current reference video frame is the first reference video frame, then each regular video frame in the candidate bitstream of the full bitstream structure is overlaid with the current reference video frame to obtain a second encoding result corresponding to each regular video frame. If the current reference video frame is not the first reference video frame, then the first encoding result of the current reference video frame is determined, and each regular video frame in the candidate bitstream of the full bitstream structure is overlaid with the first encoding result to obtain a second encoding result corresponding to each regular video frame. By combining each second encoding result and the preliminary encoding result of the candidate bitstream of the full bitstream structure, the target bitstream corresponding to the candidate bitstream of the full bitstream structure is obtained.

[0115] In one embodiment, the target bitstream determination module 706 includes an encoding padding module 7063, which performs encoding padding on each regular video frame in the candidate bitstream of the half-bitstream structure to obtain a second encoding result corresponding to each regular video frame; and combines each second encoding result with the preliminary encoding result of the candidate bitstream of the half-bitstream structure to obtain the target bitstream corresponding to the candidate bitstream of the half-bitstream structure.

[0116] Each module in the aforementioned video encoding device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware within or independently of the processor in a computer device, or stored in software within the memory of the computer device, so that the processor can call and execute the operations corresponding to each module.

[0117] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 8 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores video encoding data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a video encoding method.

[0118] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0119] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0120] In one embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0121] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and executes the computer instructions, causing the computer device to perform the steps in the above method embodiments.

[0122] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0123] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0124] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A video encoding method, characterized in that, The method includes: The process involves: acquiring a source stream to be encoded; performing field encoding on the source stream to obtain multiple sequentially arranged video frames; identifying video frames with special frame numbers from the multiple video frames; obtaining video frames with special identifiers and reference relationships between each video frame with a special identifier in response to an association operation on the video frames with special frame numbers; obtaining video frames with regular identifiers in response to a configuration operation on video frames with other frame numbers in the multiple video frames; combining the special identifiers and regular identifiers of the video frames, as well as each of the reference relationships, to obtain the arrangement structure of the video frames in the source stream; the arrangement structure includes special identifiers and regular identifiers; extracting the video frames corresponding to the special identifiers and regular identifiers from the source stream to obtain a first stream with a full stream structure; deleting the video frames corresponding to the regular identifiers in the first stream, and saving the frame number and regular identifier of the video frames corresponding to the regular identifiers to obtain a second stream with a half stream structure. Identify reference video frames in each candidate bitstream and determine the target video frame and regular video frame associated with each reference video frame; the candidate bitstream includes the first bitstream and the second bitstream; Each of the reference video frames is used to encode the associated target video frame to obtain the initial encoding result of the candidate bitstream for each bitstream structure; For each candidate bitstream among multiple candidate bitstreams with different bitstream structures, the target bitstream corresponding to the candidate bitstream structure is obtained based on the initial encoding result and regular video frames corresponding to the candidate bitstream.

2. The method according to claim 1, characterized in that, The frame number of the video frame is n, where n is a positive integer; the special frame number includes at least n+2.

3. The method according to claim 1, characterized in that, The step of determining the target video frame and regular video frame associated with each of the reference video frames includes: For each reference video frame in the candidate bitstream, the target reference relationship corresponding to the current reference video frame is obtained; Based on the target reference relationship, the current target video frame associated with the current reference video frame is obtained; Each video frame between the current reference video frame and the current target video frame is treated as a regular video frame associated with the current reference video frame.

4. The method according to claim 1, characterized in that, The step of encoding the associated target video frame according to each of the reference video frames to obtain the initial encoding result of the candidate bitstream for each bitstream structure includes: Determine the current reference video frame in the candidate bitstream, and determine the current target video frame associated with the current reference video frame; Perform inter-frame prediction on the current reference video frame to obtain the current prediction result; The current prediction result is superimposed on the current target video frame to obtain the first encoding result corresponding to the current target video frame; The first encoding result is used as the new current reference video frame, and the process returns to the step of determining the current target video frame associated with the current reference video frame to continue until the first encoding result of the target video frame associated with the last reference video frame in the candidate bitstream is obtained. The first encoding result of the target video frame associated with the last reference video frame is used as the initial encoding result of the candidate bitstream.

5. The method according to claim 1, characterized in that, The bitstream structure includes a full bitstream structure; obtaining the target bitstream corresponding to the candidate bitstream of the corresponding bitstream structure based on the initial encoding result corresponding to the candidate bitstream and regular video frames includes: Determine the current reference video frame associated with the regular video frame in the candidate bitstream of the full bitstream structure. If the current reference video frame is the first reference video frame, then superimpose each regular video frame in the candidate bitstream of the full bitstream structure with the current reference video frame to obtain the second encoding result corresponding to each regular video frame. If the current reference video frame is not the first reference video frame, then the first encoding result of the current reference video frame is determined, and each regular video frame in the candidate bitstream of the full bitstream structure is superimposed with the first encoding result to obtain the second encoding result corresponding to each regular video frame. By combining each of the second encoding results and the preliminary encoding results of the candidate bitstream of the full bitstream structure, the target bitstream corresponding to the candidate bitstream of the full bitstream structure is obtained.

6. The method according to claim 1, characterized in that, The bitstream structure includes a half-bitstream structure; obtaining the target bitstream corresponding to the candidate bitstream structure based on the initial encoding result corresponding to the candidate bitstream and regular video frames includes: Each regular video frame in the candidate bitstream of the half-bitstream structure is encoded and padded to obtain the second encoding result corresponding to each regular video frame. By combining each of the second encoding results and the preliminary encoding results of the candidate bitstream of the half-bitstream structure, the target bitstream corresponding to the candidate bitstream of the half-bitstream structure is obtained.

7. A video encoding device, characterized in that, The device includes: A candidate bitstream determination module is used to acquire a source bitstream to be encoded, perform field encoding on the source bitstream to obtain multiple video frames arranged in sequence; determine video frames with special frame numbers from the multiple video frames; in response to the association operation of the video frames with special frame numbers, obtain video frames with special identifiers and reference relationships between each video frame with a special identifier; in response to the configuration operation of video frames with other frame numbers in the multiple video frames, obtain video frames with regular identifiers; combine the special identifiers and regular identifiers of the video frames, as well as each of the reference relationships, to obtain the arrangement structure of the video frames in the source bitstream; the arrangement structure includes special identifiers and regular identifiers; extract the video frames corresponding to the special identifiers and the regular identifiers respectively from the source bitstream to obtain a first bitstream with a full bitstream structure; delete the video frames corresponding to the regular identifiers in the first bitstream, and save the frame number and regular identifier of the video frames corresponding to the regular identifiers to obtain a second bitstream with a half bitstream structure; A video frame recognition module is used to identify reference video frames in each candidate bitstream and determine the target video frame and regular video frame associated with each reference video frame; the candidate bitstream includes the first bitstream and the second bitstream; The target bitstream determination module is used to encode the associated target video frame according to each of the reference video frames to obtain the initial encoding result of the candidate bitstream for each bitstream structure; for each candidate bitstream among multiple candidate bitstreams with different bitstream structures, the target bitstream corresponding to the candidate bitstream for the corresponding bitstream structure is obtained according to the initial encoding result of the candidate bitstream and the regular video frame.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images