A video encoding method, device and electronic equipment

Abstract

The application provides a video coding method, device and electronic equipment. The method comprises: obtaining a frame sequence and a buffer module. A key frame code stream and a key reconstructed frame are generated according to a key frame; the key reconstructed frame is stored in a first buffer module, and a to-be-referenced identifier is set for the first buffer module; a forward prediction reconstructed frame is generated based on the key reconstructed frame, and the forward prediction reconstructed frame is stored in a second buffer module; the to-be-referenced identifier is set for the second buffer module, and a writable identifier is set for the first buffer module; a traversal is performed on the frame sequence to obtain a current frame, a reference reconstructed frame stored in a buffer module with the to-be-referenced identifier is obtained according to a type of the current frame, and a reconstructed frame corresponding to the current frame is generated; the reconstructed frame is stored in the buffer module with the writable identifier, the buffer module in which the reference reconstructed frame is stored is released, and a state identifier of the buffer module is updated. The memory of the first buffer module and the second buffer module is released in time, and the memory occupation time is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of video compression coding, in particular to a video coding method, device and electronic equipment. BACKGROUND

[0002] In an intelligent security monitoring system, video coding compression is a key link to ensure smooth transmission and storage of monitoring pictures. Based on the H.264 / H.265 compression protocol, the video is coded, which can reduce the transmission data volume of the video. In the process of video coding, first, an image with high quality and high stability is selected as a long-term reference frame, and the pixel values of the long-term reference frame are used to predict the coding of the next frame of image, and then the time redundancy is removed according to the pixel similarity between the continuous frames, and then the video coding is completed.

[0003] In the above process, the long-term reference frame needs to continuously occupy the system memory, so the system memory overhead is too large in the process of video coding, the memory occupation time is too long, and the performance of video coding is low. SUMMARY

[0004] The present application provides a video coding method, device and electronic equipment, which releases the memory of the cache module in time, thereby reducing the memory occupation time and improving the performance of video coding.

[0005] In a first aspect, the present application provides a video coding method, comprising:

[0006] obtaining a frame sequence, the frame sequence comprising a key frame, at least one forward prediction frame and at least one virtual frame, the first frame of the frame sequence being the key frame, and the second frame of the frame sequence being the forward prediction frame;

[0007] obtaining a cache module, the cache module being used to store a reference reconstructed frame, the reference reconstructed frame comprising a key reconstructed frame, a forward prediction reconstructed frame and a virtual reconstructed frame, the cache module comprising a first cache module and a second cache module, the cache module being provided with a state identifier, the state identifier comprising a to-be-referenced identifier and a writable identifier;

[0008] generating a key frame code stream according to the key frame, and decoding the key frame code stream to generate a key reconstructed frame;

[0009] storing the key reconstructed frame into the first cache module, and generating a forward prediction reconstructed frame based on the key reconstructed frame in the first cache module;

[0010] storing the forward prediction reconstructed frame into the second cache module, setting the state identifier of the second cache module to the to-be-referenced identifier, and setting the state identifier of the first cache module to the writable identifier;

[0011] performing a traversal on the frame sequence starting from a third frame of the frame sequence to obtain a current frame;

[0012] if the current frame is a forward predicted frame and a next frame of the current frame is a forward predicted frame, obtaining a reference reconstructed frame stored in a cache module with a state identifier as a to-be-referenced identifier;

[0013] generating a forward predicted reconstructed frame corresponding to the current frame according to the reference reconstructed frame, and storing the forward predicted reconstructed frame in a cache module with a state identifier as a writable identifier;

[0014] setting the state identifier of the cache module where the reference reconstructed frame is stored as the writable identifier, and setting the state identifier of the cache module where the forward predicted reconstructed frame corresponding to the current frame is stored as the to-be-referenced identifier.

[0015] In some embodiments, the video encoding method further comprises: if the current frame is a forward predicted frame and a next frame of the current frame is a virtual frame, or the current frame is a forward predicted frame and the current frame is a last frame of the frame sequence, obtaining a reference reconstructed frame stored in a cache module with a state identifier as a to-be-referenced identifier;

[0016] generating a forward predicted reconstructed frame corresponding to the current frame according to the reference reconstructed frame;

[0017] setting the state identifier of the cache module where the reference reconstructed frame is stored as the writable identifier.

[0018] In some embodiments, the video encoding method further comprises: if the current frame is a virtual frame, obtaining the key frame code stream, and decoding the key frame code stream again to generate the key reconstructed frame;

[0019] storing the key reconstructed frame in a cache module with a state identifier as a writable identifier, and setting the state identifier of the cache module where the key reconstructed frame is stored as the to-be-referenced identifier;

[0020] generating a virtual reconstructed frame corresponding to the current frame according to the key reconstructed frame, and storing the virtual reconstructed frame in a cache module with a state identifier as a writable identifier;

[0021] setting the state identifier of the cache module where the virtual reconstructed frame is stored as the to-be-referenced identifier, and setting the state identifier of the cache module where the key reconstructed frame is stored as the writable identifier.

[0022] In some embodiments, generating a key frame code stream according to a key frame comprises: dividing the key frame based on a preset size to obtain a key frame macroblock, the key frame macroblock comprising real pixel values;

[0023] predicting, by using an intra prediction algorithm, a predicted pixel value of the macroblock of the key frame, and calculating an intra residual between the predicted pixel value and a real pixel value;

[0024] performing a frequency domain transform on the intra residual to calculate an intra transform coefficient, and performing quantization on the intra transform coefficient to calculate an intra quantization coefficient;

[0025] performing entropy coding on the intra quantization coefficient to generate a key frame bitstream.

[0026] In some embodiments, generating a forward prediction reconstructed frame corresponding to the current frame according to the reference reconstructed frame comprises: partitioning the forward prediction frame based on a preset size to obtain a forward prediction frame macroblock;

[0027] querying a forward matching macroblock of the forward prediction frame macroblock from the reference reconstructed frame, and calculating a forward motion vector between the forward matching macroblock and the forward prediction frame macroblock;

[0028] generating a forward matching prediction macroblock according to the forward matching macroblock and the forward motion vector, and calculating a forward inter residual between the forward prediction frame macroblock and the forward matching prediction macroblock;

[0029] performing a frequency domain transform on the forward inter residual to calculate a forward inter transform coefficient, and performing quantization on the forward inter transform coefficient to calculate a forward inter quantization coefficient;

[0030] performing entropy coding on the forward inter quantization coefficient to generate a forward prediction frame bitstream;

[0031] decoding the forward prediction frame bitstream to generate the forward prediction reconstructed frame.

[0032] In some embodiments, generating a virtual reconstructed frame corresponding to the current frame according to the key reconstructed frame comprises: partitioning the virtual frame based on a preset size to obtain a virtual frame macroblock;

[0033] querying a virtual matching macroblock of the virtual frame macroblock from the key reconstructed frame, and calculating a virtual motion vector between the virtual matching macroblock and the virtual frame macroblock;

[0034] generating a virtual frame prediction macroblock according to the virtual matching macroblock and the virtual motion vector, and calculating a virtual inter residual between the virtual frame macroblock and the virtual frame prediction macroblock;

[0035] performing a frequency domain transform on the virtual inter residual to calculate a virtual inter transform coefficient, and performing quantization on the virtual inter transform coefficient to calculate a virtual inter quantization coefficient;

[0036] performing entropy coding on the virtual inter-frame quantization coefficients to generate a virtual frame code stream;

[0037] decoding the virtual frame code stream to generate a virtual reconstructed frame.

[0038] In some embodiments, the video coding method further comprises: obtaining a coding code stream, the coding code stream comprising a key frame code stream, a forward prediction frame code stream and a virtual frame code stream;

[0039] generating a coding sequence based on the arrangement order of the frame sequence, the coding sequence comprising the coding code stream;

[0040] decoding the coding code stream to generate the reference reconstructed frame;

[0041] performing loop filtering on the reference reconstructed frame to optimize the reference reconstructed frame.

[0042] In a second aspect, the embodiments of the present application provide a video coding device, comprising: an image preprocessing module configured to obtain a frame sequence, the frame sequence comprising a key frame, at least one forward prediction frame and at least one virtual frame, a first frame of the frame sequence being the key frame, and a second frame of the frame sequence being the forward prediction frame;

[0043] a cache obtaining module configured to obtain a cache module, the cache module being used to store a reference reconstructed frame, the reference reconstructed frame comprising a key reconstructed frame, a forward prediction reconstructed frame and a virtual reconstructed frame, the cache module comprising a first cache module and a second cache module, the cache module being provided with a state identifier, the state identifier comprising a to-be-referenced identifier and a writable identifier;

[0044] a video coding processing module configured to generate a key frame code stream according to the key frame, and decode the key frame code stream to generate a key reconstructed frame, store the key reconstructed frame into the first cache module and generate a forward prediction reconstructed frame based on the key reconstructed frame in the first cache module, store the forward prediction reconstructed frame into the second cache module, set the state identifier of the second cache module as the to-be-referenced identifier, and set the state identifier of the first cache module as the writable identifier, and perform traversal on the frame sequence from a third frame of the frame sequence to obtain a current frame;

[0045] a processing core selection module configured to send a scheduling instruction to the video coding processing module based on the type of the current frame, the scheduling instruction being used to instruct the video coding processing module to generate a reference reconstructed frame and store the reference reconstructed frame;

[0046] The encoder post-processing module is configured to: acquire an encoded code stream, the encoded code stream comprising a key frame code stream, a forward prediction frame code stream and a virtual frame code stream; generate an encoded sequence based on the arrangement order of the frame sequence, the encoded sequence comprising the encoded code stream; decode the encoded code stream to generate the reference reconstructed frame; and perform loop filtering on the reference reconstructed frame to optimize the reference reconstructed frame.

[0047] In some embodiments, the video encoding processing module is further configured to: if the current frame is a forward prediction frame and a next frame of the current frame is a forward prediction frame, acquire a reference reconstructed frame stored in a buffer module with a state identifier of to-be-referenced; generate a forward prediction reconstructed frame corresponding to the current frame according to the reference reconstructed frame, and store the forward prediction reconstructed frame in a buffer module with a state identifier of writable; set the state identifier of the buffer module where the reference reconstructed frame is stored as writable, and set the state identifier of the buffer module where the forward prediction reconstructed frame corresponding to the current frame is stored as to-be-referenced.

[0048] If the current frame is a forward prediction frame and a next frame of the current frame is a virtual frame, or the current frame is a forward prediction frame and the current frame is a last frame of the frame sequence, acquire a reference reconstructed frame stored in a buffer module with a state identifier of to-be-referenced; generate a forward prediction reconstructed frame corresponding to the current frame according to the reference reconstructed frame; and set the state identifier of the buffer module where the reference reconstructed frame is stored as writable.

[0049] If the current frame is a virtual frame, acquire the key frame code stream, and decode the key frame code stream again to generate the key reconstructed frame; store the key reconstructed frame in a buffer module with a state identifier of writable, and set the state identifier of the buffer module where the key reconstructed frame is stored as to-be-referenced; generate a virtual reconstructed frame corresponding to the current frame according to the key reconstructed frame, and store the virtual reconstructed frame in a buffer module with a state identifier of writable; set the state identifier of the buffer module where the virtual reconstructed frame is stored as to-be-referenced, and set the state identifier of the buffer module where the key reconstructed frame is stored as writable.

[0050] In a third aspect, an electronic device is provided, comprising a processor, a memory and a bus, the processor and the memory being communicatively connected through the bus; the memory stores computer program instructions executable by the processor, and the processor is configured to implement the video encoding method of the first aspect.

[0051] According to the technical solution, the application provides a video coding method, device and electronic equipment. The method comprises: obtaining a frame sequence and a buffer module. A key frame code stream and a key reconstructed frame are generated according to the key frame; the key reconstructed frame is stored in the first buffer module, and a to-be-referenced identifier is set for the first buffer module; a forward prediction reconstructed frame is generated based on the key reconstructed frame, and the forward prediction reconstructed frame is stored in the second buffer module; the to-be-referenced identifier is set for the second buffer module, and the writable identifier is set for the first buffer module; the frame sequence is traversed to obtain a current frame, a reference reconstructed frame stored in the buffer module with the to-be-referenced identifier is obtained according to the type of the current frame, and a reconstructed frame corresponding to the current frame is generated; the reconstructed frame is stored in the buffer module with the writable identifier, the buffer module in which the reference reconstructed frame is stored is released, and the state identifier of the buffer module is updated. The memory of the first buffer module and the second buffer module is released in time, and the memory occupation time is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0052] In order to more clearly illustrate the technical solutions of the application, the drawings needed in the embodiments will be briefly introduced below. Obviously, other drawings can also be obtained by those skilled in the art without creative labor.

[0053] Figure 1 Frame sequence diagram of the embodiment of the application;

[0054] Figure 2 Video coding diagram based on three buffer modules of the embodiment of the application;

[0055] Figure 3 Step diagram of the video coding method of the embodiment of the application;

[0056] Figure 4 Step diagram of generating a key frame code stream of the embodiment of the application;

[0057] Figure 5 Video coding diagram based on two buffer modules of the embodiment of the application;

[0058] Figure 6 Step diagram of generating a reference reconstructed frame of the embodiment of the application;

[0059] Figure 7 Structure diagram of the video coding device of the embodiment of the application;

[0060] Figure 8 Structure diagram of the electronic equipment of the embodiment of the application. DETAILED DESCRIPTION

[0061] In order to make the person skilled in the art better understand the scheme of the present application, the technical scheme in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by the person skilled in the art without creative labor should belong to the scope of protection of the present application.

[0062] It should be understood that "multiple" referred to herein means two or more. In the description of the embodiments of the present application, unless otherwise specified, " / " represents the meaning of or, for example, A / B can represent A or B; "and / or" herein is only a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist together, and B exists alone. In addition, in order to clearly describe the technical scheme of the embodiments of the present application, in the embodiments of the present application, "first", "second" and the like are used to distinguish the same items or similar items with basically the same function and effect. The person skilled in the art can understand that "first", "second" and the like do not limit the quantity and execution order, and "first", "second" and the like do not necessarily mean different.

[0063] In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device containing a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0064] In network transmission, the size of the video file affects the transmission rate and bandwidth consumption. Since the video file contains a large amount of data, it will cause slow transmission speed and increase network delay, so it is necessary to encode and compress the video file to reduce the occupied space of the video file, thereby speeding up the transmission speed and reducing the congestion of the network.

[0065] In the process of video encoding, the video is decomposed into continuous images (i.e. video frames), and a set of continuous video frames can be composed into a group of pictures (GOP). The group of pictures is the basic unit of video encoding and video transmission, and the group of pictures contains various types of video frames, such as Figure 1 As shown in the figure, the video frames are divided into key frames (IDR frames), forward prediction frames (P frames) and virtual frames (VI frames) according to different encoding methods.

[0066] It should be noted that the first frame of a GOP is a key frame, and a GOP contains only one key frame. Key frames are encoded independently without reference to other frames. Key frames can serve as long-term reference frames. A forward-predicted frame is encoded with reference to its previous frame. The previous frame of a forward-predicted frame can be a key frame, a forward-predicted frame, or a virtual frame. A forward-predicted frame can serve as a short-term reference frame to provide a reference for the encoding of the following frame. Virtual frames are encoded only with reference to key frames. Virtual frames can serve as short-term reference frames to provide a reference for the encoding of the following frame.

[0067] In some embodiments, the key frame is used as a long-term reference frame, and the reconstructed frame generated by it persists throughout the entire picture group period. When a virtual frame or a forward prediction frame is not referenced, the memory of the cache module where it is located will be reused by subsequent short-term reference frames, so the number of cache modules occupied by the picture group is 3. Figure 2 As shown, the source image F1 is a key frame, and the source images F2 to F n-1 is the forward prediction frame, F n When F1 is encoded, the reconstructed frame is stored in the buffer module BUF1. n Reference, therefore, the cache module BUF1 is retained during the entire image group cycle. F2 is encoded with reference to F1, and the resulting reconstructed frame is stored in the cache module BUF2. F3 is encoded with reference to F2, and the resulting reconstructed frame is stored in the cache module BUF3. Since the reconstructed frame in BUF2 has been referenced by F3 and is no longer referenced by other frames, when F4 is encoded with reference to F3, the resulting reconstructed frame can reuse the cache module BUF2 where the reconstructed frame of F2 is located. The above steps are repeated until the virtual frame F is encoded. n , F n By referencing the F1 cached in BUF1 to reconstruct the frame, F is generated. n Reconstruct the frame and reuse F n-2 Buffer module BUF2 for reconstructed frames. Video encoding and compression can be completed by encoding the video frames in the above manner and saving the code stream to disk or sending the code stream over the network. During the above-mentioned video encoding process of the image group, the reconstructed frames corresponding to the key frames are stored in buffer module BUF1 for a long time, resulting in excessive system memory overhead and reduced video encoding performance.

[0068] In order to reduce the memory occupied by video during encoding and improve the performance of video encoding, an embodiment of the present application provides a video encoding method. Figure 3 This is a schematic diagram of the steps of the video encoding method according to the embodiment of the present application. Figure 3 The method of this embodiment is described in detail.

[0069] Step S100: Acquire a frame sequence.

[0070] In some embodiments, the video frames are arranged in time sequence to form a plurality of groups of pictures, and the groups of pictures include key frames, forward prediction frames and virtual frames. A frame sequence is obtained based on the groups of pictures, and the frame sequence includes a key frame, at least one forward prediction frame and at least one virtual frame, as described again with reference to Figure 1

[0071] The first frame of the frame sequence is the key frame, which represents the beginning of a group of pictures and contains complete image information, and can be decoded and displayed independently without reference to other frames. The second frame of the frame sequence is the forward prediction frame, which needs to be decoded by reference to the previous frame and encoded by prediction and motion compensation to reduce data volume and improve compression efficiency. The type of subsequent frames of the frame sequence is not limited in this embodiment, for example, the third frame of the frame sequence can be a forward prediction frame or a virtual frame. The forward prediction frame is encoded by reference to the previous frame, and the virtual frame is encoded by reference to the key frame.

[0072] In step S200, a cache module is obtained.

[0073] In some embodiments, the cache module is used to store reference reconstructed frames. In the process of video encoding, the original image frames, i.e. reconstructed frames, are restored by receiving compressed video data and performing entropy decoding, inverse quantization and inverse transformation and other steps on the video data. The reference reconstructed frames include key reconstructed frames, forward prediction reconstructed frames and virtual reconstructed frames. The key reconstructed frames are reconstructed frames generated by decoding key frame code streams, the forward prediction reconstructed frames are reconstructed frames generated by decoding forward prediction frame code streams, and the virtual reconstructed frames are reconstructed frames generated by decoding virtual frame code streams.

[0074] The cache module includes a first cache module and a second cache module, and the cache module is provided with a state identifier, which includes a to-be-referenced identifier and a writable identifier. For example, the state identifier of the first cache module is the to-be-referenced identifier, indicating that the reference reconstructed frame stored in the first cache module will be referenced by subsequent video frames, and the reference reconstructed frame stored in the first cache module can be obtained when the subsequent video frames are encoded. For another example, the state identifier of the second cache module is the writable identifier, indicating that the reference reconstructed frame stored in the second cache module is no longer referenced by subsequent video frames, and the data in the second cache module can be released and a new reconstructed frame can be stored.

[0075] In step S300, a key frame code stream is generated based on the key frame, and the key frame code stream is decoded.

[0076] In some embodiments, the key frame generates the key frame code stream by intra prediction. For example, the key frame is divided into blocks, and each block is encoded by intra prediction to generate the key frame code stream. Figure 4 ​As shown, first, the key frame is segmented based on the preset size to obtain a key frame macro block including real pixel values; then, a prediction pixel value of the key frame macro block is predicted by using an intra-frame prediction algorithm, and an intra-frame residual between the prediction pixel value and the real pixel value is calculated; then, a frequency domain transformation is performed on the intra-frame residual to calculate an intra-frame transformation coefficient, and quantization is performed on the intra-frame transformation coefficient to calculate an intra-frame quantization coefficient; finally, entropy coding is performed on the intra-frame quantization coefficient to generate a key frame code stream.

[0077] It should be noted that the encoder reduces data redundancy by predicting and encoding pixel values in the current frame by using the spatial correlation within the image frame, thereby compressing data. In the intra-frame encoding of the key frame, the encoder segments the key frame into a key frame macro block, and predicts the pixel value of the key frame macro block according to the pixel values of the key frame macro block around the key frame macro block that have been encoded. The intra-frame residual between the prediction pixel value and the real pixel value is calculated, thereby achieving compression.

[0078] Further, the encoder generates a key frame code stream by performing transformation, quantization and entropy coding operations on the intra-frame residual. The energy in the key frame is concentrated on low frequency coefficients by Discrete Cosine Transform (DCT), and some high frequency coefficients are discarded to obtain intra-frame transformation coefficients, thereby reducing the data amount. Then, the intra-frame transformation coefficients are discretized, and the intra-frame transformation coefficients are mapped to a numerical set according to a preset quantization step to calculate intra-frame quantization coefficients. Finally, the intra-frame quantization coefficients are further compressed by using the statistical characteristics of the source to generate a key frame code stream, and the key frame code stream is decoded by using a decoder to generate a key reconstructed frame.

[0079] Step S400, store the key reconstructed frame into the first cache module, and generate a forward prediction reconstructed frame based on the key reconstructed frame in the first cache module.

[0080] As shown in the example, Figure 5 As shown, the source image F1 is a key frame, intra-frame encoding is performed on the key frame to generate a key reconstructed frame, and the key reconstructed frame is stored in the first cache module BUF1 as a reference reconstructed frame of the source image F2. The source image F2 obtains the key reconstructed frame from the first cache module BUF1, and generates a forward prediction reconstructed frame by referring to the key reconstructed frame.

[0081] Step S500, store the forward prediction reconstructed frame into the second cache module, set the state identifier of the second cache module to a to-be-referenced identifier, and set the state identifier of the first cache module to a writable identifier.

[0082] Again referring to Figure 5The forward-predicted reconstructed frame corresponding to source image F2 is stored in the second buffer module BUF2 as a reference reconstructed frame for source image F3. The status flag of second buffer module BUF2 is set to the pending reference flag, indicating that the cached content in second buffer module BUF2 will be referenced by subsequent frames. The status flag of first buffer module BUF1 is then set to the writable flag, indicating that the cached content in first buffer module BUF1 is no longer referenced by subsequent frames and that first buffer module BUF1 can be reused. At this point, memory space in first buffer module BUF1 is released, reducing memory usage.

[0083] Step S600: traverse the frame sequence starting from the third frame in the frame sequence to obtain the current frame.

[0084] Since different frame sequences correspond to different video frame arrangement orders, it is necessary to traverse the frame sequence starting from the third frame of the frame sequence and determine the type and position of the current frame, so as to perform different steps on the current frame according to the type and position of the current frame.

[0085] In some embodiments, the current frame is a forward predicted frame and the next frame after the current frame is a forward predicted frame. At this time, a reference reconstructed frame stored in a cache module whose status identifier is a pending reference identifier is obtained (S611); then a forward predicted reconstructed frame corresponding to the current frame is generated based on the reference reconstructed frame, and the forward predicted reconstructed frame is stored in a cache module whose status identifier is a writable identifier (S612); and the status identifier of the cache module where the reference reconstructed frame is located is set to a writable identifier, and the status identifier of the cache module where the forward predicted reconstructed frame corresponding to the current frame is located is set to a pending reference identifier (S613).

[0086] For example, see Figure 5 , the current frame is the forward prediction frame F3, the current status flag of the first cache module BUF1 is a writable flag, and the status flag of the second cache module BUF2 is a pending reference flag. The reference reconstructed frame corresponding to F2 stored in the second cache module BUF2 is obtained, and the forward prediction reconstructed frame corresponding to F3 is generated based on the reference reconstructed frame corresponding to F2, and the forward prediction reconstructed frame corresponding to F3 is stored in the first cache module BUF1. Finally, the status flag of the cache module is updated. Since the reference reconstructed frame corresponding to F2 stored in the second cache module BUF2 is no longer referenced by subsequent video frames, the status flag of the second cache module BUF2 is set to a writable flag. Since the forward prediction reconstructed frame corresponding to F3 stored in the first cache module BUF1 will be referenced by F4 as a reference reconstructed frame, the status flag of the first cache module BUF1 is set to a pending reference flag.

[0087] In some embodiments, a forward-predicted reconstructed frame corresponding to a current frame can be generated from a reference reconstructed frame based on an inter-frame coding algorithm. To generate the forward-predicted reconstructed frame corresponding to the current frame, the forward-predicted frame is first segmented based on a preset size to obtain forward-predicted frame macroblocks. For example, the forward-predicted frame macroblocks can be 16×16 pixel blocks. A forward-matching macroblock of the forward-predicted frame macroblock is then retrieved from the reference reconstructed frame, and a forward motion vector is calculated between the forward-matching macroblock and the forward-predicted frame macroblock.

[0088] Furthermore, based on the forward matching macroblock and the forward motion vector, a forward matching prediction macroblock is generated, and the forward inter-frame residual between the forward prediction frame macroblock and the forward matching prediction macroblock is calculated. The forward inter-frame residual is used to characterize the difference between the forward prediction frame macroblock and the forward matching prediction macroblock. Next, a frequency domain transform is performed on the forward inter-frame residual to calculate the forward inter-frame transform coefficients, and the forward inter-frame transform coefficients are quantized to calculate the forward inter-frame quantization coefficients. Entropy coding is performed on the forward inter-frame quantization coefficients to generate a forward prediction frame code stream. Finally, the forward prediction frame code stream is decoded to generate a forward prediction reconstructed frame.

[0089] The above inter-frame coding algorithm can utilize the temporal correlation between adjacent frames to predict a forward prediction reconstructed frame based on a reference reconstructed frame, thereby removing redundant information in the frame sequence and improving compression efficiency.

[0090] In some embodiments, the current frame is a forward predicted frame and the frame after the current frame is a virtual frame, or the current frame is a forward predicted frame and the current frame is the last frame of the frame sequence. At this time, the reference reconstructed frame stored in the cache module whose status identifier is the to-be-referenced identifier is obtained (S621), and the forward predicted reconstructed frame corresponding to the current frame is generated based on the reference reconstructed frame (S622), and the status identifier of the cache module where the reference reconstructed frame is located is set to a writable identifier (S623).

[0091] For example, see Figure 5 , the current frame is F n-1 The current status of the first buffer module BUF1 is marked as writable, and the status of the second buffer module BUF2 is marked as reference pending. The reference reconstructed frame (ie, F stored in the second buffer module BUF2) is obtained. n-2 corresponding reference reconstructed frame), and according to F n-2 The corresponding reference reconstructed frame generates F n-1 The corresponding forward prediction reconstructed frame. n-2 The corresponding reference reconstructed frame is no longer referenced by subsequent video frames, so the state flag of the second buffer module BUF2 is set to a writable flag. n-1The corresponding reference reconstructed frame is no longer referenced by subsequent video frames, thus, there is no need to store F n-1 The corresponding forward-predicted reconstructed frame, thereby reducing the memory bandwidth of the system. At this time, the state identifier of the first buffer module BUF1 is not updated and remains as the writable identifier.

[0092] In some embodiments, the current frame is a forward-predicted frame and the next frame of the current frame is a virtual frame, or the current frame is a forward-predicted frame and the current frame is the last frame of the frame sequence, since the current frame is no longer referenced by subsequent video frames, there is no need to generate the corresponding forward-predicted reconstructed frame of the current frame, thereby reducing the memory bandwidth of the system and improving the video encoding efficiency.

[0093] In some embodiments, the current frame is a virtual frame, at this time, the key frame code stream is obtained and the key frame code stream is decoded again to generate a key reconstructed frame (S631). The key reconstructed frame is then stored in the buffer module with the state identifier as the writable identifier, and the state identifier of the buffer module where the key reconstructed frame is stored is set as the to-be-referenced identifier (S632). And the virtual reconstructed frame corresponding to the current frame is generated according to the key reconstructed frame, and the virtual reconstructed frame is stored in the buffer module with the state identifier as the writable identifier (S633). Finally, the state identifier of the buffer module where the virtual reconstructed frame is stored is set as the to-be-referenced identifier, and the state identifier of the buffer module where the key reconstructed frame is stored is set as the writable identifier (S634).

[0094] For example, referring to Figure 5 , the current frame is a virtual frame F n , the state identifiers of the first buffer module BUF1 and the second buffer module BUF2 are both writable identifiers, the key frame code stream is obtained and decoded, the decoded key reconstructed frame is stored in the first buffer module BUF1, and the state identifier of the first buffer module BUF1 is set as the to-be-referenced identifier, and the virtual reconstructed frame F n corresponding to the current frame is generated according to the key reconstructed frame, and the virtual reconstructed frame F n corresponding to the current frame is stored in the second buffer module BUF2. Since the key reconstructed frame stored in the first buffer module BUF1 is no longer referenced by other frames, the state identifier of the first buffer module BUF1 is set as the writable identifier. Since the virtual reconstructed frame F n corresponding to the current frame stored in the second buffer module BUF2 will be referenced by subsequent video frames as a reference reconstructed frame, the state identifier of the second buffer module BUF2 is set as the to-be-referenced identifier.

[0095] In some embodiments, a virtual reconstructed frame corresponding to a current frame can be generated according to the key reconstructed frame by using an inter-frame coding algorithm. In the process of generating the virtual reconstructed frame, the virtual frame is first divided into virtual frame macroblocks according to a preset size. For example, the virtual frame macroblock can be a 16x16 pixel block. Then, a virtual matching macroblock of the virtual frame macroblock is queried from the key reconstructed frame, and a virtual motion vector between the virtual matching macroblock and the virtual frame macroblock is calculated.

[0096] Further, a virtual frame prediction macroblock is generated according to the virtual matching macroblock and the virtual motion vector, and a virtual inter-frame residual between the virtual frame macroblock and the virtual frame prediction macroblock is calculated. The virtual inter-frame residual is used to represent the difference between the virtual frame macroblock and the virtual frame prediction macroblock. Then, a frequency domain transform is performed on the virtual inter-frame residual to calculate virtual inter-frame transform coefficients, and a quantization is performed on the virtual inter-frame transform coefficients to calculate virtual inter-frame quantized coefficients. An entropy coding is performed on the virtual inter-frame quantized coefficients to generate a virtual frame bitstream. Finally, the virtual frame bitstream is decoded to generate the virtual reconstructed frame.

[0097] In some embodiments, an encoded sequence is generated according to the encoded bitstream, and then the video is encoded and compressed. As shown in FIG. 6, the encoded bitstream is first obtained, and the encoded bitstream includes a key frame bitstream, a forward prediction frame bitstream, and a virtual frame bitstream. Then, the encoded sequence is generated based on the arrangement order of the frame sequence, and the encoded sequence includes the encoded bitstream. Figure 6

[0098] In some embodiments, by performing a decoding operation on the encoded bitstream, a reference reconstructed frame can be generated. In the process of decoding the encoded bitstream, the encoded bitstream is first entropy decoded to extract quantized coefficients, coding modes, motion vectors, and other parameter information from the compressed bitstream. Then, the reordered data is reordered to restore its order before encoding. Then, the reordered data is inverse quantized using the quantization parameter and the quantized coefficients to recover the residual information, and the inverse transformed data is inverse transformed to recover the original spatial data, convert the residual information from the frequency domain to the spatial domain, and then obtain the reconstructed residual signal. Finally, a prediction module in the decoder is used to generate a prediction macroblock, and the prediction macroblock is added to the reconstructed residual signal to generate a reference reconstructed frame.

[0099] In some embodiments, because there are block effects and compression noise in the reference reconstructed frame, the video quality is reduced, and therefore by performing loop filtering on the reference reconstructed frame, the block boundary can be smoothed and the compression noise can be removed, thereby optimizing the reference reconstructed frame and improving the quality of video encoding.

[0100] ​According to the technical solution, the video coding method comprises: acquiring a frame sequence and a buffer module. A key frame code stream and a key reconstructed frame are generated according to the key frame; the key reconstructed frame is stored in the first buffer module, and the first buffer module is set with a to-be-referenced identifier; a forward prediction reconstructed frame is generated based on the key reconstructed frame, and the forward prediction reconstructed frame is stored in the second buffer module; the second buffer module is set with the to-be-referenced identifier, and the first buffer module is set with a writable identifier; the frame sequence is traversed to acquire a current frame, a reference reconstructed frame stored in the buffer module with the to-be-referenced identifier is acquired according to the type of the current frame, and a reconstructed frame corresponding to the current frame is generated; the reconstructed frame is stored in the buffer module with the writable identifier, and the buffer module in which the reference reconstructed frame is stored is released, and the state identifier of the buffer module is updated. The memory of the first buffer module and the second buffer module is released in time, and the memory occupation time is reduced.

[0101] The method embodiment according to the application is exemplified above, and based on the video coding method provided by the embodiment of the application, the embodiment of the application provides a video coding device. Figure 7 A structural schematic diagram of a video coding device provided by the embodiment of the application is shown in FIG. 7. Figure 7 The device comprises the following modules.

[0102] An image preprocessing module 710 is configured to acquire a frame sequence, the frame sequence comprising a key frame, at least one forward prediction frame and at least one virtual frame, the first frame of the frame sequence being the key frame, and the second frame of the frame sequence being the forward prediction frame.

[0103] A buffer acquisition module 720 is configured to acquire a buffer module, the buffer module being used to store a reference reconstructed frame, the reference reconstructed frame comprising a key reconstructed frame, a forward prediction reconstructed frame and a virtual reconstructed frame, the buffer module comprising a first buffer module and a second buffer module, and the buffer module being set with a state identifier, the state identifier comprising a to-be-referenced identifier and a writable identifier;

[0104] A video coding processing module 730 is configured to generate a key frame code stream according to a key frame, and decode the key frame code stream to generate a key reconstructed frame; store the key reconstructed frame in the first buffer module and generate a forward prediction reconstructed frame based on the key reconstructed frame in the first buffer module; store the forward prediction reconstructed frame in the second buffer module, set the state identifier of the second buffer module as the to-be-referenced identifier, and set the state identifier of the first buffer module as the writable identifier; and start traversing the frame sequence from the third frame of the frame sequence to acquire a current frame.

[0105] A processing core selection module 740 is configured to send a scheduling instruction to the video coding processing module 730 based on the type of the current frame, the scheduling instruction being used to instruct the video coding processing module 730 to generate a reference reconstructed frame and store the reference reconstructed frame.

[0106] The encoder post-processing module 750 is configured to obtain the encoded code stream, the encoded code stream comprising the key frame code stream, the forward prediction frame code stream and the virtual frame code stream; generate an encoded sequence based on the arrangement order of the frame sequence, the encoded sequence comprising the encoded code stream; decode the encoded code stream to generate a reference reconstructed frame; and perform loop filtering on the reference reconstructed frame to optimize the reference reconstructed frame.

[0107] In some embodiments, the video encoding processing module 730 is further configured to, if the current frame is a forward prediction frame and a next frame of the current frame is a forward prediction frame, obtain a reference reconstructed frame stored in the buffer module with the state identifier being the to-be-referenced identifier; generate a forward prediction reconstructed frame corresponding to the current frame according to the reference reconstructed frame, and store the forward prediction reconstructed frame in the buffer module with the state identifier being the writable identifier; set the state identifier of the buffer module where the reference reconstructed frame is stored as the writable identifier, and set the state identifier of the buffer module where the forward prediction reconstructed frame corresponding to the current frame is stored as the to-be-referenced identifier.

[0108] If the current frame is a forward prediction frame and a next frame of the current frame is a virtual frame, or the current frame is a forward prediction frame and the current frame is the last frame of the frame sequence, the reference reconstructed frame stored in the buffer module with the state identifier being the to-be-referenced identifier is obtained; the forward prediction reconstructed frame corresponding to the current frame is generated according to the reference reconstructed frame; and the state identifier of the buffer module where the reference reconstructed frame is stored is set as the writable identifier.

[0109] If the current frame is a virtual frame, the key frame code stream is obtained, and the key frame code stream is decoded again to generate a key reconstructed frame; the key reconstructed frame is stored in the buffer module with the state identifier being the writable identifier, and the state identifier of the buffer module where the key reconstructed frame is stored is set as the to-be-referenced identifier; the virtual reconstructed frame corresponding to the current frame is generated according to the key reconstructed frame, and the virtual reconstructed frame is stored in the buffer module with the state identifier being the writable identifier; the state identifier of the buffer module where the virtual reconstructed frame is stored is set as the to-be-referenced identifier, and the state identifier of the buffer module where the key reconstructed frame is stored is set as the writable identifier.

[0110] The above describes the device embodiments of the present application. For the specific execution process, technical problems and effects, alternative manners and combination manners of data, terms, names, steps, please refer to the description in the method embodiments, which will not be repeated here.

[0111] The embodiments of the present application also provide an electronic device, comprising a processor, a memory and a bus. Wherein, the processor and the memory are communicatively connected through the bus, the memory stores computer program instructions executable by the processor, and the processor is configured to implement the steps of the video encoding method in any one of the first aspect.

[0112] For example, as Figure 8As shown, the electronic device includes a processor 810, at least one communication bus 820, a user interface 830, at least one external communication interface 840 and a memory 850. Among them, the communication bus 820 is configured to realize the connection communication between the components. Among them, the user interface 830 can include a display screen, and the external communication interface 840 can include a standard wired interface and a wireless interface. Among them, the memory 850 stores a computer program. Among them, the processor 810 is used to execute the computer program stored in the memory 850.

[0113] The similar parts among the embodiments provided in the application can be referred to each other, and the specific embodiments provided above are only a few examples under the general concept of the application, and do not constitute the limitation of the protection scope of the application. For those skilled in the art, any other embodiments extended according to the application scheme without creative labor shall fall within the protection scope of the application.

Claims

1. A method of video coding, the method comprising: The method comprises the following steps: obtaining a frame sequence, the frame sequence comprising a key frame, at least one forward prediction frame and at least one virtual frame, the first frame of the frame sequence being the key frame, and the second frame of the frame sequence being the forward prediction frame; obtaining a cache module, the cache module being used for storing reference reconstruction frames, the reference reconstruction frames comprising a key reconstruction frame, a forward prediction reconstruction frame and a virtual reconstruction frame, the cache module comprising a first cache module and a second cache module, and the cache module being provided with a state identifier, the state identifier comprising a to-be-referenced identifier and a writable identifier; generating a key frame code stream according to the key frame, and decoding the key frame code stream to generate a key reconstruction frame; storing the key reconstruction frame into the first cache module, and generating a forward prediction reconstruction frame based on the key reconstruction frame in the first cache module; storing the forward prediction reconstruction frame into the second cache module, setting the state identifier of the second cache module as the to-be-referenced identifier, and setting the state identifier of the first cache module as the writable identifier; starting to perform traversal on the frame sequence from the third frame of the frame sequence to obtain a current frame; if the current frame is a forward prediction frame and the frame following the current frame is a forward prediction frame, obtaining a reference reconstruction frame stored in a cache module with the state identifier of to-be-referenced; generating a forward prediction reconstruction frame corresponding to the current frame according to the reference reconstruction frame, and storing the forward prediction reconstruction frame into a cache module with the state identifier of writable; setting the state identifier of the cache module in which the reference reconstruction frame is stored as the writable identifier, and setting the state identifier of the cache module in which the forward prediction reconstruction frame corresponding to the current frame is stored as the to-be-referenced identifier.

2. The video coding method of claim 1, wherein, The video encoding method further comprises the following steps: if the current frame is a forward prediction frame and the frame following the current frame is a virtual frame, or the current frame is a forward prediction frame and the current frame is the last frame of the frame sequence, obtaining a reference reconstruction frame stored in a cache module with the state identifier of to-be-referenced; generating a forward prediction reconstruction frame corresponding to the current frame according to the reference reconstruction frame; setting the state identifier of the cache module in which the reference reconstruction frame is stored as the writable identifier.

3. The video coding method of claim 1, wherein, The method further comprises the following steps: if the current frame is a virtual frame, obtaining the key frame code stream, and decoding the key frame code stream again to generate the key reconstruction frame; storing the key reconstruction frame into a cache module with the state identifier of writable, and setting the state identifier of the cache module in which the key reconstruction frame is stored as the to-be-referenced identifier; generating a virtual reconstruction frame corresponding to the current frame according to the key reconstruction frame, and storing the virtual reconstruction frame into a cache module with the state identifier of writable; setting the state identifier of the cache module in which the virtual reconstruction frame is stored as the to-be-referenced identifier, and setting the state identifier of the cache module in which the key reconstruction frame is stored as the writable identifier.

4. The video coding method of claim 1, wherein, The method of generating a key frame code stream according to a key frame comprises the following steps: segmenting the key frame based on a preset size to obtain a key frame macroblock, the key frame macroblock comprising real pixel values; predicting, by using an intra prediction algorithm, a predicted pixel value of the macroblock of the key frame, and calculating an intra residual between the predicted pixel value and a real pixel value; performing a frequency domain transform on the intra residual to calculate an intra transform coefficient, and performing quantization on the intra transform coefficient to calculate an intra quantized coefficient; performing entropy coding on the intra quantized coefficient to generate a key frame code stream.

5. The video coding method of claim 4, wherein, The generating of the forward prediction reconstructed frame corresponding to the current frame according to the reference reconstructed frame comprises: segmenting the forward prediction frame based on a preset size to obtain a forward prediction frame macroblock; querying a forward matching macroblock of the forward prediction frame macroblock from the reference reconstructed frame, and calculating a forward motion vector between the forward matching macroblock and the forward prediction frame macroblock; generating a forward matching prediction macroblock according to the forward matching macroblock and the forward motion vector, and calculating a forward inter-frame residual between the forward prediction frame macroblock and the forward matching prediction macroblock; performing a frequency domain transform on the forward inter-frame residual to calculate a forward inter-frame transform coefficient, and performing quantization on the forward inter-frame transform coefficient to calculate a forward inter-frame quantized coefficient; performing entropy coding on the forward inter-frame quantized coefficient to generate a forward prediction frame code stream; decoding the forward prediction frame code stream to generate the forward prediction reconstructed frame.

6. The video coding method of claim 5, wherein, The generating of the virtual reconstructed frame corresponding to the current frame according to the key reconstructed frame comprises: segmenting the virtual frame based on a preset size to obtain a virtual frame macroblock; querying a virtual matching macroblock of the virtual frame macroblock from the key reconstructed frame, and calculating a virtual motion vector between the virtual matching macroblock and the virtual frame macroblock; generating a virtual frame prediction macroblock according to the virtual matching macroblock and the virtual motion vector, and calculating a virtual inter-frame residual between the virtual frame macroblock and the virtual frame prediction macroblock; performing a frequency domain transform on the virtual inter-frame residual to calculate a virtual inter-frame transform coefficient, and performing quantization on the virtual inter-frame transform coefficient to calculate a virtual inter-frame quantized coefficient; performing entropy coding on the virtual inter-frame quantized coefficient to generate a virtual frame code stream; decoding the virtual frame code stream to generate the virtual reconstructed frame.

7. The video coding method of claim 6, wherein, Further comprising: obtaining an encoding code stream, the encoding code stream comprising a key frame code stream, a forward prediction frame code stream and a virtual frame code stream; generating an encoding sequence based on the arrangement order of the frame sequence, the encoding sequence comprising the encoding code stream; decoding the encoding code stream to generate the reference reconstructed frame; performing loop filtering on the reference reconstructed frame to optimize the reference reconstructed frame.

8. A video encoding apparatus, comprising: Comprise: an image preprocessing module configured to obtain a frame sequence, the frame sequence comprising a key frame, at least one forward prediction frame and at least one virtual frame, a first frame of the frame sequence being the key frame, and a second frame of the frame sequence being the forward prediction frame; The cache obtaining module is configured to obtain a cache module for storing reference reconstructed frames, the reference reconstructed frames including key reconstructed frames, forward prediction reconstructed frames and virtual reconstructed frames, the cache module including a first cache module and a second cache module, the cache module being provided with a state identifier, the state identifier including a to-be-referenced identifier and a writable identifier; The video encoding processing module is configured to generate a key frame code stream according to a key frame, and decode the key frame code stream to generate a key reconstructed frame; store the key reconstructed frame into the first cache module and generate a forward prediction reconstructed frame based on the key reconstructed frame in the first cache module; store the forward prediction reconstructed frame into the second cache module, set the state identifier of the second cache module as the to-be-referenced identifier, and set the state identifier of the first cache module as the writable identifier; The video encoding processing module is configured to generate a key frame code stream according to a key frame, and decode the key frame code stream to generate a key reconstructed frame; store the key reconstructed frame into the first cache module and generate a forward prediction reconstructed frame based on the key reconstructed frame in the first cache module; store the forward prediction reconstructed frame into the second cache module, set the state identifier of the second cache module as the to-be-referenced identifier, and set the state identifier of the first cache module as the writable identifier; The processing core selection module is configured to send a scheduling instruction to the video encoding processing module based on the type of the current frame, the scheduling instruction being used to instruct the video encoding processing module to generate a reference reconstructed frame and store the reference reconstructed frame; The encoder post-processing module is configured to obtain an encoding code stream, the encoding code stream including a key frame code stream, a forward prediction frame code stream and a virtual frame code stream; generate an encoding sequence based on the arrangement order of the frame sequence, the encoding sequence including the encoding code stream; decode the encoding code stream to generate the reference reconstructed frame; and perform loop filtering on the reference reconstructed frame to optimize the reference reconstructed frame. The video encoding processing module is further configured to:

9. The video coding device of claim 8, wherein, if the current frame is a forward prediction frame and a next frame of the current frame is a forward prediction frame, obtain a reference reconstructed frame stored in a cache module with a state identifier being the to-be-referenced identifier; generate a forward prediction reconstructed frame corresponding to the current frame according to the reference reconstructed frame, and store the forward prediction reconstructed frame into a cache module with a state identifier being the writable identifier; set the state identifier of the cache module where the reference reconstructed frame is stored as the writable identifier, and set the state identifier of the cache module where the forward prediction reconstructed frame corresponding to the current frame is stored as the to-be-referenced identifier; if the current frame is a forward prediction frame and a next frame of the current frame is a virtual frame, or the current frame is a forward prediction frame and the current frame is a last frame of the frame sequence, obtain a reference reconstructed frame stored in a cache module with a state identifier being the to-be-referenced identifier; generate a forward prediction reconstructed frame corresponding to the current frame according to the reference reconstructed frame; and set the state identifier of the cache module where the reference reconstructed frame is stored as the writable identifier. ​ If the current frame is a virtual frame, the key frame code stream is obtained, and the key frame code stream is decoded again to generate the key reconstructed frame; the key reconstructed frame is stored in a cache module with a state identifier as writable identifier, and the state identifier of the cache module where the key reconstructed frame is located is set as a to-be-referenced identifier; a virtual reconstructed frame corresponding to the current frame is generated according to the key reconstructed frame, and the virtual reconstructed frame is stored in a cache module with a state identifier as writable identifier; the state identifier of the cache module where the virtual reconstructed frame is located is set as a to-be-referenced identifier, and the state identifier of the cache module where the key reconstructed frame is located is set as a writable identifier.

10. An electronic device, comprising: comprising a processor, a memory and a bus; the processor and the memory are communicatively connected through the bus; the memory stores computer program instructions executable by the processor, and the processor is configured to implement the video encoding method according to any one of claims 1-7.

Images