Video coding method and device and electronic equipment

Abstract

The invention provides a video coding method and device and electronic equipment. The method comprises the following steps: acquiring a frame sequence and a cache module; generating a key frame code stream and a key reconstruction frame according to the key frame; storing the key reconstruction frame into a first cache module, and setting a to-be-referenced identifier for the first cache module; generating a forward prediction reconstruction frame based on the key reconstruction frame, and storing the forward prediction reconstruction frame into a second cache module; setting a to-be-referenced identifier for the second cache module, and setting a writable identifier for the first cache module; traversing the frame sequence to obtain a current frame, and obtaining a reference reconstruction frame stored in a cache module under the to-be-referenced identifier according to the type of the current frame to generate a reconstruction frame corresponding to the current frame; and storing the reconstruction frame into the cache module under the writable identifier, releasing the cache module where the reference reconstruction frame is located, and updating the state identifier of the cache module. According to the method, the memory occupation duration is reduced by releasing the memories of the first cache module and the second cache module in time.

Description

Technical Field

[0001] The present application relates to the technical field of video compression coding, and in particular to a video coding method, device and electronic device. Background Art

[0002] In intelligent security monitoring systems, video coding and compression is a key link to ensure smooth transmission and storage of monitoring images. Encoding videos based on the H.264 / H.265 compression protocol can reduce the amount of video transmission data. In the process of video encoding, first select high-quality and stable images as long-term reference frames, and predict the image encoding of the next frame based on the encoded pixel values ​​in the long-term reference frame, and then remove temporal redundancy based on the pixel similarity between consecutive frames to complete video encoding.

[0003] Since the long-term reference frame needs to continuously occupy the system memory in the above process, the system memory overhead is too large during the video encoding process, and the memory occupancy time is too long, resulting in low video encoding performance. Summary of the invention

[0004] The present application provides a video encoding method, device and electronic device, which releases the memory of a cache module in a timely manner, thereby reducing the memory occupancy time and improving the performance of video encoding.

[0005] In a first aspect, the present application provides a video encoding method, comprising:

[0006] Acquire a frame sequence, wherein the frame sequence includes a key frame, at least one forward prediction frame and at least one virtual frame, the first frame of the frame sequence is the key frame, and the second frame of the frame sequence is the forward prediction frame;

[0007] Acquire a cache module, the cache module is used to store reference reconstructed frames, the reference reconstructed frames include key reconstructed frames, forward predicted reconstructed frames and virtual reconstructed frames, the cache module includes a first cache module and a second cache module, the cache module is provided with a state flag, the state flag includes a to-be-referenced flag and a writable flag;

[0008] Generating a key frame code stream according to the key frame, and decoding the key frame code stream to generate a key reconstruction frame;

[0009] Storing the key reconstructed frame in the first cache module, and generating a forward prediction reconstructed frame based on the key reconstructed frame in the first cache module;

[0010] storing the forward prediction reconstructed frame into the second cache module, setting the state flag of the second cache module to a pending reference flag, and setting the state flag of the first cache module to a writable flag;

[0011] Perform traversal on the frame sequence starting from the third frame of the frame sequence to obtain a current frame;

[0012] If the current frame is a forward predicted frame and a frame subsequent to the current frame is a forward predicted frame, obtaining a reference reconstructed frame stored in a cache module whose state identifier is a to-be-referenced identifier;

[0013] Generating a forward prediction reconstructed frame corresponding to the current frame according to the reference reconstructed frame, and storing the forward prediction reconstructed frame in a cache module whose status is marked as a writable flag;

[0014] The state flag of the cache module where the reference reconstructed frame is located is set to a writable flag, and the state flag of the cache module where the forward prediction reconstructed frame corresponding to the current frame is located is set to a pending reference flag.

[0015] In some embodiments, the video encoding method further includes: if the current frame is a forward predicted frame and the frame after the current frame is a virtual frame, or the current frame is a forward predicted frame and the current frame is the last frame of the frame sequence, obtaining a reference reconstructed frame stored in a cache module whose status identifier is a to-be-referenced identifier;

[0016] Generate a forward prediction reconstructed frame corresponding to the current frame according to the reference reconstructed frame;

[0017] The state flag of the cache module where the reference reconstructed frame is located is set to a writable flag.

[0018] In some implementations, the video encoding method further includes: if the current frame is a virtual frame, acquiring the key frame bitstream, and decoding the key frame bitstream again to generate the key reconstructed frame;

[0019] The key reconstructed frame is stored in a cache module whose state flag is a writable flag, and the state flag of the cache module where the key reconstructed frame is located is set as a pending reference flag;

[0020] Generate a virtual reconstructed frame corresponding to the current frame according to the key reconstructed frame, and store the virtual reconstructed frame in a cache module whose status is marked as a writable flag;

[0021] The state flag of the cache module where the virtual reconstructed frame is located is set as a pending reference flag, and the state flag of the cache module where the key reconstructed frame is located is set as a writable flag.

[0022] In some implementations, generating a key frame bitstream according to a key frame includes: segmenting the key frame based on a preset size to obtain key frame macroblocks, wherein the key frame macroblocks include real pixel values;

[0023] Predicting the predicted pixel value of the key frame macroblock by using an intra-frame prediction algorithm, and calculating the intra-frame residual between the predicted pixel value and the true pixel value;

[0024] Performing frequency domain transformation on the intra-frame residual to calculate intra-frame transformation coefficients, and performing quantization on the intra-frame transformation coefficients to calculate intra-frame quantization coefficients;

[0025] Entropy coding is performed on the intra-frame quantization coefficients to generate a key frame code stream.

[0026] In some implementations, generating a forward prediction reconstructed frame corresponding to the current frame according to the reference reconstructed frame includes: segmenting the forward prediction frame based on a preset size to obtain forward prediction frame macroblocks;

[0027] Query the forward matching macroblock of the forward prediction frame macroblock from the reference reconstructed frame, and calculate the forward motion vector between the forward matching macroblock and the forward prediction frame macroblock;

[0028] Generate a forward matching prediction macroblock according to the forward matching macroblock and the forward motion vector, and calculate a forward inter-frame residual between the forward prediction frame macroblock and the forward matching prediction macroblock;

[0029] Performing frequency domain transformation on the forward inter-frame residual to calculate forward inter-frame transformation coefficients, and performing quantization on the forward inter-frame transformation coefficients to calculate forward inter-frame quantization coefficients;

[0030] Performing entropy coding on the forward inter-frame quantization coefficients to generate a forward prediction frame code stream;

[0031] The forward prediction frame code stream is decoded to generate a forward prediction reconstructed frame.

[0032] In some implementations, generating a virtual reconstructed frame corresponding to the current frame according to the key reconstructed frame includes: segmenting the virtual frame based on a preset size to obtain virtual frame macroblocks;

[0033] Querying a virtual matching macroblock of the virtual frame macroblock from the key reconstructed frame, and calculating a virtual motion vector between the virtual matching macroblock and the virtual frame macroblock;

[0034] Generate a virtual frame prediction macroblock according to the virtual matching macroblock and the virtual motion vector, and calculate a virtual inter-frame residual between the virtual frame macroblock and the virtual frame prediction macroblock;

[0035] Performing frequency domain transformation on the virtual inter-frame residual to calculate virtual inter-frame transformation coefficients, and performing quantization on the virtual inter-frame transformation coefficients to calculate virtual inter-frame quantization coefficients;

[0036] Performing entropy coding on the virtual inter-frame quantization coefficients to generate a virtual frame code stream;

[0037] The virtual frame code stream is decoded to generate a virtual reconstructed frame.

[0038] In some implementations, the video encoding method further includes: obtaining an encoding bitstream, wherein the encoding bitstream includes a key frame bitstream, a forward prediction frame bitstream, and a virtual frame bitstream;

[0039] Generate a coding sequence based on the arrangement order of the frame sequence, the coding sequence including the coding bit stream;

[0040] Decoding the encoded bitstream to generate the reference reconstructed frame;

[0041] Loop filtering is performed on the reference reconstructed frame to optimize the reference reconstructed frame.

[0042] In a second aspect, an embodiment of the present application provides a video encoding device, including: an image preprocessing module, configured to: obtain a frame sequence, the frame sequence including a key frame, at least one forward prediction frame and at least one virtual frame, the first frame of the frame sequence is the key frame, and the second frame of the frame sequence is the forward prediction frame;

[0043] A cache acquisition module is configured to: acquire a cache module, the cache module is used to store reference reconstructed frames, the reference reconstructed frames include key reconstructed frames, forward predicted reconstructed frames and virtual reconstructed frames, the cache module includes a first cache module and a second cache module, the cache module is provided with a state flag, the state flag includes a to-be-referenced flag and a writable flag;

[0044] The video encoding processing module is configured to: generate a key frame code stream according to the key frame, and decode the key frame code stream to generate a key reconstruction frame; store the key reconstruction frame in the first cache module and generate a forward prediction reconstruction frame based on the key reconstruction frame in the first cache module; store the forward prediction reconstruction frame in the second cache module, and set the state flag of the second cache module to a pending reference flag, and set the state flag of the first cache module to a writable flag; traverse the frame sequence starting from the third frame of the frame sequence to obtain the current frame;

[0045] A processing core selection module is configured to: send a scheduling instruction to the video encoding processing module based on the type of the current frame, wherein the scheduling instruction is used to instruct the video encoding processing module to generate a reference reconstructed frame and store the reference reconstructed frame;

[0046] The encoder post-processing module is configured to: obtain a coded bitstream, the coded bitstream includes a key frame bitstream, a forward prediction frame bitstream and a virtual frame bitstream; generate a coded sequence based on the arrangement order of the frame sequence, the coded sequence includes the coded bitstream; decode the coded bitstream to generate the reference reconstructed frame; perform loop filtering on the reference reconstructed frame to optimize the reference reconstructed frame.

[0047] In some embodiments, the video encoding processing module is further configured to: if the current frame is a forward predicted frame and the next frame after the current frame is a forward predicted frame, obtain a reference reconstructed frame stored in a cache module whose status identifier is a pending reference identifier; generate a forward predicted reconstructed frame corresponding to the current frame according to the reference reconstructed frame, and store the forward predicted reconstructed frame in a cache module whose status identifier is a writable identifier; set the status identifier of the cache module where the reference reconstructed frame is located to a writable identifier, and set the status identifier of the cache module where the forward predicted reconstructed frame corresponding to the current frame is located to a pending reference identifier;

[0048] If the current frame is a forward predicted frame and the frame after the current frame is a virtual frame, or the current frame is a forward predicted frame and the current frame is the last frame of the frame sequence, then obtaining a reference reconstructed frame stored in a cache module whose status identifier is a reference identifier; generating a forward predicted reconstructed frame corresponding to the current frame according to the reference reconstructed frame; and setting the status identifier of the cache module where the reference reconstructed frame is located to a writable identifier;

[0049] If the current frame is a virtual frame, the key frame code stream is obtained, and the key frame code stream is decoded again to generate the key reconstructed frame; the key reconstructed frame is stored in a cache module whose status identifier is a writable identifier, and the status identifier of the cache module where the key reconstructed frame is located is set as a pending reference identifier; a virtual reconstructed frame corresponding to the current frame is generated according to the key reconstructed frame, and the virtual reconstructed frame is stored in a cache module whose status identifier is a writable identifier; the status identifier of the cache module where the virtual reconstructed frame is located is set as a pending reference identifier, and the status identifier of the cache module where the key reconstructed frame is located is set as a writable identifier.

[0050] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor, a memory and a bus, wherein the processor and the memory are communicatively connected via the bus; the memory stores computer program instructions executable by the processor, and the processor is configured to implement the video encoding method described in the first aspect.

[0051] It can be seen from the above technical solutions that the present application provides a video encoding method, device and electronic device, the method comprising: obtaining a frame sequence and a cache module. Generate a key frame code stream and a key reconstruction frame according to the key frame; store the key reconstruction frame in the first cache module, and set a reference identifier for the first cache module; generate a forward prediction reconstruction frame based on the key reconstruction frame, and store the forward prediction reconstruction frame in the second cache module; set a reference identifier for the second cache module, and set a writable identifier for the first cache module; perform traversal on the frame sequence to obtain the current frame, and obtain the reference reconstruction frame stored in the cache module under the reference identifier according to the type of the current frame to generate a reconstructed frame corresponding to the current frame; store the reconstructed frame in the cache module under the writable identifier and release the cache module where the reference reconstruction frame is located, and update the status identifier of the cache module. The above method reduces the memory occupancy time by timely releasing the memory of the first cache module and the second cache module. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0053] Figure 1 This is a schematic diagram of a frame sequence according to an embodiment of the present application;

[0054] Figure 2 A schematic diagram of video encoding based on three cache modules according to an embodiment of the present application;

[0055] Figure 3 A schematic diagram of the steps of the video encoding method according to an embodiment of the present application;

[0056] Figure 4 A schematic diagram of the steps of generating a key frame code stream according to an embodiment of the present application;

[0057] Figure 5 A schematic diagram of video encoding based on two cache modules according to an embodiment of the present application;

[0058] Figure 6 A schematic diagram of the steps of generating a reference reconstruction frame according to an embodiment of the present application;

[0059] Figure 7 A schematic diagram of the structure of a video encoding device according to an embodiment of the present application;

[0060] Figure 8 A schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0061] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0062] It should be understood that the "multiple" mentioned herein refers to two or more. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in order to facilitate the clear description of the technical solution of the embodiments of the present application, in the embodiments of the present application, the words "first", "second" and the like are used to distinguish between the same items or similar items with basically the same functions and effects. Those skilled in the art can understand that the words "first", "second" and the like do not limit the quantity and execution order, and the words "first", "second" and the like do not limit them to be necessarily different.

[0063] In addition, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus that includes a series of steps or elements is not necessarily limited to those steps or elements explicitly listed, but may include other steps or elements not explicitly listed or inherent to such process, method, product, or apparatus.

[0064] In network transmission, the size of video files affects the transmission rate and bandwidth consumption. Since video files contain a large amount of data, the transmission speed will be slow and the network delay will be increased. Therefore, the video files need to be encoded and compressed to reduce the space occupied by the video files, thereby speeding up the transmission speed and reducing network congestion.

[0065] In the process of video encoding, the video is decomposed into continuous images (i.e., video frames), and a set of continuous video frames can be grouped into a group of pictures (GOP). Among them, the picture group is the basic unit of video encoding and video transmission, and the picture group contains various types of video frames, such as Figure 1 As shown, video frames are divided into key frames (IDR frames), forward prediction frames (P frames) and virtual frames (VI frames) according to different encoding methods.

[0066] It should be noted that the first frame of a picture group is a key frame, and a picture group contains only one key frame. Key frames are encoded independently without reference to other frames. Key frames can be used as long-term reference frames. Forward prediction frames are encoded with reference to their previous frames. The previous frame of forward prediction frames can be key frames, forward prediction frames, and virtual frames. Forward prediction frames can be used as short-term reference frames to provide reference for the encoding of the following frame. Virtual frames are encoded only with reference to key frames. Virtual frames can be used as short-term reference frames to provide reference for the encoding of the following frame.

[0067] In some embodiments, the key frame is used as a long-term reference frame, and the generated reconstructed frame persists in the entire picture group period. When the virtual frame or forward prediction frame is not referenced, the memory of the cache module where it is located will be reused by the subsequent short-term reference frame, so the number of cache modules occupied by the picture group is 3. Figure 2 As shown, source image F1 is a key frame, source images F2 to F n-1 is the forward prediction frame, F n When F1 is encoded, the generated reconstructed frame is stored in the buffer module BUF1. n Reference, therefore, the cache module BUF1 is retained during the entire image group cycle. F2 is encoded with reference to F1, and the resulting reconstructed frame is stored in the cache module BUF2. F3 is encoded with reference to F2, and the resulting reconstructed frame is stored in the cache module BUF3. Since the reconstructed frame in BUF2 has been referenced by F3 and will no longer be referenced by other frames, when F4 is encoded with reference to F3, the resulting reconstructed frame can reuse the cache module BUF2 where the reconstructed frame of F2 is located. The above steps are repeated until the virtual frame F is encoded. n , F n By referencing the F1 cached in BUF1, F is generated. n Reconstruct the frame and reuse F n-2 The cache module BUF2 of the reconstructed frame. The video frame is encoded in the above manner, and the code stream is saved to the disk, or the code stream is sent through the network, so that the video encoding compression can be completed. Since the reconstructed frame corresponding to the key frame is stored in the cache module BUF1 for a long time during the above video encoding process in the image group, the system memory overhead is too large, which reduces the performance of video encoding.

[0068] In order to reduce the memory occupied by a video during encoding and improve the performance of video encoding, an embodiment of the present application provides a video encoding method. Figure 3 This is a schematic diagram of the steps of the video encoding method according to an embodiment of the present application. Figure 3 The method of this embodiment is described in detail.

[0069] Step S100: Acquire a frame sequence.

[0070] In some embodiments, the video frames are arranged in time order to form a plurality of image groups, wherein the image groups include key frames, forward prediction frames and virtual frames. Figure 1 , the frame sequence includes a key frame, at least one forward prediction frame and at least one virtual frame.

[0071] The first frame of the frame sequence is a key frame, which represents the beginning of a group of images, contains complete image information, and can be decoded and displayed independently without reference to other frames. The second frame of the frame sequence is a forward prediction frame, which needs to be decoded with reference to the previous frame and encoded through prediction and motion compensation to reduce the amount of data and improve compression efficiency. The subsequent frame types of the frame sequence are not limited in this embodiment. For example, the third frame of the frame sequence can be a forward prediction frame or a virtual frame. Among them, the forward prediction frame depends on the previous frame encoding, and the virtual frame depends on the key frame encoding.

[0072] Step S200: Obtain a cache module.

[0073] In some embodiments, the cache module is used to store reference reconstructed frames. In the process of video encoding, the compressed video data is received and the steps of entropy decoding, inverse quantization and inverse transformation are performed on the video data to restore the original image frame, i.e., the reconstructed frame. Among them, the reference reconstructed frame includes a key reconstructed frame, a forward predicted reconstructed frame and a virtual reconstructed frame. The key reconstructed frame is a reconstructed frame generated after the key frame code stream is decoded, the forward predicted reconstructed frame is a reconstructed frame generated after the forward predicted frame code stream is decoded, and the virtual reconstructed frame is a reconstructed frame generated after the virtual frame code stream is decoded.

[0074] The cache module includes a first cache module and a second cache module, and the cache module is provided with a status flag, and the status flag includes a pending reference flag and a writable flag. For example, the status flag of the first cache module is a pending reference flag, which indicates that the reference reconstructed frame stored in the first cache module will be referenced by subsequent video frames, and the reference reconstructed frame stored in the first cache module can be obtained when encoding subsequent video frames. For another example, the status flag of the second cache module is a writable flag, which indicates that the reference reconstructed frame stored in the second cache module is no longer referenced by subsequent video frames, and the data in the second cache module can be released and a new reconstructed frame can be stored.

[0075] Step S300: Generate a key frame code stream according to the key frame, and decode the key frame code stream.

[0076] In some embodiments, the key frame generates a key frame bitstream through intra-frame prediction. Figure 4As shown, the key frame is first divided based on a preset size to obtain a key frame macroblock including real pixel values; then the predicted pixel value of the key frame macroblock is predicted by the intra-frame prediction algorithm, and the intra-frame residual between the predicted pixel value and the real pixel value is calculated; then the intra-frame residual is transformed in the frequency domain to calculate the intra-frame transformation coefficient, and the intra-frame transformation coefficient is quantized to calculate the intra-frame quantization coefficient; finally, entropy coding is performed on the intra-frame quantization coefficient to generate a key frame code stream.

[0077] It should be noted that the encoder uses the spatial correlation within the image frame to reduce data redundancy by predicting and encoding the pixel values ​​within the current frame, thereby compressing the data. When performing intra-frame coding on key frames, the encoder divides the key frame into key frame macroblocks and predicts the pixel values ​​of the key frame macroblocks based on the encoded pixel values ​​around the key frame macroblocks. It also calculates the intra-frame residual between the predicted pixel value and the actual pixel value to achieve compression.

[0078] Furthermore, the encoder generates a key frame bitstream by performing transformation, quantization and entropy coding operations on the intra-frame residual. The energy in the key frame is concentrated on the low-frequency coefficients through discrete cosine transform (DCT), and some high-frequency coefficients are discarded to obtain the intra-frame transform coefficients to reduce the amount of data. Then the intra-frame transform coefficients are discretized and mapped to a set of values ​​according to a preset quantization step size to calculate the intra-frame quantization coefficients. Finally, the intra-frame quantization coefficients are further compressed using the statistical characteristics of the signal source to generate a key frame bitstream, and the key frame bitstream is decoded using a decoder to generate a key reconstruction frame.

[0079] Step S400: storing the key reconstructed frame in a first cache module, and generating a forward prediction reconstructed frame based on the key reconstructed frame in the first cache module.

[0080] For example, Figure 5 As shown, the source image F1 is a key frame, intra-frame coding is performed on the key frame to generate a key reconstruction frame, and the key reconstruction frame is stored in the first buffer module BUF1 as a reference reconstruction frame of the source image F2. The source image F2 obtains the key reconstruction frame from the first buffer module BUF1, and generates a forward prediction reconstruction frame with reference to the key reconstruction frame.

[0081] Step S500: store the forward prediction reconstructed frame into the second cache module, set the state flag of the second cache module to a pending reference flag, and set the state flag of the first cache module to a writable flag.

[0082] See again Figure 5, the forward prediction reconstructed frame corresponding to the source image F2 is stored in the second cache module BUF2 as a reference reconstructed frame of the source image F3. The state flag of the second cache module BUF2 is set to a pending reference flag to indicate that the cached content in the second cache module BUF2 will be referenced by subsequent frames. The state flag of the first cache module BUF1 is set to a writable flag to indicate that the cached content in the first cache module is no longer referenced by subsequent frames, and the first cache module BUF1 can be reused. At this time, the first cache module BUF1 releases memory space to reduce the memory occupancy time.

[0083] Step S600: traverse the frame sequence starting from the third frame of the frame sequence to obtain the current frame.

[0084] Since different frame sequences correspond to different video frame arrangement orders, it is necessary to traverse the frame sequence starting from the third frame of the frame sequence and determine the type and position of the current frame, so as to perform different steps on the current frame according to the type and position of the current frame.

[0085] In some embodiments, the current frame is a forward predicted frame and the next frame after the current frame is a forward predicted frame. At this time, a reference reconstructed frame stored in a cache module whose status identifier is a to-be-referenced identifier is obtained (S611); a forward predicted reconstructed frame corresponding to the current frame is generated based on the reference reconstructed frame, and the forward predicted reconstructed frame is stored in a cache module whose status identifier is a writable identifier (S612); and the status identifier of the cache module where the reference reconstructed frame is located is set to a writable identifier, and the status identifier of the cache module where the forward predicted reconstructed frame corresponding to the current frame is located is set to a to-be-referenced identifier (S613).

[0086] For example, see Figure 5 , the current frame is the forward prediction frame F3, the current state flag of the first cache module BUF1 is a writable flag, the state flag of the second cache module BUF2 is a pending reference flag, the reference reconstructed frame corresponding to F2 stored in the second cache module BUF2 is obtained, and the forward prediction reconstructed frame corresponding to F3 is generated according to the reference reconstructed frame corresponding to F2, and the forward prediction reconstructed frame corresponding to F3 is stored in the first cache module BUF1. Finally, the state flag of the cache module is updated. Since the reference reconstructed frame corresponding to F2 stored in the second cache module BUF2 is no longer referenced by subsequent video frames, the state flag of the second cache module BUF2 is set to a writable flag. Since the forward prediction reconstructed frame corresponding to F3 stored in the first cache module BUF1 will be referenced by F4 as a reference reconstructed frame, the state flag of the first cache module BUF1 is set to a pending reference flag.

[0087] In some embodiments, a forward prediction reconstructed frame corresponding to the current frame can be generated based on a reference reconstructed frame based on an inter-frame coding algorithm. In the process of generating a forward prediction reconstructed frame corresponding to the current frame, the forward prediction frame is first divided based on a preset size to obtain a forward prediction frame macroblock. For example, the forward prediction frame macroblock can be a 16×16 pixel block. Then, a forward matching macroblock of the forward prediction frame macroblock is queried from the reference reconstructed frame, and a forward motion vector between the forward matching macroblock and the forward prediction frame macroblock is calculated.

[0088] Further, according to the forward matching macroblock and the forward motion vector, a forward matching prediction macroblock is generated, and the forward inter-frame residual between the forward prediction frame macroblock and the forward matching prediction macroblock is calculated. The forward inter-frame residual is used to characterize the difference between the forward prediction frame macroblock and the forward matching prediction macroblock. Then, a frequency domain transform is performed on the forward inter-frame residual to calculate the forward inter-frame transform coefficient, and the forward inter-frame transform coefficient is quantized to calculate the forward inter-frame quantization coefficient. And entropy coding is performed on the forward inter-frame quantization coefficient to generate a forward prediction frame code stream. Finally, the forward prediction frame code stream is decoded to generate a forward prediction reconstructed frame.

[0089] The above inter-frame coding algorithm can utilize the temporal correlation between adjacent frames to predict a forward prediction reconstructed frame based on a reference reconstructed frame, thereby removing redundant information in the frame sequence and improving compression efficiency.

[0090] In some embodiments, the current frame is a forward predicted frame and the frame after the current frame is a virtual frame, or the current frame is a forward predicted frame and the current frame is the last frame of the frame sequence. At this time, a reference reconstructed frame stored in a cache module whose status identifier is a reference identifier is obtained (S621), and a forward predicted reconstructed frame corresponding to the current frame is generated based on the reference reconstructed frame (S622), and the status identifier of the cache module where the reference reconstructed frame is located is set to a writable identifier (S623).

[0091] For example, see Figure 5 , the current frame is F n-1 , the current state flag of the first buffer module BUF1 is a writable flag, the state flag of the second buffer module BUF2 is a pending reference flag, and the reference reconstructed frame (i.e., F) stored in the second buffer module BUF2 is obtained. n-2 corresponding reference reconstructed frame), and according to F n-2 The corresponding reference reconstructed frame generates F n-1 The corresponding forward prediction reconstructed frame. n-2 The corresponding reference reconstructed frame is no longer referenced by subsequent video frames, so the state flag of the second buffer module BUF2 is set to a writable flag. n-1The corresponding reference reconstructed frame is no longer referenced by subsequent video frames, so there is no need to store the F in the first buffer module BUF1. n-1 The corresponding forward prediction reconstructs the frame, thereby reducing the memory bandwidth of the system. At this time, the state flag of the first buffer module BUF1 has not been updated and is still a writable flag.

[0092] In some embodiments, the current frame is a forward predicted frame and the frame after the current frame is a virtual frame, or the current frame is a forward predicted frame and the current frame is the last frame of the frame sequence. Since the current frame is no longer referenced by subsequent video frames, there is no need to generate a forward predicted reconstructed frame corresponding to the current frame, thereby reducing the system memory bandwidth and improving video encoding efficiency.

[0093] In some embodiments, the current frame is a virtual frame, and the key frame code stream is obtained at this time, and the key frame code stream is decoded again to generate a key reconstruction frame (S631). The key reconstruction frame is then stored in a cache module whose status is marked as a writable identifier, and the status identifier of the cache module where the key reconstruction frame is located is set as a pending reference identifier (S632). And a virtual reconstruction frame corresponding to the current frame is generated according to the key reconstruction frame, and the virtual reconstruction frame is stored in a cache module whose status is marked as a writable identifier (S633). Finally, the status identifier of the cache module where the virtual reconstruction frame is located is set as a pending reference identifier, and the status identifier of the cache module where the key reconstruction frame is located is set as a writable identifier (S634).

[0094] For example, see Figure 5 , the current frame is the virtual frame F n , the current state flags of the first buffer module BUF1 and the second buffer module BUF2 are both writable flags, the key frame code stream is obtained and decoded, the decoded key reconstructed frame is stored in the first buffer module BUF1, and the state flag of the first buffer module BUF1 is set to the reference flag, and F is generated according to the key reconstructed frame n The corresponding virtual reconstruction frame and F n The corresponding virtual reconstructed frame is stored in the second buffer module BUF2. Since the key reconstructed frame stored in the first buffer module BUF1 is no longer referenced by other frames, the state flag of the first buffer module BUF1 is set to a writable flag. n The corresponding virtual reconstructed frame will be used as a reference reconstructed frame for reference by subsequent video frames, and therefore, the state flag of the second buffer module BUF2 is set to a pending reference flag.

[0095] In some embodiments, an inter-frame coding algorithm may be used to generate a virtual reconstructed frame corresponding to the current frame based on a key reconstructed frame. In the process of generating a virtual reconstructed frame, the virtual frame is first divided based on a preset size to obtain a virtual frame macroblock. For example, the virtual frame macroblock may be a 16×16 pixel block. Then, a virtual matching macroblock of the virtual frame macroblock is queried from the key reconstructed frame, and a virtual motion vector between the virtual matching macroblock and the virtual frame macroblock is calculated.

[0096] Further, a virtual frame prediction macroblock is generated according to the virtual matching macroblock and the virtual motion vector, and a virtual inter-frame residual between the virtual frame macroblock and the virtual frame prediction macroblock is calculated. The virtual inter-frame residual is used to characterize the difference between the virtual frame macroblock and the virtual frame prediction macroblock. Next, a frequency domain transform is performed on the virtual inter-frame residual to calculate a virtual inter-frame transform coefficient, and the virtual inter-frame transform coefficient is quantized to calculate a virtual inter-frame quantization coefficient; and entropy coding is performed on the virtual inter-frame quantization coefficient to generate a virtual frame code stream. Finally, the virtual frame code stream is decoded to generate a virtual reconstructed frame.

[0097] In some embodiments, a coding sequence is generated according to the coding bit stream, thereby achieving coding compression of the video. Figure 6 As shown, firstly, a coding stream is obtained, and the coding stream includes a key frame stream, a forward prediction frame stream and a virtual frame stream. Then, a coding sequence is generated based on the arrangement order of the frame sequence, and the coding sequence includes the coding stream.

[0098] In some embodiments, a reference reconstructed frame can be generated by performing a decoding operation on a coded bitstream. In the process of decoding the coded bitstream, the coded bitstream is first entropy decoded to extract parameter information such as quantization coefficients, coding modes, motion vectors, etc. from the compressed bitstream. The entropy decoded data is then reordered to restore its order before encoding. Then, the reordered data is dequantized using the quantization parameters and quantization coefficients to restore the residual information, and the dequantized data is inversely transformed to restore the original spatial data, and the residual information is converted from the frequency domain back to the spatial domain, thereby obtaining a reconstructed residual signal. Finally, a prediction module in the decoder is used to generate a predicted macroblock, and the predicted macroblock is added to the reconstructed residual signal to generate a reference reconstructed frame.

[0099] In some embodiments, since the block effects and compression noise in the reference reconstructed frame may reduce the video quality, loop filtering may be performed on the reference reconstructed frame to smooth the block boundaries and remove the compression noise, thereby optimizing the reference reconstructed frame and improving the quality of video encoding.

[0100] It can be seen from the above technical solution that the present application provides a video encoding method, which includes: obtaining a frame sequence and a cache module. Generate a key frame code stream and a key reconstruction frame according to the key frame; store the key reconstruction frame in the first cache module, and set a reference identifier for the first cache module; generate a forward prediction reconstruction frame based on the key reconstruction frame, and store the forward prediction reconstruction frame in the second cache module; set a reference identifier for the second cache module, and set a writable identifier for the first cache module; perform traversal on the frame sequence to obtain the current frame, and obtain the reference reconstruction frame stored in the cache module under the reference identifier according to the type of the current frame to generate a reconstructed frame corresponding to the current frame; store the reconstructed frame in the cache module under the writable identifier and release the cache module where the reference reconstruction frame is located, and update the status identifier of the cache module. The above method reduces the memory occupancy time by timely releasing the memory of the first cache module and the second cache module.

[0101] The method embodiment according to the present application is described above by way of example. Based on the video encoding method provided in the embodiment of the present application, the embodiment of the present application provides a video encoding device. Figure 7 A schematic diagram of the structure of a video encoding device provided in an embodiment of the present application. Figure 7 , the device includes the following modules.

[0102] The image preprocessing module 710 is configured to obtain a frame sequence, which includes a key frame, at least one forward prediction frame and at least one virtual frame. The first frame of the frame sequence is the key frame, and the second frame of the frame sequence is the forward prediction frame.

[0103] The cache acquisition module 720 is configured to acquire a cache module, the cache module is used to store reference reconstructed frames, the reference reconstructed frames include key reconstructed frames, forward predicted reconstructed frames and virtual reconstructed frames, the cache module includes a first cache module and a second cache module, the cache module is provided with a state flag, the state flag includes a pending reference flag and a writable flag;

[0104] The video encoding processing module 730 is configured to generate a key frame code stream based on the key frame, and decode the key frame code stream to generate a key reconstruction frame; store the key reconstruction frame in the first cache module and generate a forward prediction reconstructed frame based on the key reconstruction frame in the first cache module; store the forward prediction reconstructed frame in the second cache module, and set the state flag of the second cache module to a pending reference flag, and set the state flag of the first cache module to a writable flag; traverse the frame sequence starting from the third frame of the frame sequence to obtain the current frame.

[0105] The processing core selection module 740 is configured to send a scheduling instruction to the video encoding processing module 730 based on the type of the current frame, where the scheduling instruction is used to instruct the video encoding processing module 730 to generate a reference reconstructed frame and store the reference reconstructed frame.

[0106] The encoder post-processing module 750 is configured to obtain a coded bitstream, which includes a key frame bitstream, a forward prediction frame bitstream and a virtual frame bitstream; generate a coded sequence based on the arrangement order of the frame sequence, which includes the coded bitstream; decode the coded bitstream to generate a reference reconstructed frame; and perform loop filtering on the reference reconstructed frame to optimize the reference reconstructed frame.

[0107] In some embodiments, the video encoding processing module 730 is further configured to: if the current frame is a forward predicted frame and the next frame after the current frame is a forward predicted frame, obtain a reference reconstructed frame stored in a cache module whose status identifier is a pending reference identifier; generate a forward predicted reconstructed frame corresponding to the current frame according to the reference reconstructed frame, and store the forward predicted reconstructed frame in a cache module whose status identifier is a writable identifier; set the status identifier of the cache module where the reference reconstructed frame is located to a writable identifier, and set the status identifier of the cache module where the forward predicted reconstructed frame corresponding to the current frame is located to a pending reference identifier;

[0108] If the current frame is a forward predicted frame and the frame after the current frame is a virtual frame, or the current frame is a forward predicted frame and the current frame is the last frame of the frame sequence, then obtain a reference reconstructed frame stored in a cache module whose status identifier is a reference identifier; generate a forward predicted reconstructed frame corresponding to the current frame according to the reference reconstructed frame; set the status identifier of the cache module where the reference reconstructed frame is located to a writable identifier;

[0109] If the current frame is a virtual frame, obtain the key frame code stream, and decode the key frame code stream again to generate a key reconstructed frame; store the key reconstructed frame in a cache module whose status is marked as a writable flag, and set the status flag of the cache module where the key reconstructed frame is located as a pending reference flag; generate a virtual reconstructed frame corresponding to the current frame according to the key reconstructed frame, and store the virtual reconstructed frame in a cache module whose status is marked as a writable flag; set the status flag of the cache module where the virtual reconstructed frame is located as a pending reference flag, and set the status flag of the cache module where the key reconstructed frame is located as a writable flag.

[0110] The above describes the device embodiments of the present application. For detailed descriptions of the specific execution processes of data, terms, nouns, steps, technical issues and effects, alternative methods and combinations, please refer to the descriptions in the method embodiments, which will not be repeated here.

[0111] The embodiment of the present application further provides an electronic device, comprising: a processor, a memory and a bus, wherein the processor and the memory are communicatively connected via the bus, the memory stores computer program instructions executable by the processor, and the processor is configured to implement the steps of the video encoding method described in any one of the first aspects.

[0112] For example, Figure 8As shown, the electronic device includes a processor 810, at least one communication bus 820, a user interface 830, at least one external communication interface 840 and a memory 850. The communication bus 820 is configured to realize the connection and communication between these components. The user interface 830 may include a display screen, and the external communication interface 840 may include a standard wired interface and a wireless interface. The memory 850 stores a computer program. The processor 810 is used to execute the computer program stored in the memory 850.

[0113] Similar parts between the embodiments provided in this application can be referenced to each other. The specific implementation methods provided above are only a few examples under the general concept of this application and do not constitute a limitation on the protection scope of this application. For those skilled in the art, any other implementation methods expanded based on the scheme of this application without creative work belong to the protection scope of this application.

Claims

1. A video encoding method, characterized in that: include: Acquire a frame sequence, wherein the frame sequence includes a key frame, at least one forward prediction frame and at least one virtual frame, the first frame of the frame sequence is the key frame, and the second frame of the frame sequence is the forward prediction frame; Acquire a cache module, the cache module is used to store reference reconstructed frames, the reference reconstructed frames include key reconstructed frames, forward predicted reconstructed frames and virtual reconstructed frames, the cache module includes a first cache module and a second cache module, the cache module is provided with a state flag, the state flag includes a to-be-referenced flag and a writable flag; Generating a key frame code stream according to the key frame, and decoding the key frame code stream to generate a key reconstruction frame; Storing the key reconstructed frame in the first cache module, and generating a forward prediction reconstructed frame based on the key reconstructed frame in the first cache module; storing the forward prediction reconstructed frame into the second cache module, setting the state flag of the second cache module to a pending reference flag, and setting the state flag of the first cache module to a writable flag; Perform traversal on the frame sequence starting from the third frame of the frame sequence to obtain a current frame; If the current frame is a forward predicted frame and a frame subsequent to the current frame is a forward predicted frame, obtaining a reference reconstructed frame stored in a cache module whose state identifier is a to-be-referenced identifier; Generating a forward prediction reconstructed frame corresponding to the current frame according to the reference reconstructed frame, and storing the forward prediction reconstructed frame in a cache module whose status is marked as a writable flag; The state flag of the cache module where the reference reconstructed frame is located is set to a writable flag, and the state flag of the cache module where the forward prediction reconstructed frame corresponding to the current frame is located is set to a pending reference flag.

2. The video encoding method according to claim 1, characterized in that The video encoding method further comprises: If the current frame is a forward predicted frame and the frame after the current frame is a virtual frame, or the current frame is a forward predicted frame and the current frame is the last frame of the frame sequence, then obtaining a reference reconstructed frame stored in a cache module whose status identifier is a pending reference identifier; Generate a forward prediction reconstructed frame corresponding to the current frame according to the reference reconstructed frame; The state flag of the cache module where the reference reconstructed frame is located is set to a writable flag.

3. The video encoding method according to claim 1, characterized in that: Also includes: If the current frame is a virtual frame, the key frame code stream is obtained, and the key frame code stream is decoded again to generate the key reconstructed frame; The key reconstructed frame is stored in a cache module whose status flag is a writable flag, and the status flag of the cache module where the key reconstructed frame is located is set as a pending reference flag; Generate a virtual reconstructed frame corresponding to the current frame according to the key reconstructed frame, and store the virtual reconstructed frame in a cache module whose status is marked as a writable flag; The state flag of the cache module where the virtual reconstructed frame is located is set as a pending reference flag, and the state flag of the cache module where the key reconstructed frame is located is set as a writable flag.

4. The video encoding method according to claim 1, characterized in that: The step of generating a key frame code stream according to the key frames includes: Segment the key frame based on a preset size to obtain key frame macroblocks, wherein the key frame macroblocks include real pixel values; Predicting the predicted pixel value of the key frame macroblock by using an intra-frame prediction algorithm, and calculating the intra-frame residual between the predicted pixel value and the true pixel value; Performing frequency domain transformation on the intra-frame residual to calculate intra-frame transformation coefficients, and performing quantization on the intra-frame transformation coefficients to calculate intra-frame quantization coefficients; Entropy coding is performed on the intra-frame quantization coefficients to generate a key frame code stream.

5. The video encoding method according to claim 4, characterized in that: The step of generating a forward prediction reconstructed frame corresponding to the current frame according to the reference reconstructed frame includes: Splitting the forward prediction frame based on a preset size to obtain forward prediction frame macroblocks; Query the forward matching macroblock of the forward prediction frame macroblock from the reference reconstructed frame, and calculate the forward motion vector between the forward matching macroblock and the forward prediction frame macroblock; Generate a forward matching prediction macroblock according to the forward matching macroblock and the forward motion vector, and calculate a forward inter-frame residual between the forward prediction frame macroblock and the forward matching prediction macroblock; Performing frequency domain transformation on the forward inter-frame residual to calculate forward inter-frame transformation coefficients, and performing quantization on the forward inter-frame transformation coefficients to calculate forward inter-frame quantization coefficients; Performing entropy coding on the forward inter-frame quantization coefficients to generate a forward prediction frame code stream; The forward prediction frame code stream is decoded to generate a forward prediction reconstructed frame.

6. The video encoding method according to claim 5, characterized in that: The step of generating a virtual reconstructed frame corresponding to the current frame according to the key reconstructed frame includes: Divide the virtual frame based on a preset size to obtain virtual frame macroblocks; Querying a virtual matching macroblock of the virtual frame macroblock from the key reconstructed frame, and calculating a virtual motion vector between the virtual matching macroblock and the virtual frame macroblock; Generate a virtual frame prediction macroblock according to the virtual matching macroblock and the virtual motion vector, and calculate a virtual inter-frame residual between the virtual frame macroblock and the virtual frame prediction macroblock; Performing frequency domain transformation on the virtual inter-frame residual to calculate virtual inter-frame transformation coefficients, and performing quantization on the virtual inter-frame transformation coefficients to calculate virtual inter-frame quantization coefficients; Performing entropy coding on the virtual inter-frame quantization coefficients to generate a virtual frame code stream; The virtual frame code stream is decoded to generate a virtual reconstructed frame.

7. The video encoding method according to claim 6, characterized in that: Also includes: Obtaining a coded bitstream, wherein the coded bitstream includes a key frame bitstream, a forward prediction frame bitstream and a virtual frame bitstream; Generate a coding sequence based on the arrangement order of the frame sequence, the coding sequence including the coding bit stream; Decoding the encoded bitstream to generate the reference reconstructed frame; Loop filtering is performed on the reference reconstructed frame to optimize the reference reconstructed frame.

8. A video encoding device, characterized in that: include: The image preprocessing module is configured to: acquire a frame sequence, wherein the frame sequence includes a key frame, at least one forward prediction frame and at least one virtual frame, wherein the first frame of the frame sequence is the key frame, and the second frame of the frame sequence is the forward prediction frame; A cache acquisition module is configured to: acquire a cache module, the cache module is used to store reference reconstructed frames, the reference reconstructed frames include key reconstructed frames, forward predicted reconstructed frames and virtual reconstructed frames, the cache module includes a first cache module and a second cache module, the cache module is provided with a state flag, the state flag includes a to-be-referenced flag and a writable flag; The video encoding processing module is configured to: generate a key frame code stream according to the key frame, and decode the key frame code stream to generate a key reconstruction frame; store the key reconstruction frame in the first cache module and generate a forward prediction reconstruction frame based on the key reconstruction frame in the first cache module; store the forward prediction reconstruction frame in the second cache module, and set the state flag of the second cache module to a pending reference flag, and set the state flag of the first cache module to a writable flag; Perform traversal on the frame sequence starting from the third frame of the frame sequence to obtain a current frame; A processing core selection module is configured to: send a scheduling instruction to the video encoding processing module based on the type of the current frame, wherein the scheduling instruction is used to instruct the video encoding processing module to generate a reference reconstructed frame and store the reference reconstructed frame; The encoder post-processing module is configured to: obtain a coded stream, the coded stream includes a key frame stream, a forward prediction frame stream and a virtual frame stream; generate a coded sequence based on the arrangement order of the frame sequence, the coded sequence includes the coded stream; decode the coded stream to generate the reference reconstructed frame; Loop filtering is performed on the reference reconstructed frame to optimize the reference reconstructed frame.

9. The video encoding device according to claim 8, characterized in that: The video encoding processing module is also configured to: If the current frame is a forward predicted frame and the next frame after the current frame is a forward predicted frame, a reference reconstructed frame stored in a cache module whose status identifier is a to-be-referenced identifier is obtained; a forward predicted reconstructed frame corresponding to the current frame is generated according to the reference reconstructed frame, and the forward predicted reconstructed frame is stored in a cache module whose status identifier is a writable identifier; the status identifier of the cache module where the reference reconstructed frame is located is set to a writable identifier, and the status identifier of the cache module where the forward predicted reconstructed frame corresponding to the current frame is located is set to a to-be-referenced identifier; If the current frame is a forward predicted frame and the frame after the current frame is a virtual frame, or the current frame is a forward predicted frame and the current frame is the last frame of the frame sequence, then obtaining a reference reconstructed frame stored in a cache module whose status identifier is a reference identifier; generating a forward predicted reconstructed frame corresponding to the current frame according to the reference reconstructed frame; and setting the status identifier of the cache module where the reference reconstructed frame is located to a writable identifier; If the current frame is a virtual frame, the key frame code stream is obtained, and the key frame code stream is decoded again to generate the key reconstructed frame; the key reconstructed frame is stored in a cache module whose status identifier is a writable identifier, and the status identifier of the cache module where the key reconstructed frame is located is set as a pending reference identifier; a virtual reconstructed frame corresponding to the current frame is generated according to the key reconstructed frame, and the virtual reconstructed frame is stored in a cache module whose status identifier is a writable identifier; the status identifier of the cache module where the virtual reconstructed frame is located is set as a pending reference identifier, and the status identifier of the cache module where the key reconstructed frame is located is set as a writable identifier.

10. An electronic device, characterized in that: Includes processor, memory and bus; The processor and the memory are communicatively connected via the bus; The memory stores computer program instructions executable by the processor, and the processor is configured to implement the video encoding method according to any one of claims 1-7.

Images