Video encoding method, video decoding method, encoder, decoder, and medium

Abstract

This application proposes a video encoding method, a video decoding method, a video encoder, a video decoder, and a computer storage medium based on knowledge images. The video encoding method includes: obtaining a first reference image queue configuration set for the current video frame; obtaining a reference frame index for the current video frame based on the first reference image queue configuration set; obtaining a reference knowledge image frame from a buffer according to the reference frame index; and encoding the current video frame according to the reference knowledge image frame to obtain the encoded bitstream of the current video frame. Through this video encoding method, the reference frame for video frame encoding is optimized from only being able to use the nearest knowledge image frame as the reference frame to determining the reference knowledge image frame according to the reference frame index of the video frame. This expands the multi-frame management mechanism of knowledge images, better adapts to changing scene transitions, and improves video encoding effect and efficiency.

Description

Technical Field

[0001] This application relates to the field of video encoding and decoding technology, and in particular to a video encoding method, a video decoding method, a video encoder, a video decoder, and a computer storage medium based on knowledge images. Background Technology

[0002] In video encoding and decoding, to improve compression ratio and reduce the number of codewords to be transmitted, the encoder does not directly encode and transmit pixel values. Instead, it uses intra-frame or inter-frame prediction modes to predict the pixel values ​​of the current block using reconstructed pixels from the coded blocks of the current frame or a reference frame. The pixel value predicted using a certain prediction mode is called the predicted pixel value, and the difference between the predicted pixel value and the original pixel value is called the residual. The encoder only needs to encode a certain prediction mode and the residual generated when using that prediction mode, and the decoder can decode the corresponding pixel value based on this bitstream information. This greatly reduces the number of codewords required for encoding.

[0003] The existing SVAC3 video codec standard introduces the concept of a knowledge picture (library picture), with L-frames in subsequent proposals representing knowledge pictures. Simultaneously, the concept of RL (reference library) frames is introduced; RL frames refer to P-frames or B-frames that reference only the knowledge picture. A knowledge picture is a long-term reference frame encoded using I-frames. It serves only as a reference frame and is not used for display. Knowledge pictures are identified by their knowledge picture index (IDX), rather than the POC or DOI of other frames in the bitstream.

[0004] However, existing knowledge images lack a multi-frame management mechanism, resulting in a maximum of only one knowledge image available for reference, which significantly reduces video encoding quality and efficiency. Summary of the Invention

[0005] To address the aforementioned technical problems, this application proposes a video encoding method, a video decoding method, a video encoder, a video decoder, and a computer storage medium based on knowledge images.

[0006] To address the aforementioned technical problems, this application proposes a video coding method based on knowledge images, the video coding method comprising:

[0007] Get the first reference image queue configuration set for the current video frame;

[0008] The reference frame index of the current video frame is obtained based on the first reference image queue configuration set;

[0009] Retrieve reference knowledge image frames from the cache based on the reference frame index;

[0010] The current video frame is encoded according to the reference knowledge image frame to obtain the encoded bitstream of the current video frame.

[0011] The video encoding method further includes, prior to encoding the current video frame according to the reference knowledge image frame, the following:

[0012] Based on the first reference image queue configuration set of the current video frame, obtain the knowledge image frame that is referenced by the current video frame and subsequent video frames to be encoded.

[0013] Update the cache to save the knowledge image frames that are referenced by the current video frame and subsequent video frames to be encoded, and delete the knowledge image frames that are not referenced by the current video frame and subsequent video frames to be encoded.

[0014] The knowledge image frames preceding or following the current video frame do not include the reference image queue configuration set.

[0015] The video encoding method further includes:

[0016] Obtain a second reference image queue configuration set of knowledge image frames to be encoded before or after the current video frame, and obtain the index of the reserved knowledge image frame from the second reference image queue configuration set;

[0017] Based on the retained knowledge image frame index, the corresponding knowledge image frame is retained in the cache, and knowledge image frames other than the retained knowledge image frame index are deleted from the cache.

[0018] After encoding the knowledge image frame to be encoded, the encoding result is stored in the buffer area.

[0019] The first reference image queue configuration set of the current video frame does not include the index of the reference knowledge image frame;

[0020] The step of retrieving the reference knowledge image frame from the cache based on the reference frame index includes:

[0021] When the reference frame index does not include the index of the reference knowledge image frame, the latest saved knowledge image frame is retrieved from the cache and used as the reference knowledge image frame for the current video frame.

[0022] The step of obtaining the reference frame index of the current video frame based on the first reference image queue configuration set includes:

[0023] The number of reference knowledge image frames is obtained according to the reference syntax of the current video frame;

[0024] According to the number of reference knowledge image frames, obtain the index of the reference knowledge image frame that is sorted first in the first reference image queue configuration set;

[0025] The step of encoding the current video frame according to the reference knowledge image frame to obtain the encoded bitstream of the current video frame includes:

[0026] The number of reference non-knowledge image frames is obtained according to the reference syntax of the current video frame;

[0027] Based on the number of reference non-knowledge image frames, obtain the index of the reference non-knowledge image frame that is sorted first in the first reference image queue configuration set;

[0028] Obtain the corresponding reference non-knowledge image frame according to the reference non-knowledge image frame index;

[0029] The current video frame is encoded according to the reference knowledge image frame and the reference non-knowledge image frame to obtain the encoded bitstream of the current video frame.

[0030] The step of obtaining the reference frame index of the current video frame based on the first reference image queue configuration set includes:

[0031] The number of reference knowledge image frames is obtained according to the first reference syntax of the current video frame;

[0032] According to the number of reference knowledge image frames, obtain the index of the reference knowledge image frame that is sorted first in the first reference image queue configuration set;

[0033] The step of encoding the current video frame according to the reference knowledge image frame to obtain the encoded bitstream of the current video frame includes:

[0034] The number of reference non-knowledge image frames is obtained according to the second reference syntax of the current video frame;

[0035] Based on the number of reference non-knowledge image frames, obtain the index of the reference non-knowledge image frame that is sorted first in the first reference image queue configuration set;

[0036] Obtain the corresponding reference non-knowledge image frame according to the reference non-knowledge image frame index;

[0037] The current video frame is encoded according to the reference knowledge image frame and the reference non-knowledge image frame to obtain the encoded bitstream of the current video frame.

[0038] The video encoding method further includes:

[0039] Obtain the knowledge image frames to be encoded before or after the current video frame;

[0040] After encoding the knowledge image frame to be encoded, the encoding result is stored in the buffer area, and the encoding index is written into the image header syntax of the encoded knowledge image frame.

[0041] The video encoding method further includes:

[0042] Obtain the frame type syntax of the knowledge image frames to be encoded before or after the current video frame;

[0043] When the frame type syntax is started, the encoded knowledge image frames in the buffer are cleared;

[0044] After encoding the knowledge image frame to be encoded, the encoding result is stored in the buffer area, and the index is renumbered. The renumbered encoding index is then written into the image header syntax of the encoded knowledge image frame.

[0045] The reference frame index includes a period value and a current period index;

[0046] The step of retrieving the reference knowledge image frame from the cache based on the reference frame index includes:

[0047] Obtain the true index of the reference frame according to the cycle value and the current cycle index;

[0048] The reference knowledge image frame is obtained from the cache based on the reference frame real index.

[0049] To address the aforementioned technical problems, this application proposes a video decoding method based on knowledge images, the video decoding method comprising:

[0050] Extract a reference image queue configuration set from the video stream;

[0051] The reference frame index of the video stream is obtained based on the reference image queue configuration set;

[0052] Decoded reference knowledge image frames are retrieved from the cache based on the reference frame index;

[0053] The video stream is decoded according to the reference knowledge image frames to obtain the reconstructed video frames of the video stream.

[0054] To address the aforementioned technical problems, this application also proposes a video encoder, which includes a memory and a processor coupled to the memory;

[0055] The memory is used to store program data, and the processor is used to execute the program data to implement the video encoding method described above.

[0056] To address the aforementioned technical problems, this application also proposes a video decoder, which includes a memory and a processor coupled to the memory;

[0057] The memory is used to store program data, and the processor is used to execute the program data to implement the video decoding method described above.

[0058] To address the aforementioned technical problems, this application also proposes a computer storage medium for storing program data, which, when executed by a computer, is used to implement the aforementioned video encoding method and / or video decoding method.

[0059] Compared with existing technologies, the beneficial effects of this application are as follows: the video encoder obtains a first reference image queue configuration set for the current video frame; obtains a reference frame index for the current video frame based on the first reference image queue configuration set; obtains a reference knowledge image frame from the buffer according to the reference frame index; and encodes the current video frame according to the reference knowledge image frame to obtain the encoded bitstream of the current video frame. Through the above video encoding method, the reference frame for video frame encoding is optimized from only being able to use the nearest knowledge image frame as the reference frame to determining the reference knowledge image frame according to the reference frame index of the video frame. This expands the multi-frame management mechanism of knowledge images, better adapts to changing scene switching, and improves video encoding effect and encoding efficiency. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0061] in:

[0062] Figure 1 This is a schematic diagram of an embodiment of the frame reference relationship in a bitstream containing I-frames and P-frames provided in this application;

[0063] Figure 2 This is a schematic diagram of an embodiment of the frame reference relationship in a bitstream containing I-frames, P-frames and B-frames provided in this application;

[0064] Figure 3 This is a schematic diagram of another embodiment of the frame reference relationship in a bitstream containing I-frames and P-frames provided in this application;

[0065] Figure 4 This is a schematic diagram of the frame reference relationship and the position of the knowledge image in the bitstream under the IPPP configuration in the existing technology;

[0066] Figure 5 This is a flowchart illustrating an embodiment of the video encoding method provided in this application;

[0067] Figure 6 This is a schematic diagram of an embodiment of the frame reference relationship and the position of the knowledge image in the bitstream provided in this application;

[0068] Figure 7 This is a schematic diagram of the LDPB update process during the encoding or decoding process provided in this application;

[0069] Figure 8 This is a schematic diagram of another embodiment of the frame reference relationship and the position of the knowledge image in the bitstream provided in this application;

[0070] Figure 9 yes Figure 5 The following is a schematic diagram of the specific process of step S14 in the video encoding method.

[0071] Figure 10 This is a flowchart illustrating another embodiment of the video encoding method provided in this application;

[0072] Figure 11 This is a flowchart illustrating an embodiment of the video decoding method provided in this application;

[0073] Figure 12 This is a schematic diagram of the structure of an embodiment of the video encoder provided in this application;

[0074] Figure 13 This is a schematic diagram of the structure of an embodiment of the video decoder provided in this application;

[0075] Figure 14 This is a schematic diagram of the structure of an embodiment of the computer storage medium provided in this application. Detailed Implementation

[0076] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0077] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0078] Video image data is relatively large, so it is usually necessary to compress the video pixel data (RGB, YUV, etc.). The compressed data is called a video stream, which is transmitted to the user's end via wired or wireless network for decoding and viewing. The entire video encoding process includes prediction, transformation, quantization, and encoding.

[0079] In video encoding, the most commonly used color encoding methods include YUV and RGB. The color encoding method used in this invention is YUV. Y represents luminance, which is the grayscale value of the image; U and V (i.e., Cb and Cr) represent chrominance, which are used to describe the color and saturation of the image. Each Y luminance block corresponds to one Cb and one Cr chrominance block, and each chrominance block also corresponds to only one luminance block.

[0080] A video stream consists of consecutive frames. Each frame is decoded and played in sequence to form a video picture. Common frame types in existing video encoding and decoding standards include I-frames, P-frames, and B-frames.

[0081] I-frames are intra-coded frames, meaning they are independent frames with all their own encoding and decoding information, and can be encoded and decoded independently without referencing other frames. I-frames require complete encoding of all content within the frame, generally resulting in a larger bitstream and lower compression ratio.

[0082] P-frames are inter-frame predictive coded frames, which require previous frames in the display order as reference images for encoding and decoding.

[0083] B-frames are bidirectional inter-frame predictive coded frames, requiring both past and future frames in the display order as reference frames for encoding and decoding.

[0084] Figure 1 This is a schematic diagram of the frame reference relationship in a bitstream containing I-frames and P-frames. Figure 2 This is a schematic diagram of the frame reference relationship in a bitstream containing I-frames, P-frames, and B-frames. Figure 1 and Figure 2 In this context, POC (pic_order_cnt) represents the playback order of video frames, and DOI (decode order index) represents the encoding and decoding order of video frames. From... Figure 2 As can be seen, when B-frames are present in the bitstream, the encoding / decoding order and playback order of the frames can be different.

[0085] It should be noted that the frame reference relationships in a video sequence can be combined in various ways. Figure 1 and Figure 2 This is just to illustrate a common reference relationship.

[0086] In video playback, random access is often required. For example, when frames are dropped during live streaming, the decoder cannot establish reference relationships between frames and cannot decode normally. In this case, the decoder will search for the next randomly accessed frame and start decoding again. Or, in video-on-demand, when playing video from a certain moment, the decoder generally starts decoding from the previous randomly accessed frame corresponding to that moment.

[0087] To meet the requirements of random access, the encoder periodically inserts a random access frame into the bitstream. In existing technologies, random access frames are all I-frames. However, I-frames generally have a large bitstream size, which significantly impacts bandwidth during transmission, so they are typically not inserted frequently during encoding. But the larger the periodic interval for inserting I-frames, the longer the waiting time during random access. The encoder needs to make a trade-off and choose an appropriate I-frame period. One solution to this problem in existing technologies is to insert P-frames or B-frames that only reference the previous random access frame between random access frames.

[0088] For ease of understanding, in the following description, frames with POC values ​​of 0, 1, 2... will be referred to as frames 0, 1, 2... . Figure 3 In this code, frame 0 is an I-frame, and frames 1, 2, 3, and 4 are P-frames, with frame 3 directly referencing the I-frame. If frame 1 or 2 is lost, as long as the I-frame is still in the decoder, it will not affect the normal decoding of frames 3 and 4.

[0089] When playing video on demand, if you want to randomly play the 3rd frame, you only need to read and decode frames 0 and 3. And... Figure 1 In the reference model, three frames (0, 1, 2) need to be read and decoded before the third frame can be decoded to obtain the image.

[0090] It is evident that inserting P-frames or B-frames that only reference the previous random access frame can reduce the reading and decoding overhead required for random access.

[0091] Specifically, an IDR frame is a type of random access frame. An IDR frame is a special type of I-frame that, in addition to enabling random access, will clear all buffered frames in the encoding / decoding buffer if encountered during encoding / decoding. Therefore, frames encoded / decoded after an IDR frame will not reference frames encoded / decoded before the IDR frame.

[0092] The existing SVAC3 video codec standard introduces the concept of a knowledge picture (library picture), with L-frames in subsequent proposals representing knowledge pictures. Simultaneously, the concept of RL (reference library) frames is introduced; RL frames refer to P-frames or B-frames that reference only the knowledge picture. A knowledge picture is a long-term reference frame encoded using I-frames. It serves only as a reference frame and is not used for display. Knowledge pictures are identified by their knowledge picture index (IDX), rather than the POC or DOI of other frames in the bitstream.

[0093] Regarding the structure of knowledge images in the bitstream: In existing technologies, knowledge images are encoded using I-frames. However, because the QP (Queries Per Frame) of encoded knowledge images is generally small, encoding is slow, and the bitrate is generally high, interleaving an entire frame of knowledge image bitstream into the bitstream can cause significant bitrate spikes and decoding jitter. Therefore, the patch mechanism in the existing SVAC3 standard is utilized. The knowledge image is divided into multiple patches, which are then interleaved with multiple display images. Only one patch is encoded at a time and added to the bitstream, ultimately resulting in an encoded output bitstream that interleaves the knowledge base patch bitstream with the display image bitstream. (The following text is incomplete and requires further context.) Figure 4 This relates to the frame reference relationships and the position of the knowledge image in the bitstream under the IPPP configuration in existing technologies.

[0094] Regarding the management of knowledge image cache frames: In the existing technology, when an image starts to reference a new knowledge image or starts to encode / decode a new knowledge image, the previous knowledge image is replaced. That is to say, all frames in the current sequence can refer to at most one knowledge image, and that knowledge image must be the most recently encoded / decoded knowledge image.

[0095] Regarding the indexing of knowledge images: In the existing technology, knowledge images support an index of 0-511, that is, only 512 frames of knowledge images are supported to be referenced by the entire sequence of frames.

[0096] Regarding the configuration of knowledge image frames (L frames) and RL frames: In the prior art, knowledge images are encoded as P frames or B frames by treating all original I frames as RL frames according to the I frame interval.

[0097] The Reference Picture Lists (RPL) record the reference relationships for each frame and are used to update the reference picture buffer and clean up subsequent frames that are no longer referenced. In SVAC3, the syntax of the RPL is as follows:

[0098] RPLs include RPL0 and RPL1, which record the preceding and following reference frames, respectively. The following syntax is for a specific RPL0 / RPL1. As you can see, the RPL records the number of reference frames (num_of_ref_pic) and marks whether each reference frame is a knowledge image (library_index_flag). For knowledge image reference frames, it records their index (referenced_library_picture_index); for other non-knowledge image reference frames, it records (abs_delta_doi), from which the index can be further calculated. The following table gives the syntactic definition of RPLs and their meanings:

[0099]

[0100]

[0101] For each frame of an image, there is a corresponding Reference Frame Layout (RPL). The RPL contains all reference frames used by the current frame and subsequent frames. The reference frames used by the current frame are the first num_ref_default_active_minus1 reference frames among these frames. num_ref_default_active_minus1, like the RPL, is also expressed in the syntax. That is, the RPL of a frame includes:

[0102] (1) The number of reference frames used in the current frame and the frames after the current frame.

[0103] (2) Mark all reference frames as knowledge image frames.

[0104] (3) Reference index of all reference frames.

[0105] At the same time, each frame can determine, based on the syntax num_ref_default_active_minus1, which are the first few frames among all RPL reference frames that the current frame needs to reference.

[0106] This application proposes an improved frame management method and adds a frame management mechanism for knowledge images to manage multiple knowledge image cache frames. It should be noted that in the following description, L-frame refers to the knowledge image frame.

[0107] The frame management method proposed in this application is for knowledge images and mainly includes the following aspects:

[0108] (1) LDPB manages multiple L-frames: An L-frame management mechanism, LDPB, is added to manage multiple L-frame buffers. This includes the frame entry and frame exit mechanisms for the buffer, as well as the corresponding syntax.

[0109] (2) Update LDPB based on RPL: ​​Several RPL configuration methods for L frames are given, as well as the way RPL manages the addition and removal of L frames in the LDPB buffer, and the corresponding syntax is proposed.

[0110] (3) IDR_L: Add a frame type IDR_L to clean up L frames in LDPB and enable random access function.

[0111] (4) Knowledge Image Reference Index Cyclic Mechanism: Adds processing for knowledge images that exceed the required index range.

[0112] The following explanation follows the above plan.

[0113] Please refer to details. Figure 5 , Figure 5 This is a flowchart illustrating an embodiment of the video encoding method provided in this application.

[0114] like Figure 1 As shown, the specific steps are as follows:

[0115] Step S11: Obtain the first reference image queue configuration set for the current video frame.

[0116] In this embodiment, when the video encoder encodes the current video frame, it extracts the RPL information of the current video frame, namely the first reference image queue configuration set. The first reference image queue configuration set includes the reference frame index of the current video frame, and the reference frame index includes a reference knowledge image frame index and / or a reference non-knowledge image frame index.

[0117] Step S12: Obtain the reference frame index of the current video frame based on the first reference image queue configuration set.

[0118] In this embodiment, the RPL information records the knowledge image frame and the non-knowledge image frame referenced by the current video frame. Please refer to [link / reference] for details. Figure 6 , Figure 6 The RPL information of frame 1 (P) records the index of frame 0 (RL) and the index of frame 0 (L). The knowledge image of frame 1 is generated from the original image of frame 3 in the original sequence.

[0119] In existing technologies, the buffer can only hold one knowledge image frame at a time, and the video frame to be encoded can only reference the most recent knowledge image frame, i.e., Figure 6In the prior art, the 9th frame P can only refer to the 8th frame RL and the 1st frame L, because only the 1st frame L is cached in the buffer, and the 0th frame L has been deleted.

[0120] This application uses LDPB to manage multiple L-frames, allowing multiple L-frames to be cached simultaneously in the buffer. Figure 6 As shown, the reference frame index of frame 9 (P) includes frame 8 (RL) and frame 0 (L).

[0121] Step S13: Obtain the reference knowledge image frame from the buffer according to the reference frame index.

[0122] Step S14: Encode the current video frame according to the reference knowledge image frame to obtain the encoded bitstream of the current video frame.

[0123] In this embodiment, the video encoder obtains a reference knowledge image frame from the buffer using a reference frame index. The reference knowledge image frame is then used as a reference frame to encode the current video frame, thereby obtaining the video bitstream of the current video frame.

[0124] In addition, if the reference frame index also includes the index of non-knowledge image frames, that is, the index of other P frames, other B frames or other RL frames, the video encoder can also obtain other reference non-knowledge image frames, and use the reference non-knowledge image frames pointed to by the index and the reference knowledge image frames together to encode the current video frame, thereby obtaining the video bitstream of the current video frame.

[0125] The L-frame management mechanism mentioned in step S12 above is called LDPB. It can manage multiple L-frames, meaning that multiple L-frames can exist during sequence encoding / decoding and be selected for use by subsequent encoded / decoded frames. The implementation process of this management mechanism is as follows:

[0126] First, a syntax needs to be added to specify the length of the LDPB and the maximum number of L frames that can exist in the buffer at the same time.

[0127] Secondly, for each L-frame in LDPB, its status is marked, including "referenced" and "not referenced", which respectively indicate whether the frame is referenced by the current encoding / decoding frame.

[0128] Please refer to details. Figure 7 , Figure 7 This is a schematic diagram illustrating the LDPB update process during the encoding or decoding process provided in this application. Details are as follows:

[0129] 1. Before encoding or decoding, the referenced status of all L frames in the LDPB needs to be marked according to the RPL information. If the frame in the current LDPB is not included in the RPL of the current frame, it needs to be marked as "not referenced"; otherwise, the status remains "referenced".

[0130] 2. Frames marked as "not referenced" in the LDPB indicate that subsequent frames will not reference this frame, meaning you can exit the LDPB. Clear the "not referenced" frames from the LDPB, and keep the "referenced" frames.

[0131] 3. If the current frame is an L-frame, after encoding or decoding this L-frame, the generated reconstructed frame is added to the LDPB and marked as "referenced".

[0132] The LDPB management mechanism is described below through a specific implementation method:

[0133] Taking the bitstream structure in SVAC3 as an example. In existing monitoring scenarios, when a PTZ camera is used to capture images from multiple angles at a fixed point, for example, a device cyclically captures two fixed scenes, the knowledge image generated the first time for each scene can be kept in the buffer, reducing the need for repeated transmission of the same scene's knowledge image multiple times. Please continue reading. Figure 6 , Figure 6 This is a configuration where subsequent frames reference the knowledge image from previous frames. For example, in frame 0, the camera captures the first location and generates knowledge image 0; in frame 3, it captures the second location and generates knowledge image 1; and in frame 8, it switches back to the first location. In this case, the knowledge image in frame 0 can be directly referenced, eliminating the need to retransmit a new knowledge image.

[0134] To achieve the above functionality, LDPB is added. First, the syntax `max_ldpb_size_minus1` is added to the SVAC3 sequence header to represent the maximum number of frames that LDPB can store. The value of `max_ldpb_size_minus1` is set to a range of 0 to 15. Let `max_ldpb_size_minus1 + 1` represent the maximum number of frames that LDPB can store, meaning the maximum settable number of frames that LDPB can store is 16. For example, in the current bitstream, `max_ldpb_size_minus1 = 4`, meaning a maximum of 4 knowledge images can be cached.

[0135] The update process for LDPB is as follows, taking the encoding end as an example; the decoding end is similar.

[0136] (1) LDPB addition mechanism: Before the encoding of the RL frame in frame 0, the knowledge image of frame 0 is added to the LDPB and marked as "referenced". Since subsequent frames always reference the knowledge image of frame 0, the knowledge image of frame 0 always remains in the "referenced" state. Before the encoding of the RL frame in frame 4, the knowledge image of frame 1 is encoded and added to the LDPB and marked as "referenced". Similarly, it always remains in the "referenced" state in subsequent frames.

[0137] (2) Exit LDPB mechanism: For frames with index 0 in LDPB (the same applies to frames with index 1), they are kept in LDPB because they are marked as "referenced" until they are no longer needed as reference frames due to scene switching or other reasons. At that time, this information needs to be marked in the RPL of the encoded frame at that time. LDPB marks this knowledge image reference frame as "not referenced" according to the RPL information. Then, according to the "not referenced" status, the knowledge image frame with index 0 is cleaned up.

[0138] As can be seen from the above introduction, the LDPB is continuously updated during the video encoding / decoding process. The following section will continue to introduce the content of updating the LDPB based on the RPL information of the current encoded frame.

[0139] Whether an L-frame is referenced in the LDPB is updated based on the RPL information. The methods for updating the LDPB with RPL information include, but are not limited to, the following (referred to as reference update schemes in the following description):

[0140] (1) Before encoding each frame, update the LDPB according to its RPL. The RPL syntax of each frame contains L-frame information.

[0141] (2) Update the LDPB according to its RPL before encoding all non-L frames, and include L frame information only in the RPL syntax of non-L frames.

[0142] (3) LDPB is updated only at L frames based on its RPL, and the RPL syntax of each frame contains L frame information.

[0143] (4) Update LDPB only at L frames based on their RPL, and include L frame information only in the RPL syntax of L frames.

[0144] Furthermore, the list of reference frames actually used in a frame is generated from the RPL information. Specifically, it is determined which frames in the RPL information of each frame are used to construct the reference frame list for the current frame based on the default number of active reference images in the syntax. This process can be carried out using methods including but not limited to the following (referred to as the syntax scheme in the following description):

[0145] (a) A syntax needs to be added to indicate the number of reference frames used for the current frame in the current frame's RPL, for example, using the syntax `active_num_l`, i.e., the first reference syntax. The first `active_num_l` knowledge images in the RPL are the reference frames for the current frame, and the other knowledge images are reference frames used in subsequent frames. This number represents the total number of reference knowledge image frames. Simultaneously, the syntactic meaning of the original `active_num` in the RPL, i.e., the second reference syntax, needs to be modified to indicate that the first `active_num` frames among the non-L-frame reference frames are the reference frames for the current frame. This number represents the total number of reference knowledge image frames.

[0146] (b) Maintaining the existing syntax, active_num, i.e., the reference syntax, indicates that the first few frames in the RPL, including knowledge image reference frames and non-knowledge image reference frames, are the reference frames of the current frame. It also specifies the order of the reference frames (including knowledge image frames and non-knowledge image frames) of the current frame and the reference frames (including knowledge image frames and non-knowledge image frames) used by other frames in the RPL, specifically including but not limited to:

[0147] All reference frames for the current frame are those preceding reference frames not used by the current frame. When constructing the reference list, the previous active_num frames are taken as reference frames for the current frame.

[0148] All reference frames for the current frame are after reference frames not used in the current frame. When constructing the reference list, the next active_num frame is taken as the reference frame for the current frame.

[0149] The reference update scheme and syntax scheme are described below through a specific implementation method:

[0150] Take the bitstream structure in SVAC3 as an example. Figure 8 This is a frame reference relationship configuration for an application knowledge image frame, as shown in the figure. Frame 0 (L) is generated from the original frame 0, while frame 1 (L) is generated from the original frame 3. The frames are transmitted in patches after frame 2, and the transmission is completed before frame 4. The frames are referenced in frame 4 and thereafter.

[0151] In this configuration, after the encoding and decoding of frame 1 and frame L, frame 0 and frame L still need to be retained until at least frame 4. Below are specific implementation examples of the above reference update schemes and syntax schemes:

[0152] (1) Before encoding each frame, update the LDPB according to its RPL. The RPL syntax of each frame contains L-frame information.

[0153] Please refer to details. Figure 9 , Figure 9 yes Figure 5 The diagram shows the specific process flow of step S14 in the video encoding method.

[0154] like Figure 9 As shown, before encoding the current video frame according to the reference knowledge image frame, the video encoding method may also perform the following steps:

[0155] Step S141: Obtain the knowledge image frame referenced by the current video frame and subsequent video frames to be encoded based on the first reference image queue configuration set of the current video frame.

[0156] In this embodiment of the application, after the video encoder extracts the RPL information of the current video frame, it first analyzes the syntax in the RPL:

[0157] For the syntax related to active_num, the syntax scheme (a) is adopted, which adds the syntax related to active_num_l while modifying the meaning of the original syntax related to active_num.

[0158] The sequence header syntax is as follows: the meaning of num_ref_default_active_minus1 in the original syntax is modified, and the syntax num_ref_default_l_active_minus1 is added.

[0159]

[0160] The image header syntax is as follows: Similarly, the active_num_l related syntax is added, which is transmitted only when the current frame is not an L frame. num_l_ref_active_override_flag being 0 indicates that the active_num_l of the current frame can be the same as the sequence header, and num_l_ref_active_override_flag being 1 indicates that the new value is retransmitted for this frame.

[0161]

[0162]

[0163]

[0164] Before encoding or decoding each frame, this RPL information can be used to obtain the knowledge image reference frame for the current frame, as well as which frames need to be referenced for the current frame. For example, in this case, since all frames are P-frames, only RPL0 is in the RPL.

[0165] To achieve this configuration, the number of active knowledge image references is 1 for each non-L frame, 0 for RL frames, and 1 for P frames. For L frames, the number of active knowledge image references is 0 by default, and the number of active references is also 0.

[0166] The following lists the RPL0 values ​​for some frames.

[0167] Frame 0: {L0}, DPB is added to frame 0 after encoding and decoding.

[0168] Frame 1: {0, L0}.

[0169] Frame 2: {1, L0}.

[0170] Frame 3: {2, L0}, the current frame references the knowledge images of frame 2 and frame 0.

[0171] Frame 4: {L1, L0}, currently referencing the knowledge image of frame 1, the knowledge image of frame 0 is also kept in the buffer.

[0172] Knowledge image for frame 0: {}, which is empty before encoding in LDPB and added after encoding and decoding of frame 0.

[0173] Knowledge image of frame 1: {L0}, knowledge image of frame 0 is kept in the buffer.

[0174] Step S142: Update the buffer to save the knowledge image frames referenced by the current video frame and subsequent video frames to be encoded, and delete the knowledge image frames not referenced by the current video frame and subsequent video frames to be encoded.

[0175] In this embodiment of the application, according to the reference update scheme (1), the video encoder needs to update the LDPB according to its RPL information before encoding each video frame. For example, when the current video frame is the 4th frame, the 4th frame refers to the 1st frame L frame, but the RPL information of the 4th frame contains the 0th frame L frame, indicating that the 0th frame L frame still needs to be retained in the buffer area before encoding the 4th frame.

[0176] (2) Update the LDPB according to its RPL before encoding all non-L frames, and include L frame information only in the RPL syntax of non-L frames.

[0177] The difference between this method and the reference update scheme (1) is that RPL-related syntax is not transmitted in the image header of the L frame, and the buffer is not cleared at the L frame.

[0178] For example, in this case, since all frames are P-frames, only RPL0 exists in the RPL. To achieve this configuration, the number of active knowledge image references for each non-L-frame is 1, the number of active reference images is 0 for RL-frames and 1 for P-frames. The RPL0 for some frames is listed below.

[0179] Frame 0: {L0}.

[0180] Frame 1: {0, L0}.

[0181] Frame 2: {1, L0}.

[0182] Frame 3: {2, L0}, the current frame references the knowledge images of frame 2 and frame 0.

[0183] Frame 4: {L1, L0}, currently referencing the knowledge image of frame 1, the knowledge image of frame 0 is also kept in the buffer.

[0184] Frame 0 Knowledge Image: No RPL transmitted.

[0185] Frame 1 Knowledge Image: No RPL transmitted.

[0186] In parameter update schemes (3) and (4), since the LDPB is updated only at the L frame according to its RPL, the LDPB update scheme when encoded to a knowledge image frame will be introduced below:

[0187] (3) LDPB is updated only at L frames based on its RPL, and the RPL syntax of each frame contains L frame information.

[0188] Please continue reading. Figure 10 , Figure 10 This is a flowchart illustrating another embodiment of the video encoding method provided in this application.

[0189] like Figure 10 As shown, the specific steps are as follows:

[0190] Step S21: Obtain the second reference image queue configuration set of knowledge image frames to be encoded before or after the current video frame, and obtain the index of the reserved knowledge image frame from the second reference image queue configuration set.

[0191] In this embodiment, since the LDPB is updated and cleared only at the L frame according to its RPL, the knowledge image referenced by the current frame only needs to be marked in the RPL of the remaining frames. The RPL of the remaining frames does not need to reserve a knowledge image reference frame cache for subsequent frames.

[0192] Therefore, the sequence header syntax is the same as above under this method. The image header syntax is the same as in the prior art, that is, the image header does not need to add the active knowledge image reference image number syntax, and the actual transmitted RPL does not need to include the knowledge images that the current frame does not reference but that subsequent frames need to reference.

[0193] For example, in this case, to achieve this configuration, the number of active knowledge image reference images for each non-L frame is equal to the number of knowledge image frames in the current frame's RPL; that is, all knowledge image frames in the RPL are reference frames for the current frame. For knowledge image frames, the number of active knowledge image reference images defaults to 0. In this example, since all frames are P-frame encoded, only RPL0 exists in the RPL.

[0194] The following lists the RPL0 values ​​for some frames.

[0195] Frame 0: {L0}.

[0196] Frame 1: {0, L0}.

[0197] Frame 2: {1, L0}.

[0198] Frame 3: {2, L0}, the current frame references the knowledge images of frame 2 and frame 0.

[0199] Frame 4: {L1}, currently referencing the knowledge image in Frame 1.

[0200] Knowledge image for frame 0: {}. There is currently no knowledge image in the LDPB. Add L0 to the encoded LDPB.

[0201] Knowledge image in frame 1: {L0}. L0 is retained in LDPB. L1 is added to LDPB after encoding and decoding.

[0202] Therefore, during the encoding of frames 0 to 4, LDPB updates are not triggered; subsequent steps are only executed when encoding the knowledge image of frame 0 and frame 1.

[0203] Step S22: Based on the retained knowledge image frame index, retain the corresponding knowledge image frame in the cache, and delete the knowledge image frames other than the retained knowledge image frame index from the cache.

[0204] In this embodiment, when encoding the first frame of knowledge image, the RPL of the first frame of knowledge image indicates that the 0th frame of knowledge image is to be retained. If other knowledge images exist at this time, then according to the RPL of the first frame of knowledge image, all other knowledge images need to be deleted from the buffer.

[0205] Step S23: After encoding the knowledge image frame to be encoded, store the encoding result in the buffer.

[0206] In this embodiment of the application, after the video encoder updates the buffer according to the RPL of the first frame knowledge image, it encodes the first frame knowledge image and stores the first frame knowledge image in the buffer after the encoding is completed.

[0207] (4) Update LDPB only at L frames based on their RPL, and include L frame information only in the RPL syntax of L frames.

[0208] For some simple configurations, such as in this example where the frames before the next L frame are all reference knowledge image 0, and a sequence after knowledge image 1 is generated is all reference knowledge image 1, this method can be used to save some RPL-related syntax. The specific method is as follows.

[0209] The sequence header is the same as above. In the image header, for non-knowledge image frames, there is no need to transmit the reference relationships to knowledge images; only the number of active knowledge image reference images needs to be transmitted. For knowledge image frames, RPL needs to be transmitted normally, and the number of active knowledge image reference images is 0 by default and does not need to be transmitted. Details are as follows:

[0210]

[0211]

[0212] The `reference_picture_list_set_l` function works as follows: (For non-knowledge image frames, all transmitted frames are non-knowledge image frames.)

[0213]

[0214]

[0215] Before encoding and decoding non-knowledge image frames, the knowledge image referenced by that frame is obtained from the LDPB based on the number of active knowledge image reference images transmitted for that frame. At the knowledge image location, the LDPB needs to be updated according to the RPL to remove subsequent frames that are no longer referenced.

[0216] For example, in this case, to achieve this configuration, the number of active knowledge image reference images is 0 by default for knowledge image frames. Since all frames are P-frame encoded, only RPL0 is in RPL. The RPL0 of some frames is listed below.

[0217] Knowledge image for frame 0: There is currently no knowledge image in the LDPB. Add L0 to the encoded LDPB.

[0218] Frame 0: {}, the number of active knowledge image reference images is 1, and L0 is taken from LDPB as the reference frame.

[0219] Frame 1: {0}, the number of active knowledge image reference images is 1, and L0 is taken from LDPB as the reference frame.

[0220] Frame 2: {1}, the number of active knowledge image reference images is 1, and L0 is taken from LDPB as the reference frame.

[0221] Frame 3: {2}, the number of active knowledge image reference images is 1, and L0 is taken from LDPB as the reference frame.

[0222] Knowledge image of frame 1: {L0}. L0 is retained in LDPB. After encoding, L1 is added to LDPB and placed at the beginning of LDPB.

[0223] Frame 4: {}, the number of active knowledge image reference images is 1, and L1 is taken from LDPB as the reference frame.

[0224] The reference update scheme and syntax scheme are described below through another specific implementation method:

[0225] Taking the bitstream structure in SVAC3 as an example. Figure 6 Taking the structure in the image as an example, the knowledge image of frame 0 is generated from the original image of frame 0, and the knowledge image of frame 1 is generated from the original image of frame 2.

[0226] The method of using the reference update scheme (1) + syntax scheme (b) is adopted, and the syntax is the same as that of the prior art.

[0227] For example, in this case, since all frames are P-frames, only RPL0 is present in the RPL. To achieve this configuration, for non-L frames, except for RL frames which have a maximum of 1 active reference image, the number of active reference images for all other non-L frames is 2, while the number of active reference images for knowledge image frames is 0 by default. The RPL0 values ​​for some frames are listed below.

[0228] Frame 0: {L0}.

[0229] Frame 1: {0, L0}.

[0230] Frame 2: {1, L0}.

[0231] Frame 3: {2, L0}, the current frame references the knowledge images of frames 2 and 0, and the knowledge image of frame 1 is also kept in the buffer.

[0232] Frame 4: {L1, L0}, currently referencing the knowledge image of frame 1, the knowledge image of frame 0 is also kept in the buffer.

[0233] …

[0234] Frame 8: {L0, L1}, currently referencing the knowledge image of frame 0, while the knowledge image of frame 1 is also stored in the buffer.

[0235] Knowledge image for frame 0: {}, no knowledge image currently in LDPB. Add L0 to the encoded LDPB.

[0236] Knowledge image for frame 1: {L0}, knowledge image for frame 0 is retained in the buffer. L1 is added to the encoded LDPB.

[0237] Furthermore, this application proposes a method for clearing L frames in the DPB based on the RPL. Since previous L frames may need to be retained at the L frame level, random access functionality cannot be implemented at ordinary L frames. To further facilitate random access, the concept of IDR frames is introduced into L frames, proposing the IDR_L frame for clearing the LDPB.

[0238] When encoding / decoding to an IDR_L frame, all frames in the LDPB and DPB are cleared to enable random access within the IDR_L frame. Simultaneously, the L-frame index is renumbered starting from the IDR_L frame.

[0239] The following describes a specific implementation scheme for clearing the IDR_L frame buffer:

[0240] Taking the bitstream structure in SVAC3 as an example. In existing monitoring scenarios, when scene switching occurs, such as day and night, the L-frame generated after the switch can be used as the IDR_L-frame, all previous knowledge image caches can be cleared, and the index of the knowledge image frame can be numbered starting from 0.

[0241] Specifically, the video encoder can store scene change syntax in the syntax of a video frame by pre-setting scene change syntax. The scene change syntax indicates that the video frame has changed scene, triggering the IDR_L frame clearing buffer scheme.

[0242] The existing knowledge image index range is 0-511. For new knowledge images that exceed this range, there is no corresponding method for setting a reference index. In monitoring scenarios, there are many long sequences, which will generate many frames of knowledge images. Therefore, it is necessary to design a corresponding method for referencing knowledge images that exceed the index range.

[0243] To address this, this application proposes a knowledge image reference indexing loop mechanism, as follows:

[0244] Methods for limiting the values ​​of the knowledge image reference index in RPL include, but are not limited to, the following:

[0245] (1) Set the index range to 0 to N-1 and set the cycle value Cycle. For new frames that are out of range, record them as frame 0 and increment their Cycle value by 1. Calculate the real index of the knowledge image based on the recorded Cycle value and the index Idx of the current record, and obtain the knowledge image reference frame.

[0246] For example, the actual index IDX = Cycle * N + Idx.

[0247] (2) Set the index range to 0 to N. For new frames that exceed the range N, re-record them as frame 0. At the same time, clean up all other frames in LDPB. All frames after the new frame 0 can no longer refer to the previous knowledge image.

[0248] For example, if a new knowledge image frame that is out of range is needed during the encoding process, the new knowledge image frame is set as an IDR_L frame. Starting from this frame, the index is reset to 0, and all previously cached knowledge images are cleared.

[0249] In this embodiment, the video encoder obtains a first reference image queue configuration set for the current video frame; obtains a reference frame index for the current video frame based on the first reference image queue configuration set; retrieves a reference knowledge image frame from the buffer according to the reference frame index; and encodes the current video frame according to the reference knowledge image frame to obtain the encoded bitstream of the current video frame. Through the above video encoding method, the reference frame for video frame encoding is optimized from only being able to use the nearest knowledge image frame as the reference frame to determining the reference knowledge image frame according to the reference frame index of the video frame. This expands the multi-frame management mechanism of knowledge images, better adapts to changing scene switching, and improves video encoding effect and encoding efficiency.

[0250] The method proposed in this application, which uses LDPB and corresponding RPL to manage multiple knowledge image frames, can realize the management of multiple knowledge images. Multiple knowledge images can exist in the buffer at the same time, and the previously encoded knowledge images can be referenced in the subsequent sequence. This is beneficial for the encoding of monitoring sequences in scenarios such as switching points of PTZ cameras, as it can reduce the number of times similar knowledge images are transmitted and speed up the encoding process.

[0251] The IDR_L method proposed in this application introduces the concept of IDR frames in knowledge images. When switching to a new scene during monitoring, it immediately clears the knowledge image reference frame cache to achieve random access. Simultaneously, this method can also prevent the knowledge image reference index from exceeding its range.

[0252] This application proposes a solution to the problem of knowledge image reference indexes being out of range, addressing the issue of long sequences needing to update knowledge images but being unable to be labeled.

[0253] Please continue reading. Figure 11 , Figure 11 This is a flowchart illustrating an embodiment of the video decoding method provided in this application.

[0254] like Figure 11 As shown, the specific steps are as follows:

[0255] Step S31: Extract the reference image queue configuration set from the video stream.

[0256] Step S32: Obtain the reference frame index of the video stream based on the reference image queue configuration set.

[0257] Step S33: Retrieve the decoded reference knowledge image frame from the buffer according to the reference frame index.

[0258] Step S34: Decode the video stream according to the reference knowledge image frames to obtain the reconstructed video frames of the video stream.

[0259] In this embodiment, the technical content of frame reference and frame management in the video encoding end and the video decoding end is basically the same, and will not be repeated here.

[0260] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0261] To implement the above video encoding method, this application also proposes a video encoder, which can be found in the following details. Figure 12 , Figure 12 This is a schematic diagram of the structure of an embodiment of the video encoder provided in this application.

[0262] The video encoder 400 in this embodiment includes a processor 41, a memory 42, an input / output device 43, and a bus 44.

[0263] The processor 41, memory 42, and input / output device 43 are respectively connected to the bus 44. The memory 42 stores program data, and the processor 41 is used to execute the program data to implement the video encoding method described in the above embodiments.

[0264] In this embodiment, processor 41 can also be referred to as a CPU (Central Processing Unit). Processor 41 may be an integrated circuit chip with signal processing capabilities. Processor 41 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor, or processor 41 can be any conventional processor.

[0265] To implement the above video decoding method, this application also proposes a video decoder, which can be found in the following details. Figure 13 , Figure 13 This is a schematic diagram of the structure of an embodiment of the video decoder provided in this application.

[0266] The video decoder 500 in this embodiment includes a processor 51, a memory 52, an input / output device 53, and a bus 54.

[0267] The processor 51, memory 52, and input / output device 53 are respectively connected to the bus 54. The memory 52 stores program data, and the processor 51 is used to execute the program data to implement the video decoding method described in the above embodiments.

[0268] In this embodiment, processor 51 can also be referred to as a CPU (Central Processing Unit). Processor 51 may be an integrated circuit chip with signal processing capabilities. Processor 51 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor, or processor 51 can be any conventional processor.

[0269] This application also provides a computer storage medium; please refer to the following: Figure 14 , Figure 14 This is a schematic diagram of a computer storage medium according to an embodiment of the present application. The computer storage medium 600 stores a computer program 61. When the computer program 61 is executed by a processor, it is used to implement the video encoding method and / or video decoding method of the above embodiment.

[0270] When the embodiments of this application are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0271] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A video coding method based on knowledge images, characterized in that, The video encoding method includes: Get the first reference image queue configuration set for the current video frame; The reference frame index of the current video frame is obtained based on the first reference image queue configuration set; Retrieve reference knowledge image frames from the cache based on the reference frame index; The current video frame is encoded according to the reference knowledge image frame to obtain the encoded bitstream of the current video frame; the reference frame index includes a reference knowledge image frame index and a reference non-knowledge image frame index; The step of obtaining the reference frame index of the current video frame based on the first reference image queue configuration set includes: The number of reference knowledge image frames is obtained according to the first reference syntax of the current video frame; The number of reference non-knowledge image frames is obtained according to the second reference syntax of the current video frame; According to the number of reference knowledge image frames, obtain the index of the reference knowledge image frame that is sorted first in the first reference image queue configuration set; According to the number of reference non-knowledge image frames, obtain the index of the reference non-knowledge image frame that is sorted first in the first reference image queue configuration set, wherein the reference non-knowledge image frame index is used to obtain the reference non-knowledge image frame.

2. The video encoding method according to claim 1, characterized in that, Before encoding the current video frame according to the reference knowledge image frame, the video encoding method further includes: Based on the first reference image queue configuration set of the current video frame, obtain the knowledge image frame that is referenced by the current video frame and subsequent video frames to be encoded. Update the cache to save the knowledge image frames that are referenced by the current video frame and subsequent video frames to be encoded, and delete the knowledge image frames that are not referenced by the current video frame and subsequent video frames to be encoded.

3. The video encoding method according to claim 2, characterized in that, The knowledge image frames preceding or following the current video frame do not include the reference image queue configuration set.

4. The video encoding method according to claim 1, characterized in that, The video encoding method further includes: Obtain a second reference image queue configuration set of knowledge image frames to be encoded before or after the current video frame, and obtain the index of the reserved knowledge image frame from the second reference image queue configuration set; Based on the retained knowledge image frame index, the corresponding knowledge image frame is retained in the cache, and knowledge image frames other than the retained knowledge image frame index are deleted from the cache. After encoding the knowledge image frame to be encoded, the encoding result is stored in the buffer area.

5. The video encoding method according to claim 4, characterized in that, The first reference image queue configuration set of the current video frame does not include the index of the reference knowledge image frame; The step of retrieving the reference knowledge image frame from the cache based on the reference frame index includes: When the reference frame index does not include the index of the reference knowledge image frame, the latest saved knowledge image frame is retrieved from the cache and used as the reference knowledge image frame for the current video frame.

6. The video encoding method according to claim 1, characterized in that, The reference frame index includes a reference knowledge image frame index and a reference non-knowledge image frame index; The step of obtaining the reference frame index of the current video frame based on the first reference image queue configuration set includes: The number of reference knowledge image frames and the number of reference non-knowledge image frames are obtained according to the reference syntax of the current video frame. According to the number of reference knowledge image frames, obtain the index of the reference knowledge image frame that is sorted first in the first reference image queue configuration set; According to the number of reference non-knowledge image frames, obtain the index of the reference non-knowledge image frame that is sorted first in the first reference image queue configuration set, wherein the reference non-knowledge image frame index is used to obtain the reference non-knowledge image frame. The step of encoding the current video frame according to the reference knowledge image frame to obtain the encoded bitstream of the current video frame includes: The current video frame is encoded according to the reference knowledge image frame and the reference non-knowledge image frame to obtain the encoded bitstream of the current video frame.

7. The video encoding method according to claim 1, characterized in that, The step of encoding the current video frame according to the reference knowledge image frame to obtain the encoded bitstream of the current video frame includes: The current video frame is encoded according to the reference knowledge image frame and the reference non-knowledge image frame to obtain the encoded bitstream of the current video frame.

8. The video encoding method according to claim 1, characterized in that, The video encoding method further includes: Obtain the knowledge image frames to be encoded before or after the current video frame; After encoding the knowledge image frame to be encoded, the encoding result is stored in the buffer area, and the encoding index is written into the image header syntax of the encoded knowledge image frame.

9. The video encoding method according to claim 1, characterized in that, The video encoding method further includes: Obtain the frame type syntax of the knowledge image frames to be encoded before or after the current video frame; When the frame type syntax is started, the encoded knowledge image frames in the buffer are cleared; After encoding the knowledge image frame to be encoded, the encoding result is stored in the buffer area, and the index is renumbered. The renumbered encoding index is then written into the image header syntax of the encoded knowledge image frame.

10. The video encoding method according to claim 1, characterized in that, The reference frame index includes a period value and a current period index; The step of retrieving the reference knowledge image frame from the cache based on the reference frame index includes: Obtain the true index of the reference frame according to the cycle value and the current cycle index; The reference knowledge image frame is obtained from the cache based on the reference frame real index.

11. A video decoding method based on knowledge images, characterized in that, The video decoding method includes: Extract a reference image queue configuration set from the video stream; The reference frame index of the video stream is obtained based on the reference image queue configuration set; Decoded reference knowledge image frames and decoded reference non-knowledge image frames are obtained from the cache according to the reference frame index; wherein, the reference non-knowledge image frames are obtained according to the reference non-knowledge image frame index, and the reference non-knowledge image frame index is determined according to the sorting of the number of reference non-knowledge image frames in the reference syntax in the reference image queue configuration set; The video stream is decoded according to the reference knowledge image frame and the reference non-knowledge image frame to obtain the reconstructed video frame of the video stream.

12. A video encoder, characterized in that, The video encoder includes a memory and a processor coupled to the memory; The memory is used to store program data, and the processor is used to execute the program data to implement the video encoding method as described in any one of claims 1 to 10.

13. A video decoder, characterized in that, The video decoder includes a memory and a processor coupled to the memory; The memory is used to store program data, and the processor is used to execute the program data to implement the video decoding method as described in claim 11.

14. A computer storage medium, characterized in that, The computer storage medium is used to store program data, which, when executed by a processor, is used to implement the video encoding method as described in any one of claims 1 to 10, and / or the video decoding method as described in claim 11.

Images