Method, system and device for determining reference picture for video picture coding, and storage medium

Abstract

Disclosed in the embodiments of the present application are a method, system and device for determining a reference picture for video picture coding, and a storage medium. According to the technical solution provided in the embodiments of the present application, the method comprises: detecting the current network state, and on the basis of the current network state, querying a pre-constructed coded picture buffer; when the current network state is a first specified state, selecting the closest picture of the current picture from the coded picture buffer and taking same as a reference picture, wherein the closest picture is a coded video picture closest to the current picture; and when the current network state is a second specified state, selecting the closest reliable picture of the current picture from the coded picture buffer and taking same as the reference picture, wherein the closest reliable picture is a reliable picture closest to the current picture. By means of the technical means, the probability of the current picture being successfully decoded by means of an opposing end can be increased, such that the decoding reliability of the current picture is ensured, thereby improving the stability of video data transmission and improving the real-time video communication experience.

Description

Method, system, device and storage medium for determining reference frame for video frame coding

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on September 26, 2023, with application number 202311259055.X. The entire contents of this application are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the field of video decoding technology, and in particular to a method, system, device, and storage medium for determining a reference frame for video frame encoding. Background Art

[0003] Currently, in video encoding scenarios, encoding the current frame often requires reference to previously encoded frames. These frames are called reference frames. During video encoding, previously encoded frames that are closer to the current frame are often selected as reference frames. This ensures more accurate frame prediction, reduces the amount of residual coding required, and reduces the bitrate required for storage and transmission while maintaining the same quality. This also facilitates video storage and dissemination.

[0004] However, in RTC (Real-Time Communication) scenarios, network packet loss often occurs, meaning the referenced video frame is at risk of being lost. If the referenced frame is lost, the current frame, even at the highest encoding quality, cannot be decoded. Simply selecting a video frame closer to the current frame as the reference frame may result in network packet loss, making the current frame undecodable. This can affect the smoothness and stability of real-time communication, and consequently, the user's real-time video communication experience.

[0005] Summary of the Invention

[0006] The embodiments of the present application provide a method, system, device and storage medium for determining a reference frame for video frame encoding, which can flexibly select a reference frame, ensure the reliability of current frame decoding, and solve the technical problem that the current frame cannot be decoded due to the loss of the reference frame in a network packet loss scenario.

[0007] In a first aspect, an embodiment of the present application provides a method for determining a reference frame for video frame encoding, comprising:

[0008] Detect the current network status and query the pre-built coded frame cache based on the current network status. The coded frame cache is used to store a set number of coded video frames, which include reliable frames. Reliable frames are video frames that have been successfully decoded by the other end.

[0009] When the current network state is the first specified state, selecting a frame closest to the current frame from the coded frame buffer as a reference frame, where the closest frame is an encoded video frame closest to the current frame;

[0010] When the current network state is the second specified state, a nearest reliable frame of the current frame is selected from the coded frame buffer as a reference frame, where the nearest reliable frame is a reliable frame closest to the current frame.

[0011] In a second aspect, an embodiment of the present application provides a reference frame determination system for video frame encoding, comprising:

[0012] a query module configured to detect a current network state and query a pre-built coded frame cache based on the current network state, the coded frame cache being used to store a set number of coded video frames, the coded video frames including reliable frames, which are video frames that have been successfully decoded by the peer end;

[0013] The reference module is configured to select the most recent frame of the current frame from the coded frame cache as a reference frame when the current network state is a first specified state, where the most recent frame is the coded video frame closest to the current frame; and to select the most recent reliable frame of the current frame from the coded frame cache as a reference frame when the current network state is a second specified state, where the most recent reliable frame is the reliable frame closest to the current frame.

[0014] In a third aspect, an embodiment of the present application provides a reference frame determination device for video frame encoding, including:

[0015] memory and one or more processors;

[0016] The memory is configured to store one or more programs;

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the reference frame determination method for video frame encoding as described in the first aspect.

[0018] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by a computer processor, the method for determining a reference frame for video frame encoding as described in the first aspect is configured to be executed.

[0019] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes instructions. When the instructions are executed on a computer or a processor, the computer or the processor executes the reference frame determination method for video frame encoding as described in the first aspect.

[0020] The embodiment of the present application detects the current network status and queries a pre-built coded frame cache based on the current network status. The coded frame cache is used to store a set number of coded video frames, which include reliable frames. Reliable frames are video frames that have been successfully decoded by the other end. When the current network status is a first specified state, the most recent frame of the current frame is selected from the coded frame cache as a reference frame. The most recent frame is the coded video frame closest to the current frame. When the current network status is a second specified state, the most recent reliable frame of the current frame is selected from the coded frame cache as a reference frame. The most recent reliable frame is the reliable frame closest to the current frame. Using the above technical means, the coded frame cache is queried according to the current network status. When the first specified state is good, the most recent frame of the current frame is selected as the reference frame. When the second specified state is poor, the most recent reliable frame of the current frame is selected as the reference frame. This ensures that the current frame can select a reference frame nearby, improving encoding quality. At the same time, in the case of network packet loss, by providing the most recent reliable frame, the probability of the current frame being successfully decoded by the other end can be increased, ensuring the reliability of the current frame decoding, thereby improving the stability of video data transmission and enhancing the real-time video communication experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] FIG1 is a flowchart of a method for determining a reference frame for video frame encoding provided by an embodiment of the present application;

[0022] FIG2 is a schematic diagram of reference frame selection in an embodiment of the present application;

[0023] FIG3 is a schematic diagram of a first encoding frame buffer update in an embodiment of the present application;

[0024] FIG4 is a schematic diagram of a second encoding frame buffer update in an embodiment of the present application;

[0025] FIG5 is a schematic diagram of a third encoding frame buffer update in an embodiment of the present application;

[0026] FIG6 is a schematic diagram of a fourth encoding frame buffer update in an embodiment of the present application;

[0027] FIG7 is a schematic diagram of a fifth encoding frame buffer update in an embodiment of the present application;

[0028] FIG8 is a schematic diagram of a sixth encoding frame buffer update in an embodiment of the present application;

[0029] FIG9 is a schematic structural diagram of a reference frame determination system for video frame coding provided by an embodiment of the present application;

[0030] FIG10 is a schematic structural diagram of a reference frame determination device for video frame encoding provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solutions and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. It should also be noted that, for ease of description, only parts related to the present application, not all of the contents, are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but can also have additional steps not included in the accompanying drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0032] The reference frame determination method for video frame encoding provided in this application is intended to adaptively select reference frames from the encoding frame cache according to the real-time network status, so that when the network is good, the frame reference mode of the nearest frame reference or even the sequential reference can be enabled in time. When the network packet is lost, it can give priority to ensuring that the current frame can be correctly decoded and displayed, thereby improving the stability of video data encoding and transmission.

[0033] In the relevant reference frame selection scheme, a frame reference method of a nearest frame reference or even a sequential reference is generally adopted. In a scenario with a good network, this frame reference method enables the video frame receiving end to obtain the reference frame for correct decoding. However, in a real-time communication scenario, once the network status is poor and packet loss occurs, the nearest selected reference frame may not be received by the video receiving end due to packet loss. Therefore, when the video receiving end decodes the current frame, it cannot correctly decode the current frame because it has not received the reference frame in advance. This will affect the correct decoding of the video frame and further affect the quality of real-time communication. Based on this, a reference frame determination method for video frame encoding according to an embodiment of the present application is provided to solve the technical problem that the current frame cannot be decoded due to the loss of the reference frame in a network packet loss scenario.

[0034] Example:

[0035] FIG1 is a flowchart of a method for determining a reference frame for video frame coding provided in an embodiment of the present application. The method for determining a reference frame for video frame coding provided in this embodiment can be performed by a reference frame determination device for video frame coding. The reference frame determination device for video frame coding can be implemented in software and / or hardware. The reference frame determination device for video frame coding can be composed of two or more physical entities or a single physical entity. Generally speaking, the reference frame determination device for video frame coding can be a video coding device such as a mobile phone, a tablet, or a video coding server.

[0036] The following description will be made by taking the reference frame determination device for video frame coding as the subject of the reference frame determination method for video frame coding as an example. Referring to FIG1 , the reference frame determination method for video frame coding specifically includes:

[0037] S110. Detect the current network status and query a pre-built coded frame cache based on the current network status. The coded frame cache is used to store a set number of coded video frames. The coded video frames include reliable frames. Reliable frames are video frames that have been successfully decoded by the other end.

[0038] When encoding a video frame, the embodiment of the present application queries the current network status to adaptively decide the reference frame of the video frame currently being encoded based on the network status, and defines the video frame currently being encoded as the current frame.

[0039] The current network state is determined based on the packet loss rate of video data packets transmitted in the current network. A corresponding packet loss rate threshold is set based on actual reference frame selection requirements. When the network packet loss rate is lower than the packet loss rate threshold, the current network is determined to be good, and the current network state is defined as a first network state. When the network packet loss rate reaches the packet loss rate threshold, the current network is determined to be poor, and the current network state is defined as a second network state. In this way, by determining whether the current network state is in the predefined first network state or second network state, the frame reference decision for the current frame can be adaptively made according to different network states.

[0040] The embodiment of the present application pre-constructs a coded picture buffer (CPB) to store already coded video frames that can be selectively referenced. Optionally, in the encoder, the maximum capacity of the coded frame buffer is 16 coded video frames. The specific size of the coded frame buffer can be set according to the actual application scenario, so at a certain moment there may be less than or equal to 16 coded video frames in the coded frame buffer of the encoding end. The embodiment of the present application does not impose a fixed limit on the specific coded frame buffer capacity, which will not be described here one by one. By limiting the coded frame buffer to 16 coded video frames, the decision-making requirements of the frame reference can be guaranteed, and the memory occupation of the encoder by the coded frame buffer can also be reduced.

[0041] It should be noted that the encoded video frames in the encoded frame buffer include reliable frames. Reliable frames are encoded video frames that have been successfully encoded by the encoder and sent to the peer end for successful decoding. By setting reliable frames, in the event of network packet loss, reliable frames can be used as frame references. It is understandable that since the reliable frames have been successfully decoded at the peer end, the current frame is encoded using the reliable frames as reference frames. When the peer end decodes the current frame, there must be a reference frame for the current frame, so it will not cause a situation where normal decoding cannot be performed.

[0042] In one-to-one real-time communication scenarios, the peer's response (ACK) information can be used to determine which previously transmitted encoded video frames were correctly decoded and which could not be correctly decoded due to frame loss or reference frame loss. This response information can be used to identify frames that can be correctly decoded by the peer as reliable frames. The reliable frame closest to the current frame is called the most recent reliable frame.

[0043] S120. When the current network state is the first specified state, select the nearest frame of the current frame from the coded frame cache as a reference frame, where the nearest frame is the encoded video frame closest to the current frame; when the current network state is the second specified state, select the nearest reliable frame of the current frame from the coded frame cache as a reference frame, where the nearest reliable frame is the reliable frame closest to the current frame.

[0044] Furthermore, based on the determined network status, a flexible frame reference can be made for the current frame. When the network condition is good, the current network condition is determined to be the first network condition based on network status detection. The current frame to be encoded can directly reference the most recent frame in the encoded frame cache to improve encoding quality. The most recent frame is the encoded frame closest to the current frame. When the network condition is poor, the current network condition is determined to be the second network condition based on network status detection. The current frame can reference the most recent reliable frame in the encoded frame cache. The most recent reliable frame is the reliable frame closest to the current frame. This increases the probability that the current frame will be correctly decoded, resulting in smoother video.

[0045] Finally, based on the reference frame identified above, the current frame is encoded using that reference frame, and the reference frame information is added to the encoded data of the current frame. Upon receiving the encoded data of the current frame, the peer (video receiver) can use the reference frame information to find the corresponding decoded video frame and use it as a reference for decoding the current frame.

[0046] In this way, for real-time video communication scenarios, the reference frame determination method for video frame encoding can be used to adaptively adjust the frame reference relationship according to the network conditions. When the network is good, the nearest frame reference method can be used to improve the video quality. When the network is poor, the frame reference strategy can be adaptively adjusted to prioritize the smooth playback of the video and avoid the situation where the video frame cannot be played, thereby improving the real-time video communication experience. Optionally,

[0047] Exemplarily, before selecting the nearest reliable frame of the current frame from the coded frame buffer as the reference frame, the method further includes: at the initial coding moment, taking the initially coded key frame as the nearest reliable frame and putting it into the coded frame buffer.

[0048] When encoding a key frame or refreshing a key frame, the key frame is directly used as the most recent reliable frame. Even if the key frame has not yet been transmitted to the peer, or has been transmitted but the reply information has not been successfully returned, due to the lack of reliable frames in the encoding frame buffer, the key frame is placed in the encoding frame buffer as the most recent reliable frame. The current frame then uses the most recent frame in the encoding buffer or the most recent reliable frame as the reference frame based on network conditions.

[0049] Referring to Figure 2, a schematic diagram of reference frame selection according to an embodiment of the present application is provided. As shown in Figure 2, x-16 to x-1 are 16 consecutive encoded frames. As encoding progresses, the encoded frame buffer gradually fills with 16 frames. At this time, when the network is in the first network state, the encoded frame buffer often contains the 16 consecutive frames that have been encoded most recently. Assuming that x-4 is the most recent reliable frame, since the network is good, the current frame can directly reference frame x-1.

[0050] Optionally, based on the above-mentioned method for determining the reference frame for video frame encoding, an embodiment of the present application further constructs a data storage structure corresponding to a set number. When the frame sequence of the encoded video frames in the encoding frame cache is continuous, the data storage structure at the corresponding sequence position is selected to store the identification information of the reference frame according to the sequence position of the reference frame in the encoding frame cache, and the current frame is encoded in combination with the data storage structure at the corresponding sequence position. The set number of data storage structures corresponds one-to-one to the encoded video frames in the encoding frame cache.

[0051] The encoder uses the RPS (Reference Picture Set) data storage structure to store reference frame identification information. That is, each frame to be encoded has a corresponding RPS structure. The RPS indicates the current frame reference status and coded frame buffer management. Specifically, it indicates which frames in the coded frame buffer are referenced by the current frame, which frames may be referenced by subsequent frames, and which frames are not referenced.

[0052] When the current frame uses the RPS structure, it can use the existing RPS structure in the SPS to write into the SPS Nal Unit data structure. That is, there are 16 predefined RPS structures in the SPS Nal Unit. These 16 predefined RPS structures all assume that the coded frame buffer stores the 16 most recently encoded consecutive frames. RPS structure No. 0 indicates that the current frame x references frame x-1, and other frames in the coded frame buffer may also be referenced by subsequent frames. RPS structure No. 1 indicates that the current frame x references frame x-2, and other frames in the coded frame buffer may also be referenced by subsequent frames. And so on, the current frame references the reference frame at that position, and the corresponding RPS structure number is encoded into the bitstream. When encoding the RPS structure of the current frame X as shown in Figure 2, it is only necessary to encode the corresponding RPS structure number, such as 0.

[0053] Based on the encoded video frames stored in the encoding frame buffer, the present application combines the update timing of the encoded video frames and the update timing of the reliable frames to update the encoding frame buffer in real time.

[0054] Referring to Figure 3, when the network deteriorates, the nearest reliable frame tends to become increasingly distant from the current frame. At this point, because the 16 frames in the coded frame buffer remain continuous, when encoding the frames to be encoded, such as frame sequence 30-40, their RPS structures can still be directly encoded by the position number of the corresponding RPS structure in the referenced SPS. For example, if the network state is detected to be the second network state, and the frame to be encoded with frame number 40 references the nearest reliable frame with frame number 24, then the RPS structure number 15 in the referenced SPS can be directly encoded.

[0055] Referring to Figure 4 , as the network continues to deteriorate, in order to ensure that the most recent reliable frame in the second network state always remains in the coded frame buffer, when preparing to encode frame number 41, frame number 25 is removed from the coded frame buffer, while the most recent reliable frame number 24 is retained.

[0056] Wherein, when the frame sequence of the encoded video frames in the encoding frame buffer is discontinuous, a new data storage structure is created, identification information of the reference frame is stored based on the newly created data storage structure, and the current frame is encoded in combination with the newly created data storage structure.

[0057] As shown in Figure 4, it is understandable that when preparing to encode the frame with frame number 41, the 16 frames in the coded frame buffer are no longer continuous. Therefore, when encoding the corresponding RPS structure for frame number 41, the RPS number in the referenced SPS cannot be directly encoded, but a new RPS structure must be encoded. Assuming that its reference is the most recent reliable frame, the newly created RPS data structure indicates that the frame references frame number 24. At the same time, the video frame with frame number 26-4 in the coded frame buffer may also be referenced by subsequent frames and needs to be retained.

[0058] Furthermore, in the case that the frame sequence of the encoded video frames in the encoding frame buffer is discontinuous, the encoding frame buffer is updated based on a preset maximum reference distance, a minimum number of consecutive frames, and a frame difference.

[0059] Referring to Figure 5, if the network continues to be poor and there has been no recent reliable frame update, the 16 frames in the coded frame buffer are no longer continuous. Therefore, when encoding the corresponding RPS information for frames numbered 41-61, the RPS number in the referenced SPS cannot be directly encoded, but a new RPS structure must be encoded.

[0060] Referring to the last row shown in Figure 5, frame number 61 can still reference frame number 24. In this case, the reference distance is 61-24=35. In practical applications, a maximum reference distance can be set, and when the maximum reference distance is exceeded, a keyframe refresh is required. When frames need to be removed from the coded frame buffer, frames 28, 32, 36, etc. are retained, with an interval of 4 between these frames. This frame interval is defined as the frame difference. Although the frames in the coded frame buffer are no longer continuous, some frames are still continuous, such as 52, 53, 54, 55, 56, 67, 58, 59, and 60, a total of 9 frames. This number of continuous frames is defined as the minimum number of continuous frames. The maximum reference distance, frame difference, and minimum number of continuous frames are set according to actual needs, and the encoded video frames in the coded frame buffer are then adaptively adjusted. Referring to Figure 6, assuming that the most recent reliable frame has not been updated and the number of continuous frames in the coded frame buffer has reached the minimum number of continuous frames, the unreliable frame with the smallest frame number is removed. This enables long-distance frame referencing, maintaining smooth playback even when the network is poor without inserting keyframes, thus avoiding the increased congestion that keyframe insertion can cause. In systems with significant latency, even when the peer response is delayed, the most recent reliable frame can be effectively updated. This allows video encoding to fully utilize a flexible reference mechanism even over high-latency networks, and to smoothly revert to a short-distance reference method when the network improves, enhancing frame referencing flexibility.

[0061] Afterwards, it is detected that the latest reliable frame in the coding frame buffer is the coded video frame in the coding frame buffer that is farthest from the current frame. When a new reliable frame is received, the original latest reliable frame is removed from the coding frame buffer.

[0062] As shown in Figure 7, as the network gradually improves, the most recent reliable frame is updated, maintaining smooth playback without inserting key frames and successfully recovering to a better network state. This avoids the situation where the most recent reliable frame is mistakenly removed from the encoding frame buffer, resulting in an inability to update reliable frames, which would result in a longer overall encoding reference distance and poor encoding quality.

[0063] As the network continues to improve, when the 16 frames in the coded frame buffer become continuous again, the RPS information corresponding to the frame to be coded can be encoded by simply encoding the corresponding RPS number in the SPS. Referring to Figure 8, if the network is still in the second network state at this time and the most recent reliable frame is frame 75, frame 78 references frame 75 and only needs to encode RPS structure number 15 when encoding the RPS information. If the network has returned to the first network state at this time, frame 77 can be directly referenced and only needs to encode RPS structure number 0 when encoding the RPS information.

[0064] Optionally, for situations where the network is neither particularly good nor particularly bad, the third and fourth network states can be set based on the real-time network packet loss rate to adaptively select corresponding reference frames under the corresponding network state. For example, some frames to be encoded can be allowed to reference the most recent frame, and some frames to be encoded can be allowed to reference the most recent reliable frame at intervals to achieve a balance between video quality and video smoothness. The specific reference frame selection strategy can be configured according to actual encoding requirements. The embodiments of this application do not impose fixed restrictions on the specific selection strategy, and they will not be detailed here.

[0065] In the above, by detecting the current network status, a pre-built coded frame cache is queried based on the current network status. The coded frame cache is used to store a set number of coded video frames, which include reliable frames. Reliable frames are video frames that have been successfully decoded by the other end. When the current network status is a first specified state, the most recent frame of the current frame is selected from the coded frame cache as a reference frame. The most recent frame is the coded video frame closest to the current frame. When the current network status is a second specified state, the most recent reliable frame of the current frame is selected from the coded frame cache as a reference frame. The most recent reliable frame is the reliable frame closest to the current frame. Using the above technical means, the coded frame cache is queried based on the current network status. When the first specified state is good, the most recent frame of the current frame is selected as the reference frame. When the second specified state is poor, the most recent reliable frame of the current frame is selected as the reference frame. This ensures that the current frame can select a nearby reference frame, improving encoding quality. At the same time, in the event of network packet loss, by providing the most recent reliable frame, the probability of the current frame being successfully decoded by the other end can be increased, ensuring the reliability of the current frame decoding, thereby improving the stability of video data transmission and enhancing the real-time video communication experience.

[0066] Based on the above embodiment, FIG9 is a schematic diagram of a reference frame determination system for video frame coding provided by this application. Referring to FIG9 , the reference frame determination system for video frame coding provided by this embodiment specifically includes: a query module 21 and a reference module 22.

[0067] The query module 21 is configured to detect the current network status and query a pre-built coded frame cache based on the current network status. The coded frame cache is used to store a set number of coded video frames, wherein the coded video frames include reliable frames, which are video frames that have been successfully decoded by the peer end.

[0068] The reference module 22 is configured to select the most recent frame of the current frame from the coding frame cache as a reference frame when the current network state is a first specified state, where the most recent frame is the encoded video frame closest to the current frame; and to select the most recent reliable frame of the current frame from the coding frame cache as a reference frame when the current network state is a second specified state, where the most recent reliable frame is the reliable frame closest to the current frame.

[0069] Specifically, the reference frame determination system for video frame encoding further includes:

[0070] The coded frame buffer is updated in real time based on the update timing of the coded video frames and the update timing of the reliable frames.

[0071] In the case that the frame sequence of the encoded video frames in the encoding frame buffer is discontinuous, the encoding frame buffer is updated based on a preset maximum reference distance, a minimum number of consecutive frames, and a frame difference.

[0072] It is detected that the latest reliable frame in the coding frame buffer is the coded video frame in the coding frame buffer that is farthest from the current frame. When a new reliable frame is received, the original latest reliable frame is removed from the coding frame buffer.

[0073] Before selecting the nearest reliable frame of the current frame from the coded frame buffer as the reference frame, it also includes:

[0074] At the initial encoding time, the initially encoded key frame is used as the most recent reliable frame and is placed in the encoding frame buffer.

[0075] Specifically, the reference frame determination system for video frame encoding further includes:

[0076] Construct a data storage structure corresponding to a set number. When the frame sequence of the encoded video frames in the encoding frame buffer is continuous, select the data storage structure at the corresponding sequence position according to the sequence position of the reference frame in the encoding frame buffer to store the identification information of the reference frame, and encode the current frame in combination with the data storage structure at the corresponding sequence position. The set number of data storage structures corresponds one-to-one to the encoded video frames in the encoding frame buffer.

[0077] In addition, encoding the current frame based on the selected reference frame also includes:

[0078] When the frame sequence of the encoded video frames in the encoding frame buffer is discontinuous, a new data storage structure is created, identification information of the reference frame is stored based on the newly created data storage structure, and the current frame is encoded in combination with the newly created data storage structure.

[0079] In the above, by detecting the current network status, a pre-built coded frame cache is queried based on the current network status. The coded frame cache is used to store a set number of coded video frames, which include reliable frames. Reliable frames are video frames that have been successfully decoded by the other end. When the current network status is a first specified state, the most recent frame of the current frame is selected from the coded frame cache as a reference frame. The most recent frame is the coded video frame closest to the current frame. When the current network status is a second specified state, the most recent reliable frame of the current frame is selected from the coded frame cache as a reference frame. The most recent reliable frame is the reliable frame closest to the current frame. Using the above technical means, the coded frame cache is queried based on the current network status. When the first specified state is good, the most recent frame of the current frame is selected as the reference frame. When the second specified state is poor, the most recent reliable frame of the current frame is selected as the reference frame. This ensures that the current frame can select a nearby reference frame, improving encoding quality. At the same time, in the event of network packet loss, by providing the most recent reliable frame, the probability of the current frame being successfully decoded by the other end can be increased, ensuring the reliability of the current frame decoding, thereby improving the stability of video data transmission and enhancing the real-time video communication experience.

[0080] The reference frame determination system for video frame encoding provided in the embodiment of the present application can be configured to execute the reference frame determination method for video frame encoding provided in the above embodiment, and has corresponding functions and beneficial effects.

[0081] Based on the above practical example, an embodiment of the present application further provides a reference frame determination device for video frame coding. Referring to Figure 10, the reference frame determination device for video frame coding includes: a processor 31, a memory 32, a communication module 33, an input device 34, and an output device 35. The memory, as a computer-readable storage medium, can be configured to store software programs, computer executable programs, and modules, such as program instructions / modules corresponding to the reference frame determination method for video frame coding described in any embodiment of the present application (for example, the query module and reference module in the reference frame determination system for video frame coding). The communication module is configured to perform data transmission. The processor executes the various functional applications and data processing of the device by running the software programs, instructions, and modules stored in the memory, thereby implementing the above-mentioned reference frame determination method for video frame coding. The input device can be configured to receive input digital or character information and generate key signal input related to user settings and function control of the device. The output device can include a display device such as a display screen. The reference frame determination device for video frame coding provided above can be configured to execute the reference frame determination method for video frame coding provided in the above-mentioned embodiment, and has corresponding functions and beneficial effects.

[0082] Based on the above embodiments, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a computer processor, the computer-executable instructions are configured to execute a method for determining a reference frame for video frame encoding. The storage medium can be any of various types of memory devices or storage devices. Of course, the computer-executable instructions of the computer-readable storage medium provided in an embodiment of the present application are not limited to the method for determining a reference frame for video frame encoding as described above, and can also execute related operations in the method for determining a reference frame for video frame encoding provided in any embodiment of the present application.

[0083] Based on the above embodiments, the embodiments of the present application also provide a computer program product. The technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer program product is stored in a storage medium and includes a number of instructions for enabling a computer device, a mobile terminal or a processor therein to execute all or part of the steps of the reference frame determination method for video frame encoding described in each embodiment of the present application.

Claims

1. A method for determining a reference frame for video frame coding, wherein: include: Detecting the current network status, querying a pre-constructed coding frame cache based on the current network status, wherein the coding frame cache is used to store a set number of coded video frames, wherein the coded video frames include reliable frames, and the reliable frames are video frames that have been successfully decoded by the peer end; When the current network state is the first specified state, selecting a frame closest to the current frame from the coded frame buffer as a reference frame, the latest frame being the coded video frame closest to the current frame; When the current network state is the second specified state, a nearest reliable frame of the current frame is selected from the coding frame buffer as a reference frame, where the nearest reliable frame is the reliable frame closest to the current frame.

2. The method for determining a reference frame for video frame coding according to claim 1, wherein: The method further comprises: The coded frame buffer is updated in real time in combination with the update timing of the coded video frame and the update timing of the reliable frame.

3. The method for determining a reference frame for video frame coding according to claim 2, wherein: The method further comprises: In the case that the frame sequence of the encoded video frames in the encoding frame buffer is discontinuous, the encoding frame buffer is updated based on a preset maximum reference distance, a minimum number of consecutive frames and a frame difference.

4. The method for determining a reference frame for video frame coding according to claim 2, wherein: The method further comprises: It is detected that the most recent reliable frame in the coding frame buffer is the coded video frame in the coding frame buffer that is farthest from the current frame, and when a new reliable frame is received, the original most recent reliable frame is moved out of the coding frame buffer.

5. The method for determining a reference frame for video frame coding according to claim 2, wherein: Before selecting the nearest reliable frame of the current frame from the coding frame buffer as a reference frame, the method further comprises: At the initial encoding time, the initially encoded key frame is used as the most recent reliable frame and is placed in the encoding frame buffer.

6. The method for determining a reference frame for video frame coding according to claim 1, wherein: The method further comprises: Constructing a data storage structure corresponding to the set number, when the frame sequence of the encoded video frames in the encoding frame buffer is continuous, according to the sequence position of the reference frame in the encoding frame buffer, selecting the data storage structure corresponding to the sequence position to store the identification information of the reference frame, encoding the current frame in combination with the data storage structure corresponding to the sequence position, and the set number of data storage structures There is a one-to-one correspondence with the encoded video frames in the encoding frame buffer.

7. The method for determining a reference frame for video frame coding according to claim 6, wherein: The method further comprises: In the case where the frame sequence of the encoded video frames in the encoding frame buffer is discontinuous, a new data storage structure is created, identification information of the reference frame is stored based on the newly created data storage structure, and the current frame is encoded in combination with the newly created data storage structure.

8. A reference frame determination system for video frame coding, wherein: include: A query module is configured to detect a current network state and query a pre-constructed coded frame cache based on the current network state, wherein the coded frame cache is used to store a set number of coded video frames, wherein the coded video frames include reliable frames, and the reliable frames are video frames that have been successfully decoded by the peer end; The reference module is configured to select the nearest frame of the current frame from the coding frame cache as the reference frame when the current network state is a first specified state, and the nearest frame is the encoded video frame closest to the current frame; when the current network state is a second specified state, select the nearest reliable frame of the current frame from the coding frame cache as the reference frame, and the nearest reliable frame is the reliable frame closest to the current frame.

9. A reference frame determination device for video frame encoding, wherein: include: memory and one or more processors; The memory is configured to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the reference frame determination method for video frame encoding as described in any one of claims 1-7.

10. A computer-readable storage medium, wherein: The computer-readable storage medium stores computer-executable instructions, which, when executed by a computer processor, are configured to execute the reference frame determination method for video frame encoding as described in any one of claims 1-7.

11. A computer program product, wherein: The computer program product includes instructions, and when the instructions are executed on a computer or a processor, the computer or the processor executes the reference frame determination method for video frame encoding as described in any one of claims 1-7.

Images