Method and device for recovering reference frame during video decoding

Abstract

The invention discloses a method for recovering a reference frame during video decoding. In a video decoding process, it is found that a current video frame to be decoded lacks a reference frame. And judging whether the decoding frame buffer DPB is empty or not. And if yes and there is a "backup reference frame", using the "backup reference frame" as a "recovered reference frame". If not, a decoded video frame closest to the missing reference frame is found in the DPB to serve as a restored reference frame. And using the'recovered reference frame 'to replace the'missing reference frame', and decoding the current video frame to be decoded. In the execution process of the method, each time a key frame is decoded, the decoded key frame is set as a backup reference frame. According to the method and the device, the problem that the reference frame cannot be found during video decoding is solved with extremely low computing resource and storage resource occupation.

Description

Technical Field

[0001] The present application relates to a digital video decoding technology, and in particular to a method for restoring reference frames during video decoding. Background Art

[0002] A video is composed of a group of video frames (also called picture frames). The content of adjacent video frames in sequence is similar, and only a part of the content changes. Inter-frame prediction utilizes the content correlation of adjacent video frames, and during video decoding, the pixel values of the decoded video frames are used to predict the pixel values of the video frame to be decoded. The decoded video frames used are the reference frames for the video frame to be decoded.

[0003] In order to remove information redundancy in the spatial and temporal domains, intra-frame coding technology and inter-frame coding technology are adopted when encoding an input video frame. According to the adopted coding technology, the encoded video frames can be divided into I frames encoded only by intra-frame coding technology, and P frames and B frames encoded by a combination of intra-frame coding technology and inter-frame coding technology. When an I frame is encoded, it does not refer to other encoded video frames. When P frames and B frames are encoded, they refer to one or more other encoded video frames. Since the coding efficiency of P frames and B frames is relatively high, and I frames have the function of blocking error propagation, generally, I frames are encoded periodically during video coding, and most of the remaining video frames are encoded as P frames or B frames.

[0004] If starting from a certain I frame, the decoding of subsequent video frames does not depend on the decoded video frames before this I frame, then this I frame is called a key frame. All video frames from a key frame until before the next key frame are called a group of pictures (GOP). When a video stream is damaged, it only affects the GOP where the damaged video frame is located and does not spread to other GOPs of non-damaged video frames.

[0005] According to the playback order of the video frame to be decoded and the reference frame, the inter-frame coding technology is divided into forward reference and backward reference. Forward reference means that the playback order of the reference frame is earlier than that of the video frame to be decoded. Backward reference means that the playback order of the reference frame is later than that of the video frame to be decoded. P frames can only perform forward reference. B frames can perform forward reference, or backward reference, or bidirectional reference.

[0006] When there is a backward reference, the reference frame needs to be decoded before the video frame to be decoded and played after the video frame to be decoded. The decoding order and the playback order are not exactly the same. Therefore, there is a re-order process on the video decoder. The re-order process means that after each video frame is decoded in the decoding order, it is not played immediately, but first stored in the decoded picture buffer (DPB), and then taken out from the DPB in the playback order for playback. If a played video frame is still referenced by an undecoded video frame, it remains in the DPB. The played video frame that is no longer referenced is removed from the DPB.

[0007] During video playback, due to reasons such as file corruption and packet loss in network transmission, some video frames may be discarded. This will cause the problem of not being able to find a reference frame when decoding subsequent video frames. The video frames in a GOP often reference each other continuously, which will make a series of video frames in the GOP unable to be decoded normally.

[0008] An existing solution to the problem of not being able to find a reference frame during video decoding is to discard all the video frames to be decoded that have a dependency relationship with the reference frame, that is, no longer decode the "video frames to be decoded that have a dependency relationship with the reference frame", and wait until the next key frame arrives to resume the normal decoding of subsequent video frames. This method can ensure that the new GOP can be decoded normally without errors, but many video frames will be discarded in the GOP where the reference frame cannot be found, resulting in long-term stuttering during video playback.

[0009] Another existing solution to the problem of not being able to find a reference frame during video decoding is to select a video frame adjacent to the lost reference frame for interpolation to create a reference frame to replace the lost reference frame. This method can ensure the normal progress of the video decoding process, but there are also the following disadvantages.

[0010] First, the interpolation algorithm will bring additional computational complexity. On some platforms with limited computing resources, there is not enough computing power to perform the interpolation operation, or the system will stutter during the interpolation operation.

[0011] Second, in some cases, it may not be possible to find a video frame adjacent to the lost reference frame, so interpolation cannot be performed.

[0012] Third, the DPB is located in the system memory, and the decoded video frames in the DPB need to go through multiple read and write operations during the playback or reference process. To save memory space and read / write bandwidth, the video decoder uses framebuffer compression (FBC) technology to access the decoded video frames in a lossless compression manner. If the reference frames are stored in a compressed manner, the adjacent frames need to be decompressed first, then interpolated, and then compressed again, which is a more cumbersome process.

[0013] Fourth, when playing videos using digital right management (DRM) technology, the CPU is not allowed to access the content of the video frames, so this method cannot be applied. Summary of the Invention

[0014] The technical problem to be solved by this application is: how to solve the problem of not finding a reference frame during video decoding, and it is required that the video playback is not stuck, the calculation amount is small, and the applicable range is wide.

[0015] To solve the above technical problem, this application proposes a method for restoring reference frames during video decoding, which includes the following steps. Step S11: During the video decoding process, it is found that the current video frame to be decoded lacks a reference frame. Step S12: Determine whether the decoding frame buffer DPB is empty. If it is, go to step S13. If not, find the decoded video frame closest to the "missing reference frame" in the DPB as the "restored reference frame", and set the "restored reference frame" as the "backup reference frame", and then go to step S14. Step S13: Determine whether there is a "backup reference frame". If there is, use the "backup reference frame" as the "restored reference frame", and then go to step S14. If not, discard the current video frame to be decoded and stop decoding the current video frame to be decoded. Step S14: Use the "restored reference frame" to replace the "missing reference frame" and decode the current video frame to be decoded. During the execution of the method, each time a key frame is decoded, the decoded key frame is set as the "backup reference frame".

[0016] Further, in step S12, if the DPB is not empty and multiple decoded video frames simultaneously meet the condition of being closest to the "missing reference frame", select the decoded video frame with a higher playback order as the "restored reference frame".

[0017] Further, during the execution of the method, the update of the "backup reference frame" only changes in the following two cases: one is when a reference frame is missing again, update the "backup reference frame" with "a decoded video frame closest to the'missing reference frame'"; the other is when a key frame appears, update the "backup reference frame" with the decoded key frame.

[0018] Furthermore, the "restored reference frame" and the "backed-up reference frame" are saved in any of the following ways. Method 1: When a decoded video frame exists in the memory and is used as the "restored reference frame" or the "backed-up reference frame", instead of copying it to another storage location, it is used as a reference to extend its life cycle, and its storage space is released only when it is no longer useful. Method 2: When a decoded video frame exists in the memory and is used as the "restored reference frame" or the "backed-up reference frame", it is copied to another storage location, so there are multiple copies of the same content in the memory, but their life cycles do not affect each other.

[0019] This application also proposes a method for restoring reference frames during video decoding, which includes the following steps. Step S21: After the video starts playing, backup the decoded video frames as "backed-up reference frames" according to a preset "backup interval". Step S22: During the video decoding process, it is found that the current video frame to be decoded lacks a reference frame. Step S23: Determine whether the decoding frame buffer DPB is empty. If it is, use the "backed-up reference frame" as the "restored reference frame". If not, find the decoded video frame closest to the "missing reference frame" in the DPB as the "restored reference frame", and set the "restored reference frame" as the "backed-up reference frame". Step S24: Use the "restored reference frame" to replace the "missing reference frame" and decode the current video frame to be decoded. During the execution of the method, each time a key frame is decoded, the decoded key frame is set as the "backed-up reference frame".

[0020] Furthermore, in step S23, if the DPB is not empty and multiple decoded video frames simultaneously meet the condition of being closest to the "missing reference frame", select the decoded video frame with a higher playback order as the "restored reference frame".

[0021] Furthermore, during the execution of the method, the update of the "backed-up reference frame" only changes in the following three cases: one is to select a decoded video frame to update the "backed-up reference frame" according to the preset "backup interval"; the second is when a lack of a reference frame appears again, update the "backed-up reference frame" with the decoded video frame closest to the "missing reference frame"; the third is when a key frame appears, update the "backed-up reference frame" with the decoded key frame.

[0022] Further, the "restored reference frame" and the "backed-up reference frame" are saved in any of the following ways. Method 1: When a decoded video frame exists in the memory and is used as the "restored reference frame" or the "backed-up reference frame", instead of copying it to another storage location, it is used as a reference to extend its life cycle, and its storage space is released only when it is no longer used for any purpose. Method 2: When a decoded video frame exists in the memory and is used as the "restored reference frame" or the "backed-up reference frame", it is copied to another storage location, so there are multiple copies of the same content in the memory, but their life cycles do not affect each other.

[0023] The present application also proposes a device for restoring a reference frame during video decoding, including a discovery unit, a first processing unit, a second processing unit, and a restoration unit. The discovery unit is configured to discover that the current video frame to be decoded lacks a reference frame during the video decoding process. The first processing unit is configured to determine whether the DPB is empty; if so, transfer the processing to the second processing unit; if not, find a decoded video frame closest to the "missing reference frame" in the DPB as the "restored reference frame", and set the "restored reference frame" as the "backed-up reference frame", and then transfer the processing to the restoration unit; the first processing unit also sets the decoded key frame as the "backed-up reference frame" every time a key frame is decoded. The second processing unit is configured to determine whether there is a "backed-up reference frame"; if so, use the "backed-up reference frame" as the "restored reference frame", and then transfer the processing to the restoration unit; if not, discard the current video frame to be decoded and stop decoding the current video frame to be decoded. The restoration unit is configured to use the "restored reference frame" to replace the "missing reference frame" and decode the current video frame to be decoded.

[0024] The present application also proposes a device for restoring a reference frame during video decoding, including a periodic backup unit, a discovery unit, a third processing unit, and a restoration unit. The periodic backup unit is configured to backup the decoded video frame as the "backed-up reference frame" according to a preset "backup interval" after the video starts playing. The discovery unit is configured to discover that the current video frame to be decoded lacks a reference frame during the video decoding process. The third processing unit is configured to determine whether the DPB is empty; if so, use the "backed-up reference frame" as the "restored reference frame"; if not, find a decoded video frame closest to the "missing reference frame" in the DPB as the "restored reference frame", and set the "restored reference frame" as the "backed-up reference frame"; the third processing unit also sets the decoded key frame as the "backed-up reference frame" every time a key frame is decoded. The restoration unit is configured to use the "restored reference frame" to replace the "missing reference frame" and decode the current video frame to be decoded.

[0025] The technical effect achieved by this application is that it solves the problem of not being able to find a reference frame during video decoding with extremely low occupancy of computing resources and storage resources. Description of the Drawings

[0026] Figure 1 FIG. is a schematic flowchart of the first embodiment of the method for restoring a reference frame during video decoding proposed by this application.

[0027] Figure 2 FIG. is a schematic flowchart of the second embodiment of the method for restoring a reference frame during video decoding proposed by this application.

[0028] Figure 3 FIG. is a schematic structural diagram of the first embodiment of the apparatus for restoring a reference frame during video decoding proposed by this application.

[0029] Figure 4 FIG. is a schematic structural diagram of the second embodiment of the apparatus for restoring a reference frame during video decoding proposed by this application.

[0030] Explanation of the reference numerals in the figures: Discovery unit 11, First processing unit 12, Second processing unit 13, Restoration unit 14, Periodic backup unit 21, Discovery unit 22, Third processing unit 23, Restoration unit 24. Detailed Embodiments

[0031] Please refer to Figure 1 , the first embodiment of the method for restoring a reference frame during video decoding proposed by this application includes the following steps.

[0032] Step S11: During video decoding, it is discovered that the current video frame to be decoded lacks a reference frame.

[0033] Step S12: Determine whether the DPB is empty.

[0034] If it is, proceed to step S13.

[0035] If not, find the decoded video frame closest to the "missing reference frame" in the DPB as the "restored reference frame", and set the "restored reference frame" as the "backed-up reference frame". Subsequently, proceed to step S14. If multiple decoded video frames simultaneously meet the condition of being closest to the "missing reference frame", for example, being located before and after the "missing reference frame" and having the same distance from the "missing reference frame" (i.e., the same number of video frames apart), then select the decoded video frame with a higher playback order as the "restored reference frame".

[0036] In this step, any of the following methods can be used to find the decoded video frame closest to the "missing reference frame" in the DPB. Method 1: Arrange all the decoded video frames in the DPB in the decoding order, and then find the decoded video frame with the smallest interval from the "missing reference frame". Method 2: Arrange all the decoded video frames in the DPB in the playback order, and then find the decoded video frame with the smallest interval from the "missing reference frame".

[0037] Step S13: Determine whether there is a "backup reference frame".

[0038] If yes, use the "backup reference frame" as the "restored reference frame". Then proceed to step S14.

[0039] If no, it means that the "missing reference frame" cannot be restored. Discard the current video frame to be decoded and stop decoding the current video frame to be decoded. Then start the decoding process for the next video frame.

[0040] Step S14: Use the "restored reference frame" to replace the "missing reference frame" and decode the current video frame to be decoded.

[0041] During the execution of the above Embodiment 1, each time a key frame is decoded, the decoded key frame is set as the "backup reference frame". In any step, if there is already a "restored reference frame" when setting the "restored reference frame", update the content of the "restored reference frame". If there is already a "backup reference frame" when setting the "backup reference frame", update the content of the "backup reference frame".

[0042] In the above Embodiment 1, the update of the "backup reference frame" only changes in the following two cases. One is when a reference frame is missing again, update the "backup reference frame" with "the decoded video frame closest to the'missing reference frame'". The other is when a key frame appears, update the "backup reference frame" with the decoded key frame.

[0043] Please refer to Figure 2 , the second embodiment of the method for restoring reference frames during video decoding proposed in this application includes the following steps.

[0044] Step S21: After the video starts playing, back up the decoded video frames as the "backup reference frame" according to the preset "backup interval". For example, if 12 video frames are decoded per second, back up one video frame every 6 (i.e., every 5 intervals), or every 12 (i.e., every 11 intervals) decoded video frames.

[0045] Step S22: During the video decoding process, it is found that the current video frame to be decoded lacks a reference frame.

[0046] Step S23: Determine whether the DPB is empty.

[0047] If it is, use the "backup reference frame" as the "restored reference frame". Subsequently, proceed to step S24.

[0048] If not, find the decoded video frame in the DPB that is closest to the "missing reference frame" as the "restored reference frame", and set the "restored reference frame" as the "backup reference frame". Subsequently, proceed to step S24. If there are multiple decoded video frames that simultaneously satisfy being the closest to the "missing reference frame" (i.e., having the same number of video frames in between), then select the decoded video frame with a higher playback order as the "restored reference frame".

[0049] In this step, finding the decoded video frame in the DPB that is closest to the "missing reference frame" can be done in any of the following ways. Method 1: Arrange all the decoded video frames in the DPB in the decoding order, and then find the decoded video frame with the smallest interval from the "missing reference frame". Method 2: Arrange all the decoded video frames in the DPB in the playback order, and then find the decoded video frame with the smallest interval from the "missing reference frame".

[0050] Step S24: Use the "restored reference frame" to replace the "missing reference frame" and decode the current video frame to be decoded.

[0051] During the execution of the above-mentioned second embodiment, each time a key frame is decoded, the decoded key frame is set as the "backup reference frame". In any step, if there is already a "restored reference frame" when setting the "restored reference frame", then update the content of the "restored reference frame". If there is already a "backup reference frame" when setting the "backup reference frame", then update the content of the "backup reference frame".

[0052] In the above-mentioned second embodiment, the update of the "backup reference frame" only changes in the following three cases. One is to select a decoded video frame according to the preset "backup interval" to update the "backup reference frame". The second is when a missing reference frame appears again, use the "decoded video frame closest to the'missing reference frame'" to update the "backup reference frame". The third is when a key frame appears, use the decoded key frame to update the "backup reference frame".

[0053] In the above two embodiments, the "restored reference frame" and the "backed-up reference frame" can be saved in any of the following ways. Method 1: When the decoded video frame exists in the memory (for example, the DPB), and it is used as the "restored reference frame" or the "backed-up reference frame", instead of copying it to another storage location, it is used as a reference to extend its lifecycle, and its storage space is released only when it is no longer used for any purpose. Method 2: When the decoded video frame exists in the memory (for example, the DPB), and it is used as the "restored reference frame" or the "backed-up reference frame", it is copied to another storage location, such as the specifically set "storage location A for the restored reference frame" and "storage location B for the backed-up reference frame". In this way, there are multiple copies of the same content in the memory, but their lifecycles do not affect each other. Both of these saving methods are applicable to the present invention, and the difference lies in the occupation of memory space and the resource consumption during content copying.

[0054] Please refer to Figure 3 , Embodiment 1 of the apparatus for restoring reference frames during video decoding proposed in this application includes a discovery unit 11, a first processing unit 12, a second processing unit 13, and a restoration unit 14. Figure 3 The shown apparatus corresponds to Figure 1 the shown method.

[0055] The discovery unit 11 is used to discover that the current video frame to be decoded lacks a reference frame during the video decoding process.

[0056] The first processing unit 12 is used to determine whether the DPB is empty. If it is, the processing is transferred to the second processing unit 13. If not, find the decoded video frame closest to the "missing reference frame" in the DPB as the "restored reference frame", and set the "restored reference frame" as the "backed-up reference frame". Subsequently, the processing is transferred to the restoration unit 14. The first processing unit 12 also sets the decoded key frame as the "backed-up reference frame" every time a key frame is decoded.

[0057] The second processing unit 13 is used to determine whether there is a "backed-up reference frame". If there is, use the "backed-up reference frame" as the "restored reference frame". Subsequently, the processing is transferred to the restoration unit 14. If not, it means that the "missing reference frame" cannot be restored, and the current video frame to be decoded is discarded without further decoding of the current video frame to be decoded.

[0058] The restoration unit 14 is used to replace the "missing reference frame" with the "restored reference frame" and decode the current video frame to be decoded.

[0059] Please refer to Figure 4, Embodiment 2 of the apparatus for restoring reference frames during video decoding proposed in this application includes a periodic backup unit 21, a discovery unit 22, a third processing unit 23, and a restoration unit 24. Figure 4 The apparatus shown corresponds to Figure 2 the method shown. Figure 4 The discovery unit 22 in Figure 3 is equivalent to the discovery unit 11 in Figure 4 The restoration unit 24 in Figure 3 is equivalent to the restoration unit 14 in

[0060] The periodic backup unit 21 is configured to, after the video starts playing, backup the decoded video frames as "backup reference frames" according to a preset "backup interval".

[0061] The discovery unit 22 is configured to, during the video decoding process, discover that the current video frame to be decoded lacks a reference frame.

[0062] The third processing unit 23 is configured to determine whether the DPB is empty. If so, use the "backup reference frame" as the "restoration reference frame". Subsequently, it is transferred to the restoration unit 24 for processing. If not, find the decoded video frame closest to the "missing reference frame" in the DPB as the "restoration reference frame", and set the "restoration reference frame" as the "backup reference frame". Subsequently, it is transferred to the restoration unit 24 for processing. The third processing unit 23 also sets the decoded key frame as the "backup reference frame" each time a key frame is decoded.

[0063] The restoration unit 24 is configured to use the "restoration reference frame" to replace the "missing reference frame" and decode the current video frame to be decoded.

[0064] Compared with the prior art, this application has the following beneficial effects.

[0065] First, the computational complexity is low, with basically no additional computational workload added, and the final result can be obtained simply based on a small amount of logical judgment and comparison.

[0066] Second, the two embodiments are oriented towards different application scenarios, and the main difference lies in the different update frequencies of the "backup reference frames".

[0067] Embodiment 1 does not require periodic backup of decoded frames (decoded video frames), which is a demand-driven update method. It has less occupation of storage resources and less impact on power consumption, but the restored video quality may not be good. Embodiment 1 is suitable for scenarios where storage resources or power consumption are relatively tight, such as battery-powered mobile devices (smartphones, smartwatches, etc.).

[0068] Example 2 requires regular backup of decoded frames, which is an active update method. The impact on storage resources and power consumption depends on its update cycle. If the update cycle is short, the resource occupancy is large and the power consumption impact is significant, but the recovery quality is higher. If the update cycle is long, the resource occupancy is small and the power consumption impact is minor, but the recovery quality is low. Example 2 is applicable to scenarios where storage resources or power consumption are relatively abundant, such as smart TVs, conference systems, etc.

[0069] Third, every time a key frame is decoded, the decoded key frame is used to update the "backup reference frame" to avoid the mismatch of the reference frame content caused by scene switching, thereby improving the recovery quality.

[0070] Fourth, it is applicable to reference frames in FBC format without the need for FBC decompression, interpolation calculation, and recompression of video frames.

[0071] Fifth, the CPU does not need to access and edit the content of video frames and is applicable to playing videos using DRM technology.

[0072] The above are only the preferred embodiments of the present application and are not used to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the protection scope of the present application.

Claims

1. A method for reference frame recovery during video decoding, characterized in that It includes the following steps; Step S11: During video decoding, it is found that the current video frame to be decoded lacks a reference frame; Step S12: Determine whether the decoded picture buffer (DPB) is empty; If it is, go to Step S13; If not, find the decoded video frame in the DPB that is closest to the "missing reference frame" as the "restored reference frame", and set the "restored reference frame" as the "backup reference frame", then go to Step S14; Step S13: Determine whether there is a "backup reference frame"; If it is, use the "backup reference frame" as the "restored reference frame", then go to Step S14; If not, discard the current video frame to be decoded and stop decoding the current video frame to be decoded; Step S14: Use the "restored reference frame" to replace the "missing reference frame" and decode the current video frame to be decoded; During the execution of the method, each time a key frame is decoded, the decoded key frame is set as the "backup reference frame".

2. The method for restoring a reference frame during video decoding according to claim 1, wherein In Step S12, if the DPB is not empty and there are multiple decoded video frames that are equally close to the "missing reference frame", select the decoded video frame with a higher playback order as the "restored reference frame".

3. The method for reference frame recovery during video decoding according to claim 1, wherein During the execution of the method, the update of the "backup reference frame" only changes in the following two cases: one is when a reference frame is missing again, the "backup reference frame" is updated with "a decoded video frame that is closest to the'missing reference frame'"; The other is when a key frame appears, the decoded key frame is used to update the "backup reference frame".

4. The method for restoring a reference frame during video decoding according to claim 1, characterized in that, The "restored reference frame" and the "backup reference frame" are saved in any of the following ways; Method 1: When the decoded video frame exists in the memory and is used as the "restored reference frame" or the "backup reference frame", it is not copied to another storage location, but is used as a reference to extend its life cycle, and its storage space is only released when it is no longer used for any purpose; Method 2: When the decoded video frame exists in the memory and is used as the "restored reference frame" or the "backup reference frame", it is copied to another storage location, so there are multiple copies of the same content in the memory, but their life cycles do not affect each other.

5. A method for reference frame recovery during video decoding, characterized in that, It includes the following steps; Step S21: After the video starts playing, backup the decoded video frame as the "backup reference frame" according to the preset "backup interval"; Step S22: During video decoding, it is found that the current video frame to be decoded lacks a reference frame; Step S23: Determine whether the decoded picture buffer (DPB) is empty; If it is, use the "backup reference frame" as the "restored reference frame"; If not, find the decoded video frame in the DPB that is closest to the "missing reference frame" as the "restored reference frame", and set the "restored reference frame" as the "backup reference frame"; Step S24: Use the "restored reference frame" to replace the "missing reference frame" and decode the current video frame to be decoded; During the execution of the method, each time a key frame is decoded, the decoded key frame is set as the "backup reference frame".

6. The method for restoring a reference frame during video decoding according to claim 5, wherein In step S23, if the DPB is not empty and there are multiple decoded video frames that simultaneously satisfy being the closest to the "missing reference frame", select the decoded video frame with a higher playback order as the "restored reference frame".

7. The method for restoring a reference frame during video decoding according to claim 5, characterized in that, During the execution of the method, the update of the "backup reference frame" only changes in the following three cases: First, select a decoded video frame according to the preset "backup interval" to update the "backup reference frame"; second, when a reference frame is missing again, use "a decoded video frame that is the closest to the'missing reference frame'" to update the "backup reference frame"; Third, when a key frame appears, use the decoded key frame to update the "backup reference frame".

8. The method for restoring a reference frame during video decoding according to claim 5, characterized in that The "restored reference frame" and the "backup reference frame" are saved in any of the following ways; Method 1: When the decoded video frame exists in the memory and is used as the "restored reference frame" or the "backup reference frame", instead of copying it to another storage location, it is used as a reference to extend its life cycle, and its storage space is only released when it is no longer used for any purpose; Method 2: When the decoded video frame exists in the memory and is used as the "restored reference frame" or the "backup reference frame", it is copied to another storage location, so there are multiple copies of the same content in the memory, but their life cycles do not affect each other.

9. A device for restoring reference frames during video decoding, characterized in that, It includes a discovery unit, a first processing unit, a second processing unit, and a restoration unit; The discovery unit is used to discover that the current video frame to be decoded lacks a reference frame during video decoding; The first processing unit is used to determine whether the DPB is empty; if so, transfer the processing to the second processing unit; If not, find a decoded video frame that is the closest to the "missing reference frame" in the DPB as the "restored reference frame", and set the "restored reference frame" as the "backup reference frame", and then transfer the subsequent processing to the restoration unit; the first processing unit also sets the decoded key frame as the "backup reference frame" every time a key frame is decoded; The second processing unit is used to determine whether there is a "backup reference frame"; If so, use the "backup reference frame" as the "restored reference frame", and then transfer the subsequent processing to the restoration unit; if not, discard the current video frame to be decoded and no longer decode the current video frame to be decoded; The restoration unit is used to use the "restored reference frame" to replace the "missing reference frame" and decode the current video frame to be decoded.

10. A device for restoring reference frames during video decoding, characterized in that, It includes a periodic backup unit, a discovery unit, a third processing unit, and a restoration unit; The periodic backup unit is used to backup the decoded video frame as the "backup reference frame" according to the preset "backup interval" after the video starts playing; The discovery unit is used to discover that the current video frame to be decoded lacks a reference frame during video decoding; The third processing unit is used to determine whether the DPB is empty; if so, use the "backup reference frame" as the "restored reference frame"; if not, find the decoded video frame closest to the "missing reference frame" in the DPB as the "restored reference frame", and set the "restored reference frame" as the "backup reference frame"; the third processing unit also sets the decoded key frame as the "backup reference frame" every time a key frame is decoded; The restoration unit is used to replace the "missing reference frame" with the "restored reference frame" and decode the current video frame to be decoded.

Images