Video data storage method, device, electronic device and readable storage medium

Abstract

The embodiment of the present application provides a video data storage method, device, electronic device and readable storage medium, the method comprising: in response to decoding processing of a video frame to be reconstructed, when there is no reference video frame of the video frame to be reconstructed in the external storage of the video decoder, and the reference video frame exists in the main storage outside the video decoder, reading the reference video frame from the main storage; storing the read reference video frame in the internal cache of the video decoder. The method of the present application can reduce the power consumption of the electronic device.

Description

Technical Field

[0001] The present application relates to the field of video processing technology, and in particular to a video data storage method, device, electronic device and computer-readable storage medium. Background Art

[0002] With the development of new video standards, such as MPEG-1 / 2 / 4 / 5, H.263 / 264 / 265 / 266, VP8 / 9, etc., the video screen size is getting larger and larger, and the video frame rate is getting higher and higher, resulting in the problem of high power consumption of electronic devices in traditional video data storage methods. Summary of the invention

[0003] The embodiments of the present application provide a video data storage method, device, electronic device, and computer-readable storage medium, which can reduce the power consumption of the electronic device.

[0004] A video data storage method, comprising:

[0005] In response to decoding processing of the video frame to be reconstructed, if a reference video frame of the video frame to be reconstructed does not exist in an external storage of the video decoder, and the reference video frame exists in a main storage outside the video decoder, reading the reference video frame from the main storage; and

[0006] The reference video frame in the main storage is stored in an internal buffer of the video decoder.

[0007] A video data reading method, the method comprising:

[0008] In response to decoding processing of the acquired video code stream, a reference video block not existing in the external storage of the video decoder is read from the main storage outside the video decoder to the internal cache of the video decoder and then motion compensation or block copy is performed;

[0009] Alternatively, the reference video block is read from an external storage of the video decoder and then motion compensation or block copying is performed.

[0010] A video data storage device, comprising:

[0011] a first reading module, configured to, in response to decoding processing of the video frame to be reconstructed, read the reference video frame from the main storage when there is no reference video frame of the video frame to be reconstructed in the external storage of the video decoder and the reference video frame exists in the main storage outside the video decoder; and

[0012] The reference video frame storage module is used to store the reference video frame read from the main storage into the internal cache of the video decoder.

[0013] A video data reading device, comprising:

[0014] A second reading module is used for reading a reference video block that does not exist in the external storage of the video decoder from the main storage outside the video decoder to the internal cache of the video decoder in response to the decoding process of the acquired video code stream, and then performing motion compensation or block copying;

[0015] The third reading module is used for reading the reference video block from the external storage of the video decoder and then performing motion compensation or block copying.

[0016] An electronic device comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of each method.

[0017] A computer-readable storage medium stores a computer program, which implements the steps of each method when executed by a processor.

[0018] The above-mentioned video data storage method, device, electronic device and readable storage medium, in response to the decoding process of the video frame to be reconstructed, when there is no reference video frame of the video frame to be reconstructed in the external storage of the video decoder, and there is a reference video frame in the main storage outside the video decoder, the reference video frame is read from the main storage, and the reference video frame read from the main storage is stored in the internal cache of the video decoder; that is, the video data existing in the external storage of the video decoder is not stored in the internal cache of the video decoder, and the video data in the main storage is stored in the internal cache of the video decoder. Compared with the traditional video data, whether it is located in the external storage or the main storage, it will be placed in the internal cache of the decoder by the decoder. In this embodiment, only the video data that does not exist in the external storage and exists in the main storage needs to be placed in the internal cache of the decoder, which reduces the use of the internal cache of the video decoder, so that the efficiency of the internal cache of the decoder is increased, and the power consumption is further reduced specifically for the data that is still only left in the main storage. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 A schematic diagram of the architecture of a traditional video data storage method in one embodiment;

[0021] Figure 2A conventional video decoder data processing architecture in one embodiment;

[0022] Figure 3 A schematic diagram of the architecture of a traditional video data storage method in another embodiment;

[0023] Figure 4 A schematic diagram of a curve showing the power consumption of a system in a conventional manner in one embodiment;

[0024] Figure 5 is a flowchart of a video data storage method in one embodiment;

[0025] Figure 6 A schematic diagram of a video data storage architecture in one embodiment;

[0026] Figure 7 A schematic diagram of an architecture using a system cache in one embodiment;

[0027] Figure 8 A schematic diagram of an architecture using a system buffer in one embodiment;

[0028] Fig. 9 A schematic diagram of a decoding process in an embodiment;

[0029] Fig.10 is a flowchart of a method for reading video data in one embodiment;

[0030] Fig.11 A schematic diagram of a process of reconstructing a video frame in one embodiment;

[0031] Fig.12 is a schematic diagram of a storage method of a conventional video decoder in an embodiment;

[0032] Fig.13 is a schematic diagram of a storage method of internal storage of a video decoder in one embodiment;

[0033] Fig.14 A structural block diagram of a video data storage device according to an embodiment;

[0034] Fig.15 is a structural block diagram of a video data reading device according to an embodiment;

[0035] Fig.16 FIG. 1 is a schematic diagram of the internal structure of an electronic device in an embodiment. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0037] It is understood that the terms "first", "second", etc. used in this application can be used in this article to describe various data, but these data are not limited by these terms. These terms are only used to distinguish the first data from another data. For example, without departing from the scope of this application, the first number condition can be called the second number condition, and similarly, the second number condition can be called the first number condition. Both the first number condition and the second number condition are number conditions, but they are not the same number condition.

[0038] In one embodiment, conventional electronic devices basically use DRAM as the storage space for video frames. Figure 1 FIG. 1 is a schematic diagram of the architecture of a traditional video data storage method in one embodiment. Figure 1 The CPU includes a central processing unit (CPU), a video decoder (VDEC), a display processor (Display Processing Unit) and a dynamic random access memory controller (DRAM Controller, DRAMC). The CPU includes a master port (master), the video decoder includes an internal cache (cache) and a master port (master), the display processor includes a master port (master), and the dynamic random access memory controller includes a slave port (slave). Figure 1 The middle arrow represents video data. When video data exists, the video decoder can obtain the video data from the dynamic random access memory controller for video decoding, reconstruct the video frame, and transmit the reconstructed video frame to the display processor for display.

[0039] Figure 2 The conventional video decoder data processing architecture in one embodiment is shown. Video bitstreams are input into the video decoder for decoding, and a video frame (frame buffer) and some temporary data (temporary data) such as temporal motion vector (TMV) are generated. The video bitstreams, video frames and temporary data are all stored in DRAM.

[0040] Traditional video decoders can have corresponding cache strategies for processing various video bitstreams. However, with the emergence of new video standards such as H.265 / 266, AV, MPEG-5, etc., for videos with increasingly larger sizes and higher frame rates, traditional technologies use Figure 3 The architecture shown in the figure can widen the DRAM bandwidth or increase the DRAM frequency to accelerate data throughput. Figure 3 The schematic diagram of the architecture of a conventional video data storage method in another embodiment includes a central processing unit, a video decoder, a display processor, and a multi-channel dynamic random access memory controller (Multi-channel DramCtrl), wherein the multi-channel dynamic random access memory controller can be used to control multiple dynamic random access memories (DRAM).

[0041] Figure 4 FIG. 1 is a schematic diagram of a curve of system power consumption in a traditional manner in an embodiment. Figure 4 The horizontal axis is the position in the image (Picture Position), and the vertical axis is the system power consumption (system power). The dotted line represents the upper limit of system power consumption (System Power Barrier). Figure 4 As shown in the figure, the system DRAM needs to consume a high system power consumption to meet the speed requirements of the video decoder. When the video decoder is overly dependent on DRAM or other power-consuming storage, the video decoder speed may not meet the requirements or the system may overheat due to the limited power consumption limit that the system can provide. The traditional method will cause the video decoder to consume a high power consumption when the expected time is completed.

[0042] In order to solve the problem of high power consumption of electronic devices, an embodiment of the present application provides a video data storage method, which responds to the decoding processing of the video frame to be reconstructed, and when there is no reference video frame of the video frame to be reconstructed in the external storage of the video decoder, and when there is a reference video frame in the main storage outside the video decoder, reads the reference video frame from the main storage; stores the reference video frame read from the main storage in the internal cache of the video decoder; compared with traditional video data, no matter it is located in the external storage or the main storage, it will be placed in the internal cache of the decoder by the decoder. In this embodiment, only the video data that does not exist in the external storage but exists in the main storage needs to be placed in the internal cache of the decoder, which reduces the use of the internal cache of the video decoder, increases the efficiency of the internal cache of the decoder, and further reduces the power consumption of the data that is still only left in the main storage.

[0043] The video data storage method is described below in conjunction with a specific embodiment. In the embodiment of the present application, the main storage is DRAM as an example for description. Figure 5 FIG. 1 is a flow chart of a method for storing video data in an embodiment. The method for storing video data in this embodiment is described by taking the method running on an electronic device as an example. Figure 5 As shown, the video data storage method includes steps 502 to 504.

[0044] Step 502, in response to decoding processing of the video frame to be reconstructed, if there is no reference video frame of the video frame to be reconstructed in the external storage of the video decoder, and there is a reference video frame in the main storage outside the video decoder, read the reference video frame from the main storage.

[0045] Among them, the video frame to be reconstructed refers to a video frame that has not been successfully constructed. After the video is compressed, if a video frame is needed, decoding processing is required. The external storage of the video decoder refers to the storage located in the electronic device and other than the main storage and the internal cache of the video decoder. For example, the external storage of the video decoder can be at least one of the system cache and the system buffer. Main storage refers to main memory (Main memory), referred to as main memory. Main storage is an important component of computer hardware. Its function is to store instructions and data, and can be directly randomly accessed by the central processing unit (CPU). The main storage can specifically be DRAM (Dynamic Random Access Memory).

[0046] The reference video frame can be used to reconstruct the target video frame corresponding to the video frame to be reconstructed. The reference video frame can be a video frame displayed before or after the video frame to be reconstructed. The reference video frame is specifically a video frame that has been reconstructed before the video frame to be reconstructed, or the reference video frame can also be a reconstructed part of the video frame to be reconstructed, such as the block vector used for intra block copy in the screen content coding technology to distinguish the range of the picture to be used. The provisions for the reference picture range allowed in the screen content coding technology have been detailed in various video standards and will not be repeated here.

[0047] Specifically, in response to the decoding process of the video frame to be reconstructed, the electronic device obtains the target motion vector or block vector corresponding to the video frame to be reconstructed. It is detected whether there is a reference video frame of the video frame to be reconstructed in the external storage of the video decoder. If there is no reference video frame corresponding to the target motion vector or block vector in the external storage of the video decoder, and there is a reference video frame in the main storage outside the video decoder, the electronic device reads the reference video frame from the main storage, and stores the read reference video frame in the internal cache of the video decoder.

[0048] Step 504: store the reference video frame read from the main storage into the internal buffer of the video decoder.

[0049] Specifically, the electronic device stores the reference video frame read from the main storage into the internal cache of the video decoder.

[0050] The video data storage method in this embodiment, in response to the decoding process of the video frame to be reconstructed, when there is no reference video frame of the video frame to be reconstructed in the external storage of the video decoder, and there is a reference video frame in the main storage outside the video decoder, reads the reference video frame from the main storage, and stores the reference video frame read from the main storage into the internal cache of the video decoder, that is, the video data existing in the external storage of the video decoder is not stored in the internal cache of the video decoder, and the video data in the main storage is stored in the internal cache of the video decoder. Compared with the traditional video data, whether it is located in the external storage or the main storage, it will be placed in the internal cache of the decoder by the decoder. In this embodiment, only the video data that does not exist in the external storage and exists in the main storage needs to be placed in the internal cache of the decoder, which reduces the use of the internal cache of the video decoder, so that the efficiency of the internal cache of the decoder is increased, and the power consumption is further reduced specifically for the data that is still only left in the main storage.

[0051] In one embodiment, Figure 6 The figure is a schematic diagram of the architecture of video data storage in one embodiment. The white part indicates storage in DRAM, that is, this part of data will be stored in the internal cache of the video decoder later, and the gray part indicates storage in the external storage of the video decoder. Figure 3 Storage method, Figure 6 The video bit stream, video frame data, and temporary data are partially stored in DRAM, and partially stored in the external storage of the video decoder.

[0052] In one embodiment, each video frame is divided into a plurality of blocks. When there is no reference video frame of the to-be-reconstructed video frame in the external storage of the video decoder, and there is a reference video frame in the main storage outside the video decoder, reading the reference video frame from the main storage includes:

[0053] When the reference video block corresponding to the to-be-reconstructed video frame does not exist in the external storage of the video decoder and the reference video block exists in the main storage outside the video decoder, the reference video block is read from the main storage.

[0054] Storing the reference video frame read from the main storage in the internal cache of the video decoder includes: storing the reference video block read from the main storage in the internal cache of the video decoder.

[0055] The block refers to a partial area of ​​a video frame. For example, the size of the block may be 16×16 pixels, 64×64 pixels, etc., but is not limited thereto.

[0056] Specifically, in response to the decoding process of the acquired video, the electronic device divides each video frame in the video into a plurality of blocks and obtains the motion vector corresponding to each block. The electronic device determines the reference video block in the reference video frame referenced by the video frame to be reconstructed based on the motion vector of the current block in the current video frame. If the reference video block corresponding to the video frame to be reconstructed does not exist in the external storage of the video decoder, and the reference video block exists in the main storage outside the video decoder, the electronic device reads the reference video block from the main storage. The electronic device stores the reference video block read from the main storage in the internal cache of the video decoder.

[0057] In this embodiment, when there is no reference video block corresponding to the video frame to be reconstructed in the external storage of the video decoder, and there is a reference video block in the main storage outside the video decoder, the electronic device reads the reference video block from the main storage and stores it in the internal cache of the video decoder. Since the internal cache in the video decoder is relatively small, the use of storage blocks is more suitable for the internal cache in the video decoder, thereby improving the accuracy of video data storage.

[0058] In one embodiment, the video data storage method further includes: when the reference video frame exists in the external storage of the video decoder, reading the reference video frame from the external storage of the video decoder.

[0059] Specifically, when there is a reference video frame in the external storage of the video decoder, the electronic device reads the reference video frame from the external storage of the video decoder, and reconstructs a target video frame corresponding to the video frame to be reconstructed based on the reference video frame read from the external storage of the video decoder.

[0060] In this embodiment, when there is a reference video frame in the external storage of the video decoder, the reference video frame is read from the external storage of the video decoder, and data is accessed through the external storage of the video decoder, which can increase the efficiency of the internal cache of the decoder and further reduce power consumption specifically for the data that is still only left in the main storage.

[0061] In one embodiment, in response to the decoding process of the video frame to be reconstructed, when there is no reference video frame of the video frame to be reconstructed in the external storage of the video decoder, and there is a reference video frame in the main storage outside the video decoder, before reading the reference video frame from the main storage, the video data storage method further includes:

[0062] Step (a1), in response to decoding processing of the acquired video, obtains the motion vector corresponding to each video frame in the video.

[0063] Among them, the type of video is not limited. For example, it can be a downloaded video stored locally on the electronic device, or it can be a video obtained from a server. The video can be a TV series video, a movie video, a TV series video, etc. The length of the video is not limited, for example, it can be 10 seconds, 1 minute, 10 minutes, etc. are not limited to this. Decoding processing refers to the process of decoding and restoring the encoded video to reconstruct the video frame. Decoding processing can also be called decoding processing. Motion Vector (MV) represents the relative displacement between the block of the current video frame and the matching block in its reference image. Block Vector (BV) represents the relative displacement between the block of the current video frame and the matching block in the reconstructed current image frame.

[0064] Specifically, the electronic device obtains the motion vector corresponding to each video frame in the video through the video decoder in response to the decoding process of the acquired video. It can be understood that the electronic device decodes some video frames in the video in response to the decoding process of the acquired video to obtain the motion vector corresponding to each video frame. For example, the video includes 100 video frames, and the electronic device can decode the first 50 video frames of the 100 video frames.

[0065] Step (a2), determining the number of times each video frame is referenced based on the motion vector corresponding to each video frame.

[0066] The reference count refers to the number of times a video frame is referenced by other video frames when reconstructing the video frame. For example, video frame A is referenced by video frame B to be reconstructed, video frame C to be reconstructed, and video frame Z to be reconstructed, and is referenced 26 times in total.

[0067] Specifically, the electronic device determines the reference video frame corresponding to each video frame based on the motion vector corresponding to each video frame in the video, and counts the number of times the reference video frame is referenced. Taking video frame X, video frame Y and video frame Z as an example, video frame X is referenced by video frame Y and video frame Z, then video frame X is used as the reference video frame of video frame Y and video frame Z, and the number of times video frame X is referenced is 2.

[0068] In step (a3), the video frames whose reference times satisfy the first number condition are stored in a main storage outside the video decoder.

[0069] The first number condition can be adjusted as needed. The number represented by the first number condition is less than the number represented by the second number condition. For example, the first number condition is satisfied when it is equal to 1.

[0070] Specifically, the electronic device stores the video frames whose reference times meet the first number condition in the main storage outside the video decoder. The video decoder can subsequently read the video frames meeting the first number condition from the main storage and store them in the internal cache of the video decoder.

[0071] Step (a4), storing the video frames whose reference times satisfy the second number condition in the external storage of the video decoder; wherein the times represented by the second number condition is greater than the times represented by the first number condition.

[0072] The second number condition can be adjusted as needed. The number represented by the second number condition is greater than the number represented by the first number condition. For example, the reference number satisfies the first number condition when the reference number satisfies 1, and the reference number satisfies the second number condition when the reference number satisfies more than 1. Alternatively, the reference number satisfies the first number condition when the reference number is less than or equal to 2, and the reference number satisfies the second number condition when the reference number is greater than 2, etc., and are not limited thereto.

[0073] Specifically, the electronic device stores the video frame whose reference number satisfies the second number condition in the external storage of the video decoder. The external storage of the video decoder refers to the storage located in the electronic device but not in the video decoder. The external storage of the video decoder can be an on-chip memory. For example, the external storage of the video decoder can be at least one of a system cache and a system buffer area.

[0074] The video data storage method in this embodiment obtains the motion vector corresponding to each video frame in the video in response to the decoding process of the acquired video, determines the reference times of each video frame based on the motion vector of each video frame, and then knows how many times the video frame will be used; stores the video frames whose reference times meet the first number condition in the internal cache of the video decoder, and stores the video frames whose reference times meet the second number condition in the external storage of the video decoder, that is, stores the video frames whose reference times are less in the internal cache of the video decoder, and stores the video frames whose reference times are more in the external storage of the video decoder; reduces the use of the internal cache of the video decoder, utilizes the low-power external storage, and thus reduces the power consumption of the electronic device. In addition, since the internal cache of the video decoder is limited, the video frames whose use times are less are cached in the video decoder, and the video frames in the external cache can be updated faster, which can make full use of the external cache to speed up the decoding process.

[0075] In one embodiment, the external storage of the video decoder includes at least one of a system cache or a system buffer. The system cache and the system buffer are both on-chip memories. The system cache and the system buffer are both static random access memories (SRAM). In this embodiment, by storing the video frames that meet the second number condition in at least one of the system cache and the system buffer, since the speed of reading data from the system cache and the system buffer is fast and the power consumption is small, it is unnecessary to store the video data from the system cache and the system buffer again in the internal storage of the decoder, and the decoding speed of the decoder is maintained, thereby reducing the power consumption of the electronic device.

[0076] In one embodiment, determining the number of times each video frame is referenced based on the motion vector corresponding to each video frame includes: determining the reference video frame referenced by the current video frame based on the motion vector of the current video frame; and counting the number of times the reference video frame is referenced.

[0077] Specifically, the electronic device decodes the acquired video, and first decodes to obtain the motion vector of the video frame. The motion vector of the video frame is used to represent the displacement between the current video frame and the reference video frame. The electronic device determines the reference video frame referenced by the current video frame based on the motion vector of the current frame, and counts the number of times the reference video frame is referenced, so the number of times each video frame is referenced can be obtained. For example, the current video frame is video frame C, and the reference video frames referenced by video frame C are video frames A and video frame B. Then the number of times video frame A is referenced is 1, and the number of times video frame B is referenced is also 1. Then when the current video frame is video frame D, and the reference video frames referenced by video frame D are also video frames A and video frame B, then the number of times video frame A is referenced is 2, and the number of times video frame B is referenced is also 2, so the number of times each video frame in the video is referenced can be obtained.

[0078] The video data storage method in the present embodiment determines the reference video frame referenced by the current video frame based on the motion vector of the current video frame, and counts the number of times the reference video frame is referenced, that is, by knowing which video frame the current video frame is and what the motion vector is, it is possible to obtain the reference video frame referenced by the current video frame. Thus, the number of times each video frame in the video is referenced can be obtained, and the video data is stored based on the reference number. Compared with traditional video data, whether located in external storage or main storage, the decoder will put the decoder's internal cache into the decoder's internal cache. In the present embodiment, only the video data that does not exist in the external storage but exists in the main storage needs to be put into the decoder's internal cache, which reduces the use of the video decoder's internal cache, increases the efficiency of the decoder's internal cache, and further reduces the power consumption of the data that is still only in the main storage.

[0079] In one embodiment, the video data storage method further includes: when the number of video frames between the current video frame and the target reference video frame reaches a preset number, stop counting the number of times the target reference video frame is referenced, and obtain the number of times the target reference video frame is referenced. The preset number can be set according to actual needs. The preset number is used to characterize the maximum number of interval frames between the video frame and the reference video frame. Since there are generally not too many frames between the current video frame and the referenced video frame, when the number of video frames between the current video frame and the target reference video frame reaches a preset number, the video frames after the current video frame generally no longer refer to the target reference video frame, so the number of times the target reference video frame is referenced can be directly obtained.

[0080] In one embodiment, each video frame is divided into multiple blocks; determining the reference video frame referenced by the current video frame based on the motion vector of the current video frame includes: determining the reference video block in the reference video frame referenced by the current video frame based on the motion vector of the current block in the current video frame.

[0081] Counting the number of times the reference video frame is referenced includes: counting the number of times the reference video block in the reference video frame is referenced.

[0082] Storing the video frame whose reference times satisfy the first number condition in the main storage outside the video decoder includes: storing the reference video block whose reference times satisfy the first number condition in the main storage outside the video decoder.

[0083] Storing the video frame whose reference times satisfy the second number condition in the external storage of the video decoder includes: storing the reference video block whose reference times satisfy the second number condition in the external storage of the video decoder.

[0084] The block refers to a partial area of ​​a video frame. For example, the size of the block may be 16×16 pixels, 64×64 pixels, etc., but is not limited thereto.

[0085] Specifically, in response to the decoding process of the acquired video, the electronic device divides each video frame in the video into a plurality of blocks and obtains the motion vector corresponding to each block. The electronic device determines the reference video block in the reference video frame referenced by the current video frame based on the motion vector of the current block in the current video frame. The electronic device counts the number of times the reference video block is referenced. The electronic device stores the reference video block whose reference number meets the first number condition in the internal cache of the video decoder. The electronic device stores the reference video block whose reference number meets the second number condition in the external storage of the video decoder.

[0086] In this embodiment, based on the motion vector of the current block, the reference video block to be referenced is determined, the number of times the reference video block is referenced is counted, and the reference video blocks whose number of references meets the first number condition are stored in the main storage, and the reference video blocks whose number of references meets the second number condition are stored in the external storage of the video decoder. In this way, the reference video block to be referenced can be determined based on the current block, and the stored video data is more accurate. Since the internal cache in the video decoder is small, the use of storage blocks is more suitable for the internal cache in the video decoder.

[0087] In this embodiment, Figure 7 FIG. 1 is a schematic diagram of an architecture using a system cache in one embodiment, which includes a central processing unit, a video decoder, a display processor, a dynamic random access memory controller and a system cache. The system cache can obtain data from the DRAMC. Figure 7 The system cache size in is 4MB (megabytes). And the system cache belongs to a type of SRAM (Static Random-Access Memory). Then, in response to the decoding process of the acquired video, the motion vector corresponding to each video frame in the video is obtained through the video decoder, and the number of references of each video frame is determined based on the motion vector corresponding to each video frame; the video frames whose reference times meet the first number condition are read from DRAM and stored in the internal cache of the video decoder; some video frames whose reference times meet the second number condition are read from DRAM and stored in the system cache. It can be understood that the system cache may not be directly connected to DRAMC, Figure 6 This is just an example.

[0088] In this embodiment, Figure 8FIG. 1 is a schematic diagram of an architecture using a system buffer in an embodiment, which includes a central processing unit, a video decoder, a display processor, a dynamic random access memory controller and a system buffer. Figure 7 The system cache size in is 4MB (megabytes). And the system buffer belongs to a type of SRAM (Static Random-Access Memory). Then, in response to the decoding process of the acquired video, the motion vector corresponding to each video frame in the video is obtained through the video decoder, and the number of references of each video frame is determined based on the motion vector corresponding to each video frame; the video frame whose number of references meets the first number condition is read from the DRAM and stored in the internal cache of the video decoder; the video frame whose number of references meets the second number condition is directly stored in the system cache.

[0089] In one embodiment, the video data storage method further includes: when the internal buffer in the video decoder is full and a new video frame is stored, deleting the least important video frame in the internal buffer.

[0090] Specifically, when the internal buffer of the video decoder is full and a new video frame needs to be stored, the electronic device deletes the least important video frame in the internal buffer, thereby ensuring that the video decoder can always store data and ensure the operation of the video decoder. The least important frame can be the oldest video frame, or it can be judged in other ways.

[0091] In one embodiment, Fig. 9 As shown, it is a schematic diagram of the decoding process in an embodiment. The figure includes I frames, B frames and P frames. Among them, I frames are intra-frame compressed, and only the information of the I frame itself needs to be used during decoding. P frames are forward predictive coded frames, that is, when decoding P frames, it is necessary to refer to the information of the previous related frames to decode. B frames are bidirectional predictive coded frames, and when decoding, it is necessary to refer to both the previous existing frames and the subsequent frames to be decoded. It can be understood that some video standards allow the two reference objects of the B frame to be the previous frames. Among them, the display order (Display Order) is as follows Fig. 9 The order from left to right is shown in the figure. The decoding order is Fig. 9 The order of the following is 0, 3, 2, 4, 1, 7, 6, 8, 5. Fig. 9The decoding order in is different from the display order. The arrow indicates a reference, for example, the B frame of display order 2 is reconstructed after referring to the I frame of display order 0 and the P frame of display order 1, and the B frame of display order 2 can be used as a reference video frame for the frames of display order 1 and display order 3. In this way, the video frame of the video can be reconstructed. The blank part in the figure indicates storage in DRAM, that is, it will be stored in the internal cache of the video decoder later. The gray part indicates storage in the external storage of the video decoder. Then the I frame of display order 0, the B frame of display order 2, the B frame of display order 3, the P frame of display order 4, the B frame of display order 6 and the B frame of display order 8 all have some blocks stored in the external storage of the video decoder, and the other part is stored in DRAM or the internal cache of the video decoder. The B frame of display order 1, the B frame of display order 5 and the B frame of display order 7 are all stored in DRAM. In this embodiment, after performing some analysis on the video, it is known which frames or blocks are not suitable for storage in DRAM, or not suitable for storage in the internal cache of the video decoder, and are suitable for storage in the external storage outside the video decoder, such as at least one of the system cache and the system buffer. Under such storage distribution, the decoder will properly use the internal cache of the decoder, so that the data that only exists in the main storage outside the video decoder can survive longer in the internal cache of the decoder, so as to further reduce the system bandwidth and achieve the purpose of extreme power saving.

[0092] The basic concept of the embodiment of the present application is to properly utilize every available on-chip memory (on-chip memory, i.e., video decoder external storage and internal storage) to reduce DRAM access, and the internal cache strategy of the video decoder needs to be changed when other on-chip memories are used to assist. The entire reference video frame or a partial block of a reference video frame in the video, and the data that is reused relatively many times are stored in a low-power storage space such as a system cache or a system buffer. The remaining data that cannot be placed in the cache or the system buffer will be transferred to the internal cache of the video decoder for processing.

[0093] A video data storage method, comprising:

[0094] Step (b1), in response to decoding processing of the acquired video, obtains the motion vector corresponding to each video frame in the video.

[0095] Step (b2), based on the motion vector of the current block in the current video frame, determining a reference video block in a reference video frame to which the current video frame refers.

[0096] Step (b3), counting the number of times the reference video block in the reference video frame is referenced.

[0097] In step (b4), the reference video blocks whose reference times satisfy the first number condition are stored in a main storage outside the video decoder.

[0098] Step (b5) stores the reference video blocks whose reference times satisfy the second number condition in an external storage of the video decoder, wherein the times represented by the second number condition are greater than the times represented by the first number condition.

[0099] Step (b6), in response to the decoding process of the video frame to be reconstructed, if the reference video block corresponding to the video frame to be reconstructed does not exist in the external storage of the video decoder, and if the reference video block exists in the main storage outside the video decoder, read the reference video block from the main storage. The external storage of the video decoder includes at least one of a system cache or a system buffer area.

[0100] Step (b7), storing the read reference video block into an internal buffer of the video decoder.

[0101] Step (b8), when there is a reference video frame in the external storage of the video decoder, read the reference video frame from the external storage of the video decoder.

[0102] Step (b9), when the internal buffer in the video decoder is full and a new video frame is stored, the least necessary video frame in the internal buffer is deleted (the least necessary one may be the oldest one or selected by other evaluation methods).

[0103] The video data storage method in this embodiment obtains the motion vector corresponding to each video frame in the video in response to the decoding process of the acquired video, determines the number of times each video frame is referenced based on the motion vector of each video frame, and then knows how many times the video frame will be used; stores the video frames whose reference times meet the first number condition into the internal cache of the video decoder, and stores the video frames whose reference times meet the second number condition into the external storage of the decoder, that is, the video frames with fewer reference times are stored in the internal cache of the video decoder, and the video frames with more reference times are stored in the external storage of the decoder; compared with the traditional video data, no matter it is located in the external storage or the main storage, it will be placed in the internal cache of the decoder by the decoder. In this embodiment, only the video data that does not exist in the external storage but exists in the main storage needs to be placed in the internal cache of the decoder, which reduces the use of the internal cache of the video decoder, increases the efficiency of the internal cache of the decoder, and further reduces the power consumption of the data that is still only left in the main storage. In addition, since the internal cache of the video decoder is limited and the decoding processing speed through the internal cache is fast, video frames that are used less frequently are cached in the video decoder, and the cached video frames can be updated faster, which can make full use of the cache of the video decoder to speed up the decoding process.

[0104] Fig.10 FIG. 1 is a flow chart of a method for reading video data in an embodiment. The method for storing video data in this embodiment is described by taking operation on an electronic device as an example. Fig.10 As shown, the video data storage method includes steps 1002 to 1004.

[0105] Step 1002, in response to decoding processing of the acquired video code stream, a reference video block that does not exist in the external storage of the video decoder is read from the main storage outside the video decoder to the internal cache of the video decoder and then motion compensation or block copy is performed.

[0106] The main storage refers to the main storage of the electronic device. The main storage can be DRAM, for example. Motion compensation, also known as motion compensation, is a method of describing the difference between nearby frames (nearby here means nearby in the encoding relationship, and the two frames may not be nearby in the playback order). Specifically, it describes how each small block of the previous frame moves to a certain position in the video frame to be reconstructed.

[0107] Specifically, in response to the decoding process of the acquired video code stream, the electronic device obtains the target motion vector of the video frame to be reconstructed; the electronic device reads the reference video block that does not exist in the external storage of the video decoder from the main storage outside the video decoder to the internal cache of the video decoder according to the target motion vector, and then performs motion compensation; the reference video block is the block of the video frame referenced by the video frame to be reconstructed. Alternatively, the electronic device obtains the block vector of the video frame to be reconstructed; reads the reference video block that does not exist in the external storage of the video decoder from the main storage to the internal cache of the video decoder according to the block vector, and then performs block copying. Among them, the video frame to be reconstructed refers to a video frame that has not been successfully constructed. The target motion vector of the video frame to be reconstructed represents the displacement between the current video frame and the reference video block. The video data stored in the internal cache of the video decoder and the external storage of the video decoder may be the same or different. For example, the video contains 100 video frames, namely video frame 1, video frame 2, video frame 3...video frame 100. The internal cache of the video decoder includes video frame 1 ... video frame 50, and the external storage of the video decoder includes video frame 51, video frame 52 ... video frame 100.

[0108] Step 1004 , or performing motion compensation or block copying after reading the reference video block from the external storage of the video decoder.

[0109] Specifically, the electronic device reads the reference video block from the external storage of the video decoder and then performs motion compensation. The electronic device reads the reference video block from the external storage of the video decoder, and reconstructs the target video frame based on the read reference video block. For example, the electronic device performs weighted fusion on the read reference video block to reconstruct the target video frame. Alternatively, the electronic device reads the reference video block from the external storage of the video decoder and then performs block copying.

[0110] The video data reading method in this embodiment responds to the decoding processing of the acquired video code stream, reads the reference video block that does not exist in the external storage of the video decoder from the main storage to the internal cache of the video decoder and then performs motion compensation or block copy, or reads the reference video block from the external storage of the video decoder and then performs motion compensation or block copy, that is, it can read part of the video data from the outside of the video decoder, utilizes the low-power external storage, and also reduces the use of the internal cache of the video decoder, thereby reducing the power consumption of the electronic device.

[0111] In one embodiment, in response to decoding processing of the acquired video code stream, a reference video block that does not exist in the external storage of the video decoder is read from the external main storage of the video decoder to the internal cache of the video decoder and then motion compensation is performed; or a reference video block is read from the external storage of the video decoder and then motion compensation or block copy is performed, including:

[0112] In response to the decoding process of the acquired video code stream, searching from the external storage of the video decoder, if the reference video block is found in the external storage, then reading the reference video block from the external storage and performing motion compensation or block copying;

[0113] If the reference video block is not found in the external storage, the reference video block is found from the main storage outside the video decoder, and the reference video block is read from the main storage to the internal cache of the video decoder and then motion compensation or block copying is performed.

[0114] Specifically, the electronic device searches for the corresponding reference video block from the external storage of the video decoder according to the target motion vector of the video frame to be reconstructed. When the reference video block corresponding to the video frame to be reconstructed is found in the external storage, the reference video block corresponding to the video frame to be reconstructed is read from the external storage and then motion compensation or block copying is performed. When the reference video block corresponding to the video frame to be reconstructed is not found in the external storage, the electronic device searches for the reference video block corresponding to the video frame to be reconstructed from the main storage, reads the reference video block from the main storage to the internal cache of the video decoder and then performs motion compensation or block copying. The reference video block is a part of the video area in the reference video frame. The number of reference video blocks corresponding to the video frame to be reconstructed is not limited. For example, it can be 1, 2, 3... but not limited to this.

[0115] Taking the system cache as the external storage, the DRAM as the main storage, and the VDEC cache as the internal cache of the video decoder as an example, the electronic device searches the system cache for the corresponding reference video block according to the target motion vector of the video frame to be reconstructed. In the case where the corresponding reference video block is found in the system cache, the reference video block corresponding to the video frame to be reconstructed is read from the system cache. In the case where the reference video block corresponding to the video frame to be reconstructed is not found in the system cache, the reference video block corresponding to the video frame to be reconstructed is found from the DRAM, and the reference video block is read from the main storage to the VDEC cache to perform motion compensation.

[0116] In this embodiment, when all reference video blocks corresponding to the video frames to be reconstructed are not found in the external storage, reference video blocks corresponding to the remaining video frames to be reconstructed are found from the main storage, and the reference video blocks are read from the main storage to the internal cache of the video decoder to perform motion compensation or block copying.

[0117] In this embodiment, when the reference video block is found from the external storage of the video decoder, the reference video block is read from the external storage and then motion compensation or block copying is performed; when the reference video block is not found in the external storage, the reference video block is found from the main storage outside the video decoder, the reference video block is found from the main storage, and the reference video block is read from the main storage to the internal cache of the video decoder and then motion compensation or block copying is performed; the internal cache of the video decoder only reads data from the main storage, compared with the traditional method in which the data stored in the internal cache of the video decoder is consistent with the data stored in the external storage, it can save unnecessary cache operations inside the video decoder, and because the amount of video data in the internal cache of the video decoder is small, the bandwidth of the main storage can be further reduced, thereby reducing the power consumption of the electronic device.

[0118] In one embodiment, the video data reading method also includes: obtaining motion vectors corresponding to each video frame in the video in response to decoding processing of the acquired video; determining the number of references of each video frame based on the motion vectors corresponding to each video frame; reading video frames whose reference numbers meet a first number condition from the main storage outside the video decoder and storing them in the internal cache of the video decoder; storing video frames whose reference numbers meet a second number condition in the external storage of the video decoder; the number represented by the second number condition is greater than the number represented by the first number condition.

[0119] In this embodiment, in response to the decoding process of the acquired video, the motion vector corresponding to each video frame in the video is obtained, and the reference times of each video frame are determined based on the motion vector of each video frame, so that it can be known how many times the video frame will be used; the video frame whose reference times meet the first number condition is stored from the main storage to the internal cache of the video decoder, and the video frame whose reference times meet the second number condition is stored to the external storage of the decoder, that is, the video frame with less reference times is stored in the internal cache of the video decoder, and the video frame with more reference times is stored in the external storage of the decoder; the use of the internal cache of the video decoder is reduced, and the low-power external storage is used, thereby reducing the power consumption of the electronic device. In addition, since the internal cache of the video decoder is limited, the video frames with less use times are cached in the video decoder, and the speed at which these video frames in the external cache are updated can be faster, which can make full use of the external cache to speed up the decoding process.

[0120] In one embodiment, the external storage of the video decoder includes at least one of a system cache and a system buffer. The system cache and the system buffer are both on-chip memories. The system cache and the system buffer are both static random access memories (SRAM). In this embodiment, by storing the video frames that meet the second number condition in at least one of the system cache and the system buffer, the decoding speed of the video decoder can be maintained because the data read from the system cache and the system buffer is fast and consumes less power, and the data stored in these caches can be kept from being placed in the internal cache of the decoder.

[0121] In one embodiment, Fig.11 As shown, it is a schematic diagram of the process of reconstructing a video frame in an embodiment. It includes A video frame 1102, B video frame 1104, C video frame 1106, and video frame 1108 to be reconstructed, and A video frame 1102, A video frame 1104 and A video frame 1106 are all video frames that have been reconstructed, while video frame 1108 to be reconstructed has not been reconstructed. Among them, the white parts in A video frame 1102, A video frame 1104 and A video frame 1106 refer to the storage in DRAM, that is, the storage in the internal cache of the video decoder; the blocks in the gray shaded part are stored in the external storage of the video decoder. And A video frame 1102, B video frame 1104 and C video frame 1106 are all reference video frames of video frame 1108 to be reconstructed. As can be seen from the figure, different blocks of video frames can be stored in different storages. Fig.11 In the description, a 16×16 block and a B frame are taken as an example for the video frame 1108 to be reconstructed.

[0122] A small square in the video frame to be reconstructed represents a current block. Fig.11The reference blocks in are not reused. The electronic device obtains the 1st, 2nd, 3rd, 4th, 5th, 6th and 7th sub-target motion vectors of the video frame to be reconstructed, and can determine the reference blocks based on the sub-target motion vectors. For example, the 1st block of the video frame 1108 to be reconstructed can be reconstructed from block 0 in the A video frame 1102 and block 1 in the A video frame 1104. The 2nd block of the video frame 1108 to be reconstructed can be reconstructed from block 2 in the A video frame 1102 and block 3 in the A video frame 1106. The 3rd block of the video frame 1108 to be reconstructed can be reconstructed from block 4 in the A video frame 1104 and block 5 in the A video frame 1104. The 4th block of the video frame 1108 to be reconstructed can be reconstructed from block 6 in the A video frame 1104 and block 7 in the A video frame 1106. The 5th block of the video frame to be reconstructed 1108 can be reconstructed from the 8th block in the A video frame 1102 and the 9th block in the A video frame 1104. The 6th block of the video frame to be reconstructed 1108 can be reconstructed from the A block in the A video frame 1104 and the B block in the A video frame 1106. The 7th block of the video frame to be reconstructed 1108 can be reconstructed from the C block in the A video frame 1102 and the D block in the A video frame 1104.

[0123] In one embodiment, Fig.12 FIG. 1 is a schematic diagram of a storage method of a conventional video decoder in an embodiment. Fig.11 Take the middle block as an example. Fig.12 The storage method of the video decoder is to store blocks in order. For example, the first step stores block 0, the second step stores blocks 1, 2, 3, 4, 5, 6 and 7, and when a new block is stored, the third step stores blocks 6, 7, 8, 9, A, B, C and D. In other words, Fig.12 The internal cache of the video decoder in the above figure temporarily stores the blocks read from any area, whether they are read from DRAM (white part) or from the external storage of the video decoder (gray part). This method saves DRAM throughput by not requiring the internal cache of the video decoder at all, and also stores the video data in the internal cache of the video decoder, wasting the reading and writing energy consumption of the internal cache of the video decoder.

[0124] In one embodiment, Fig.13 FIG. 1 is a schematic diagram of a storage method of the internal storage of a video decoder in one embodiment. The white portion in the internal cache of the video decoder refers to the block originally stored in the DRAM, that is, the block to be stored in the internal cache of the video decoder later. Fig.12 way, Fig.13The internal cache of the video decoder only stores blocks from DRAM. Therefore, the blocks stored in the external storage of the video decoder and the blocks stored in the internal cache of the video decoder will not be stored repeatedly. The blocks stored in DRAM are distinguished and read into the internal cache of the video decoder. The other blocks are stored in the external storage of the video decoder, so that the overall reading of DRAM is reduced during the decoding process, the bandwidth of DRAM is reduced, and the power consumption of electronic equipment is reduced. The data in the system buffer / cache is power-saving, and the access speed is fast and controllable. If such data is cached again using the VDEC internal cache, it will not help the decoding speed and consume one more SRAM access energy. In addition, if there is already a power-saving system cache / buffer that stores part of the frame buffer data, the remaining data not stored in the systemcache / buffer entering the VDEC internal cache will actually save DRAM bandwidth.

[0125] In one embodiment, Table 1 is a schematic diagram of the energy consumption difference between SRAM and DRAM in one embodiment. The energy consumption of SRAM (Static Random-Access Memory) is about 5PJ (picojoules), while the energy consumption of DRAM is about 640PJ (picojoules), which is 2 orders of magnitude different from each other. In terms of relative energy consumption, the relative energy consumption of SRAM is 10 2 , while the relative energy consumption of DRAM is 10 4 , the difference between the two is 2 orders of magnitude. Although Table 1 shows that accessing DRAM consumes more than 100 times the power of accessing SRAM, even if there is a system buffer / cache in the electronic device, the amount is limited, and it is impossible to expand the on-chip SRAM outside the VDEC indefinitely. Therefore, making good use of the internal cache of the original video decoder can further reduce the required DRAM bandwidth when using the system cache / buffer, achieving more extreme power saving and low power consumption.

[0126] Table 1

[0127] Storage Name Energy consumption (energy pJ) Relative Energy Cost Static Random Access Memory 5 <![CDATA[10 2 ]]> Dynamic Random Access Memory 640 <![CDATA[10 4 ]]>

[0128] It should be understood that although Figure 5 and Fig.10The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 5 and Fig.10 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0129] Fig.14 FIG. 4 is a structural block diagram of a video data storage device according to an embodiment of the present invention. Fig.14 As shown, a video data storage device includes a first reading module 1402 and a reference video frame storage module 1404, wherein:

[0130] A first reading module 1402 is configured to, in response to decoding processing of the video frame to be reconstructed, read the reference video frame from the main storage when there is no reference video frame of the video frame to be reconstructed in the external storage of the video decoder and there is a reference video frame in the main storage outside the video decoder; and

[0131] The reference video frame storage module 1404 is used to store the reference video frame read from the main storage into the internal cache of the video decoder.

[0132] The video data storage device in the present embodiment responds to the decoding processing of the video frame to be reconstructed, when there is no reference video frame of the video frame to be reconstructed in the external storage of the video decoder and there is a reference video frame in the main storage outside the video decoder, reads the reference video frame from the main storage, and stores the reference video frame read from the main storage in the internal cache of the video decoder, that is, the video data existing in the external storage of the video decoder is not stored in the internal cache of the video decoder, and the video data in the main storage is stored in the internal cache of the video decoder. Compared with the traditional video data, no matter it is located in the external storage or the main storage, it will be placed in the internal cache of the decoder by the decoder. In the present embodiment, only the video data that does not exist in the external storage but exists in the main storage needs to be placed in the internal cache of the decoder, which reduces the use of the internal cache of the video decoder, increases the efficiency of the internal cache of the decoder, and further reduces the power consumption of the data that is still only left in the main storage.

[0133] In one embodiment, each video frame is divided into a plurality of video blocks. The first reading module 1402 is used to read the reference video block from the main storage when there is no reference video block corresponding to the video frame to be reconstructed in the external storage of the video decoder and there is a reference video block in the main storage outside the video decoder. The reference video frame storage module 1404 is used to store the reference video block read from the main storage into the internal cache of the video decoder.

[0134] In this embodiment, when there is no reference video block corresponding to the video frame to be reconstructed in the external storage of the video decoder, and there is a reference video block in the main storage, the electronic device reads the reference video block from the main storage and stores it in the internal cache of the video decoder. Since the internal cache in the video decoder is relatively small, the use of storage blocks is more suitable for the internal cache in the video decoder, thereby improving the accuracy of video data storage.

[0135] In one embodiment, the first reading module 1402 is further configured to read the reference video frame from the external storage of the video decoder when the reference video frame exists in the external storage of the video decoder.

[0136] In this embodiment, when there is a reference video frame in the external storage of the video decoder, the reference video frame is read from the external storage of the video decoder, and data is accessed through the external storage of the video decoder, which can increase the efficiency of the internal cache of the decoder and further reduce power consumption specifically for the data that is still only left in the main storage.

[0137] In one embodiment, the video data storage device further includes a motion vector acquisition module, a number determination module, a first storage module and a second storage module, wherein:

[0138] A motion vector acquisition module, for obtaining a motion vector corresponding to each video frame in the video in response to decoding processing of the acquired video;

[0139] A times determination module, used to determine the times each video frame is referenced based on the motion vector corresponding to each video frame;

[0140] A first storage module, used for storing the video frame whose reference number satisfies the first number condition into the main storage of the video decoder;

[0141] The second storage module is used to store the video frames whose reference times satisfy a second number condition in an external storage of the video decoder; wherein the times represented by the second number condition is greater than the times represented by the first number condition.

[0142] In this embodiment, in response to the decoding process of the acquired video, the motion vector corresponding to each video frame in the video is obtained, and the number of times each video frame is referenced is determined based on the motion vector of each video frame, so that it can be known how many times the video frame will be used; the video frame whose reference number meets the first number condition is stored in the internal cache of the video decoder, and the video frame whose reference number meets the second number condition is stored in the external storage of the decoder, that is, the video frame with fewer reference times is stored in the internal cache of the video decoder, and the video frame with more reference times is stored in the external storage of the decoder; the use of the internal cache of the video decoder is reduced, and the low-power external storage is used, thereby reducing the power consumption of the electronic device. In addition, since the internal cache of the video decoder is limited, the video frames with fewer uses are cached in the video decoder, and the speed at which these video frames in the external cache are updated can be faster, and the external cache can be fully utilized to speed up the decoding process.

[0143] In one embodiment, the number determination module is used to determine the reference video frame referenced by the current video frame based on the motion vector of the current video frame; and count the number of times the reference video frame is referenced.

[0144] The video data storage device in this embodiment determines the reference video frame referenced by the current video frame based on the motion vector of the current video frame, and counts the number of times the reference video frame is referenced. That is, by knowing which video frame the current video frame is and what the motion vector is, it can be obtained what the reference video frame referenced by the current video frame is. The number of times each video frame in the video is referenced can be obtained, and video data is stored based on the reference number, thereby reducing the power consumption of electronic equipment.

[0145] In one embodiment, the number determination module is further used to stop counting the number of times the target reference video frame is referenced when the number of video frames between the current video frame and the target reference video frame reaches a preset number, and obtain the number of times the target reference video frame is referenced. Since there are generally not too many frames between the current video frame and the referenced video frame, when the number of video frames between the current video frame and the target reference video frame reaches a preset number, the video frames after the current video frame generally will not refer to the target reference video frame, so the number of times the target reference video frame is referenced can be directly obtained.

[0146] In one embodiment, each video frame is divided into a plurality of blocks. The number determination module is used to determine the reference video block in the reference video frame referenced by the current video frame based on the motion vector of the current block in the current video frame; count the number of times the reference video block in the reference video frame is referenced; store the reference video block whose reference number meets the first number condition in the internal cache of the video decoder; and store the reference video block whose reference number meets the second number condition in the external storage of the video decoder.

[0147] In this embodiment, based on the motion vector of the current block, the reference video block to be referenced is determined, the number of times the reference video block is referenced is counted, and the reference video blocks whose number of references meets the first number condition are stored in the internal storage of the video decoder, and the reference video blocks whose number of references meets the second number condition are stored in the external storage of the video decoder. In this way, the reference video block to be referenced can be determined based on the current block, and the stored video data is more accurate. Since the internal cache in the video decoder is small, the storage block method is more suitable for the internal cache in the video decoder.

[0148] In one embodiment, the external storage of the video decoder includes at least one of a system cache or a system buffer area. In this embodiment, by storing the video frames that meet the second number condition in at least one of the system cache and the system buffer area, the decoding speed of the video decoder can be maintained and the power consumption of the electronic device can be reduced because the data read from the system cache and the system buffer area is fast and the power consumption is small.

[0149] In one embodiment, the main storage external to the video decoder includes dynamic random access memory.

[0150] In one embodiment, the first storage module is also used to delete the least needed video frame in the internal cache when the internal cache in the video decoder is full and a new video frame needs to be stored. In this embodiment, when the internal cache in the video decoder is full and a new video frame needs to be stored, the electronic device deletes the least needed video frame in the internal cache, which can ensure that the video decoder can always store data and ensure the operation of the video decoder. For example, it is possible to determine which video frames are the least needed by age. The least needed video frame can be, for example, the oldest. In other embodiments, other evaluation methods can also be used to select the least needed video frames.

[0151] Fig.15 FIG. 1 is a structural block diagram of a video data reading device according to an embodiment. Fig.15 As shown, a video data reading device includes a second reading module 1502 and a third reading module 1504, wherein:

[0152] A second reading module 1502 is used for reading a reference video block that does not exist in the external storage of the video decoder from the main storage outside the video decoder to the internal cache of the video decoder in response to the decoding process of the acquired video code stream, and then performing motion compensation or block copying;

[0153] or,

[0154] The third reading module 1504 is used to read the reference video block from the external storage of the video decoder and then perform motion compensation or block copying.

[0155] The video data reading device in this embodiment responds to the decoding processing of the acquired video code stream, reads the reference video block that does not exist in the external storage of the video decoder from the main storage to the internal cache of the video decoder and then performs motion compensation or block copy, or reads the reference video block from the external storage of the video decoder and then performs motion compensation or block copy, that is, part of the video data is stored externally of the video decoder, low-power external storage is utilized, the use of the internal cache of the video decoder is reduced, and the bandwidth of the main storage is also reduced, thereby reducing the power consumption of the electronic device.

[0156] In one embodiment, the second reading module 1502 is used to search from the external storage of the video decoder in response to the decoding process of the acquired video code stream, and when the reference video block is found in the external storage, the reference video block is read from the external storage and then motion compensation or block copy is performed;

[0157] The third reading module 1504 is used to find the reference video block from the main storage outside the video decoder when the reference video block is not found in the external storage, and read the reference video block from the main storage to the internal cache of the video decoder to perform motion compensation or block copying.

[0158] In the present embodiment, when the reference video block is found from the external storage of the video decoder, the reference video block is read from the external storage; when the reference video block is not found in the external storage, the reference video block is found from the main storage, and the reference video block is found from the main storage, and the reference video block is read from the main storage to the internal cache of the video decoder and then motion compensation is performed; then the internal cache of the video decoder only reads data from the main storage, which is compared with the traditional method in which the internal cache of the video decoder is consistent with the data stored in the external storage, and unnecessary cache operations inside the video decoder can be saved, thereby reducing the power consumption of the electronic device.

[0159] In one embodiment, the video data reading device also includes a storage module, which is used to obtain the motion vector corresponding to each video frame in the video in response to the decoding processing of the acquired video; determine the reference number of each video frame based on the motion vector corresponding to each video frame; store the video frames whose reference number meets the first number condition in the main storage outside the video decoder; store the video frames whose reference number meets the second number condition in the external storage of the video decoder; wherein the number represented by the second number condition is greater than the number represented by the first number condition.

[0160] In this embodiment, in response to the decoding process of the acquired video, the motion vector corresponding to each video frame in the video is obtained, and the reference times of each video frame are determined based on the motion vector of each video frame, so that it can be known how many times the video frame will be used; the video frame whose reference times meet the first number condition is stored in the main storage, and the video frame whose reference times meet the second number condition is stored in the external storage of the decoder, that is, the video frame with a small number of references is stored in the main storage, and the video frame with a large number of references is stored in the external storage of the decoder; thereby reducing the use of the internal cache of the video decoder, utilizing the low-power external storage, and thus reducing the power consumption of the electronic device. In addition, since the internal cache of the video decoder is limited, the video frames with a small number of uses are cached in the video decoder, and the speed at which these video frames in the external cache are updated can be faster, and the external cache can be fully utilized to speed up the decoding process. In one embodiment, the external storage of the video decoder includes at least one of a system cache and a system buffer. In this embodiment, video frames that meet the second number condition are stored in at least one of the system cache and the system buffer area. Since the data is read from the system cache and the system buffer area at a faster speed and with lower power consumption, there is no need to store the video data from the system cache and the system buffer area again in the decoder's internal storage, thereby reducing the power consumption of the electronic device.

[0161] The division of each module in the above-mentioned video data storage device and video data reading device is only for illustration. In other embodiments, the video data storage device and video data reading device can be divided into different modules as needed to complete all or part of the functions of the above-mentioned video data storage device and video data reading device.

[0162] The specific definition of the video data storage device and the video data reading device can be found in the definition of the method above, which will not be repeated here. The various modules in the above-mentioned video data storage device and video data reading device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0163] Fig.16 FIG. 1 is a schematic diagram of the internal structure of an electronic device in one embodiment. Fig.16As shown, the electronic device includes a processor and a memory connected via a system bus. The processor is used to provide computing and control capabilities to support the operation of the entire electronic device. The memory may include a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The computer program can be executed by the processor to implement the video data storage method and the video data reading method provided in the following embodiments. The internal memory provides a high-speed cache operating environment for the operating system computer program in the non-volatile storage medium. The electronic device can be any terminal device such as a mobile phone, a tablet computer, a PDA (Personal Digital Assistant), a POS (Point of Sales), a vehicle-mounted computer, a wearable device, etc.

[0164] The implementation of each module in the video data storage device and the video data reading device provided in the embodiment of the present application can be in the form of a computer program. The computer program can be run on a terminal or a server. The program module constituted by the computer program can be stored in the memory of an electronic device. When the computer program is executed by a processor, the steps of the method described in the embodiment of the present application are implemented.

[0165] The embodiment of the present application further provides a computer-readable storage medium, one or more non-volatile computer-readable storage media containing computer-executable instructions, which, when executed by one or more processors, enable the processors to perform the steps of the video data storage method.

[0166] The embodiment of the present application further provides a computer-readable storage medium, one or more non-volatile computer-readable storage media containing computer-executable instructions, when the computer-executable instructions are executed by one or more processors, the processors execute the steps of the video data reading method.

[0167] A computer program product comprising instructions, when executed on a computer, causes the computer to perform a video data storage method.

[0168] A computer program product comprising instructions, when being run on a computer, enables the computer to execute a video data reading method.

[0169] Any reference to memory, storage, database or other medium used in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM), which is used as an external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0170] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A video data storage method, characterized in that: include: In response to decoding the video frame to be reconstructed, if a reference video frame of the video frame to be reconstructed does not exist in the external storage of the video decoder and the reference video frame exists in the main storage outside the video decoder, read the reference video frame from the main storage; as well as storing the reference video frame read from the primary storage into an internal cache of the video decoder; The video data in the external storage of the video decoder is not stored in the internal cache of the video decoder.

2. The method according to claim 1, characterized in that Each video frame is divided into multiple blocks; When there is no reference video frame of the to-be-reconstructed video frame in the external storage of the video decoder and the reference video frame exists in the main storage outside the video decoder, reading the reference video frame from the main storage comprises: When the reference video block corresponding to the to-be-reconstructed video frame does not exist in the external storage of the video decoder and the reference video block exists in the main storage outside the video decoder, reading the reference video block from the main storage; The step of storing the reference video frame read from the main storage into an internal cache of the video decoder comprises: The reference video block read from the main storage is stored in an internal buffer of the video decoder.

3. The method according to claim 1, characterized in that The method further comprises: In a case where the reference video frame exists in an external storage of the video decoder, the reference video frame is read from the external storage of the video decoder.

4. The method according to claim 1, characterized in that: The method further comprises: In response to decoding the acquired video, a motion vector corresponding to each video frame in the video is obtained; Determine the number of times each video frame is referenced based on the motion vector corresponding to each video frame; storing the video frame whose reference number meets the first number condition in a main storage outside the video decoder; The video frame whose referenced number of times satisfies a second number condition is stored in an external storage of the video decoder, wherein the number represented by the second number condition is greater than the number represented by the first number condition.

5. The method according to any one of claims 1 to 4, characterized in that: The external storage of the video decoder includes at least one of a system cache or a system buffer area.

6. The method according to any one of claims 1 to 4, characterized in that: The external main storage of the video decoder includes a dynamic random access memory.

7. A video data reading method, characterized in that: The method comprises: In response to decoding processing of the acquired video code stream, a reference video block not existing in the external storage of the video decoder is read from the main storage outside the video decoder to the internal cache of the video decoder and then motion compensation or block copy is performed; or performing motion compensation or block copying after reading reference video blocks from an external storage of a video decoder; The video data in the external storage of the video decoder is not stored in the internal cache of the video decoder.

8. The method according to claim 7, characterized in that In response to the decoding process of the acquired video code stream, a reference video block not existing in the external storage of the video decoder is read from the main storage outside the video decoder to the internal cache of the video decoder and then motion compensation or block copy is performed; Alternatively, the reference video block is read from the external storage of the video decoder and then motion compensation or block copy is performed, including: In response to the decoding process of the acquired video code stream, searching from the external storage of the video decoder, and in the case where a reference video block is found in the external storage, performing motion compensation or block copying after reading the reference video block from the external storage; When the reference video block is not found in the external storage, the reference video block is found from the main storage outside the video decoder, and the reference video block is read from the main storage to the internal cache of the video decoder and then motion compensation or block copying is performed.

9. The method according to claim 7, characterized in that: The method further comprises: In response to decoding the acquired video, a motion vector corresponding to each video frame in the video is obtained; Determine the number of times each video frame is referenced based on the motion vector corresponding to each video frame; storing the video frame whose reference number meets the first number condition in a main storage outside the video decoder; The video frame whose referenced number of times satisfies a second number condition is stored in an external storage of the video decoder, wherein the number represented by the second number condition is greater than the number represented by the first number condition.

10. The method according to any one of claims 7 to 9, characterized in that: The external storage of the video decoder includes at least one of a system cache or a system buffer area.

11. The method according to any one of claims 7 to 9, characterized in that: The external main storage of the video decoder includes a dynamic random access memory.

12. A video data storage device, characterized in that: include: A first reading module is configured to, in response to decoding processing of a video frame to be reconstructed, read the reference video frame from the main storage when there is no reference video frame of the video frame to be reconstructed in the external storage of the video decoder and the reference video frame exists in the main storage outside the video decoder; as well as A reference video frame storage module, used for storing the reference video frame read from the main storage into the internal cache of the video decoder; The video data in the external storage of the video decoder is not stored in the internal cache of the video decoder.

13. A video data reading device, characterized in that: include: A second reading module is used for reading a reference video block that does not exist in the external storage of the video decoder from the main storage outside the video decoder to the internal cache of the video decoder in response to the decoding process of the acquired video code stream, and then performing motion compensation or block copying; or, A third reading module is used for reading a reference video block from an external storage of the video decoder and then performing motion compensation or block copying; The video data in the external storage of the video decoder is not stored in the internal cache of the video decoder.

14. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the computer program is executed by the processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 11.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

16. A computer program product comprising instructions, characterized in that When the method is executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 11.

Images