Decoding data acquisition method and device, equipment and storage medium

Abstract

The invention discloses a decoding data acquisition method and device, equipment and a storage medium, and solves the problem of low overall efficiency of NLE caused by low decoding data acquisition efficiency in the prior art. The method comprises the following steps: decoding a target frame image in a material track fragment to generate decoding data of the target frame image; according to a multi-level cache strategy, storing the decoded data into a multi-level cache space; the multi-level caching strategy comprises the following steps: if the decoded data cached in the first storage space does not reach the upper limit of the first storage space, caching the decoded data of the target frame image in the first storage space; if the decoded data cached in the first storage space reaches the upper limit of the first storage space, caching the decoded data of the target frame image in a second storage space; determining a target storage space from the multi-level cache space; and reading the decoding data of the target frame image from the target storage space, rendering the target frame image based on the decoding data, and displaying the preview image.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a method and device for obtaining decoded data, an electronic device, and a computer-readable storage medium. Background Art

[0002] When a user uses a non-linear editing (NLE) system to edit materials such as video files, the user may perform a seek operation on the materials loaded into a video editing track.

[0003] Performing a seek operation on the material loaded on the video editing track refers to dragging the timeline on the video editing track, or dragging the material loaded on the video editing track, so that the timeline position and the material change relative positions along the time axis of the video editing track, locating the target playback position of the material according to the timeline position, and then rendering and presenting at least the image frame located at the target playback position in the material. Among them, the material loaded on the video editing track can be called a material track clip.

[0004] According to the prior art, after performing a seek operation, the NLE system will decode the material currently loaded into the memory in real time according to the located target playback position, so as to obtain the decoded data of the image frame corresponding to the target playback position, and then render and present the image frame based on the decoded data.

[0005] The method of real-time decoding of video files is used to obtain decoded data of image frames matching the target playback position. However, the problem is that the efficiency of obtaining decoded data is low, thereby reducing the overall efficiency of NLE. Summary of the invention

[0006] The disclosed embodiments provide a method for obtaining decoded data, which is used to solve the problem in the prior art that, in response to a preview operation triggered on a video editing track, real-time decoding of materials is used to obtain decoded data of image frames matching a target playback position, which results in low efficiency in obtaining decoded data and further leads to low overall efficiency of NLE.

[0007] The embodiments of the present disclosure also provide a decoding data acquisition device, an electronic device, and a computer-readable storage medium.

[0008] The embodiments of the present disclosure adopt the following technical solutions:

[0009] A method for obtaining decoded data, comprising:

[0010] For a material track segment presented on a video editing track, decoding a target frame image in the material track segment to generate decoded data of the target frame image;

[0011] According to the multi-level cache strategy, the decoded data of the target frame image is stored in the multi-level cache space; wherein the multi-level cache space at least includes a first storage space and a second storage space, and the read and write speed of the first storage space is greater than the read and write speed of the second storage space; the multi-level cache strategy includes: if the decoded data of the frame image cached in the first storage space does not reach the upper limit of the first storage space, then the decoded data of the target frame image is cached in the first storage space; if the decoded data of the frame image cached in the first storage space has reached the upper limit of the first storage space, then the decoded data of the target frame image is cached in the second storage space;

[0012] In response to a preview operation triggered on a video editing track, if the timeline position of the target frame image in the material track segment is previewed, a target storage space is determined from the multi-level cache space, and the target storage space is used to store decoded data of the target frame image;

[0013] If the decoded data of the target frame image is stored in the multi-level cache space, the decoded data of the target frame image is read from the target storage space, and the target frame image is rendered based on the decoded data and a preview image is displayed.

[0014] A decoding data acquisition device, comprising:

[0015] A decoding unit, configured to decode a target frame image in a material track segment presented on a video editing track, and generate decoded data of the target frame image;

[0016] A storage unit, for storing the decoded data of the target frame image in a multi-level cache space according to a multi-level cache strategy; wherein the multi-level cache space at least includes a first storage space and a second storage space, and the read and write speed of the first storage space is greater than the read and write speed of the second storage space; the multi-level cache strategy includes: if the decoded data of the frame image cached in the first storage space does not reach the upper limit of the first storage space, then the decoded data of the target frame image is cached in the first storage space; if the decoded data of the frame image cached in the first storage space has reached the upper limit of the first storage space, then the decoded data of the target frame image is cached in the second storage space;

[0017] a storage space determination unit, configured to respond to a preview operation triggered on a video editing track, and if the timeline position of the target frame image in the material track segment is previewed, determine a target storage space from the multi-level cache space, wherein the target storage space is used to store decoded data of the target frame image;

[0018] A data acquisition and display unit is used to read the decoded data of the target frame image from the target storage space if the decoded data of the target frame image is stored in the multi-level cache space, and render the target frame image based on the decoded data and display a preview image.

[0019] An electronic device, comprising: a memory and a processor, wherein:

[0020] The memory is used to store programs; the processor is coupled to the memory and is used to execute the programs stored in the memory to execute the above-mentioned decoding data acquisition method.

[0021] A computer-readable storage medium storing a computer program, wherein the computer program can implement the above-mentioned decoding data acquisition method when executed by a computer.

[0022] At least one of the above technical solutions adopted in the embodiments of the present disclosure can achieve the following beneficial effects:

[0023] Since a multi-level cache space including at least a first storage space and a second storage space is used to store the decoded data generated by decoding the target frame image in the material track segment, based on this, in response to the preview operation triggered on the video editing track, when the timeline position of the target frame image in the material track segment is previewed, if the decoded data of the target frame image is stored in the multi-level cache space, then the decoded data of the target frame image can be obtained from the target storage space of the multi-level cache space to render and preview the image of the target frame, without the need to obtain the decoded data of the target frame image by real-time decoding of the video file as in the prior art, thereby improving the efficiency of obtaining the decoded data, and further improving the overall efficiency of the NLE. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present disclosure. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure. In the drawings:

[0025] Figure 1a A specific implementation flow chart of a method for obtaining decoded data provided by an embodiment of the present disclosure;

[0026] Figure 1b It is a schematic diagram of the interface of a non-linear editing system;

[0027] Figure 2 A schematic diagram of a specific implementation process of the decoding data acquisition method provided by the embodiment of the present disclosure in a non-linear editing system;

[0028] Figure 3A schematic diagram of a specific implementation process of the decoding data acquisition method provided by the embodiment of the present disclosure in a non-linear editing system;

[0029] Figure 4 A schematic diagram of the specific structure of a decoding data acquisition device provided by an embodiment of the present disclosure;

[0030] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solutions and advantages of the present disclosure clearer, the technical solutions of the present disclosure will be clearly and completely described below in combination with the specific embodiments of the present disclosure and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0032] Those skilled in the art may appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided by the embodiments of the present disclosure are also applicable to similar technical problems.

[0033] The terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable where appropriate, which is merely a way of distinguishing objects of the same attributes when describing the embodiments of the present disclosure. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, so that a process, method, system, product, or device that includes a series of units is not necessarily limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0034] In order to solve the problem that in the prior art, in response to a preview operation triggered on a video editing track, real-time decoding of materials is used to obtain decoding data of image frames matching a target playback position, which leads to low decoding data acquisition efficiency and further leads to low overall efficiency of NLE, an embodiment of the present disclosure provides a decoding data acquisition method.

[0035] The execution subject of the method may be, for example, a computing device such as a mobile phone, a personal computer, a wearable device, or software installed on the computing device, such as a player or decoder of a video file, etc. The embodiment of the present disclosure does not limit what computing device or software the execution subject is.

[0036] In addition, different steps of the method may be implemented by different computing devices or software, and the embodiments of the present disclosure do not limit which computing device or software is used to implement which step.

[0037] For ease of description, the method provided by the embodiment of the present disclosure is described in detail below by taking the execution subject of the method as a non-linear editing system installed on a computing device as an example.

[0038] Please refer to the instruction manual Figure 1a , is a specific implementation flow chart of a method for obtaining decoded data provided by an embodiment of the present disclosure, comprising the following steps:

[0039] Step 11: For the material track segment presented on the video editing track, the non-linear editing system decodes the target frame image in the material track segment to generate decoded data of the target frame image;

[0040] For step 11, in an optional implementation, the non-linear editing system may, for example, decode the target image frame contained in the material track segment in response to a user's preview instruction for the material track segment presented on the video editing track, thereby generating decoded data of the target frame image.

[0041] In step 11, the "target frame image" may not specifically refer to a certain frame image, but generally refers to an image in a material track segment that is decoded by a linear editing system.

[0042] Step 12: The nonlinear editing system stores the decoded data of the target frame image into the multi-level cache space according to the multi-level cache strategy;

[0043] The multi-level cache space mentioned here at least includes: a first storage space and a second storage space.

[0044] In the embodiment of the present disclosure, the read and write speed of the first storage space is greater than the read and write speed of the second storage space.

[0045] In a specific example, the first storage space may be, for example, a dedicated storage space; the second storage space may be, for example, a shared storage space.

[0046] The dedicated storage space mentioned here refers to a storage space that is specifically used to store the decoded data obtained by decoding the target frame image in the material track segment, and is not used to store other data. In contrast, the shared storage space refers to a storage space that can be used to store the decoded data obtained by decoding the target frame image in the material track segment, but can also be used to store other data.

[0047] In an optional implementation, by setting an identifier for the storage space to indicate whether the storage space is a dedicated storage space or a shared storage space, it can be indicated whether the storage space is a dedicated storage space or a shared storage space. When storing data in a dedicated storage space (or a shared storage space), the identifier can be used to determine whether the storage space is a dedicated storage space or a shared storage space, and further, according to the pre-set storage rule that "the dedicated storage space is specifically used to store decoded data obtained by decoding the target frame image in the material track segment, and is not used to store other data; and the shared storage space can be used to store decoded data obtained by decoding the target frame image in the material track segment, and can also be used to store other data", it can be determined whether the data to be stored can be stored in the dedicated storage space (or shared storage space).

[0048] Alternatively, dedicated storage space refers to storage space that is used by the graphics processing unit (GPU) of a computing device and is not shared with other applications; while shared storage space refers to storage space that can be used by all applications but is used preferentially by the GPU.

[0049] In a specific example, the first storage space may be, for example, a video memory; and the second storage space may be, for example, a memory.

[0050] In the embodiments of the present disclosure, the multi-level cache strategy may include, but is not limited to: if the decoded data of the frame image cached in the first storage space has not reached the upper limit of the first storage space, then the decoded data of the target frame image is cached in the first storage space; if the decoded data of the frame image cached in the first storage space has reached the upper limit of the first storage space, then the decoded data of the target frame image is cached in the second storage space.

[0051] As for the specific implementation of step 12, in an optional implementation, the decoded data generated by executing step 11 may be first stored in the local memory of the computing device on which the non-linear editing system is installed, so that the non-linear editing system may obtain the decoded data from the memory and then store the decoded data in the multi-level cache space. Of course, when the second storage space is the memory, and the decoded data of the frame image stored in the first storage space has reached the upper limit of the first storage space, the non-linear editing system may not perform the operation of obtaining the decoded data from the memory and storing it in the second storage space.

[0052] In an optional implementation, to achieve flexible collaborative storage of data, the first storage space may include: a first sub-storage space and a second sub-storage space. In a specific example, when the first storage space is a video memory, the first sub-storage space is a dedicated video memory, and the second sub-storage space is a shared video memory.

[0053] Among them, dedicated video memory refers to the memory used by the GPU of the computing device itself and not shared with other applications; while shared video memory refers to the memory that can be used by all applications but is used by the GPU first.

[0054] In an optional embodiment, when the first storage space includes a first sub-storage space and a second sub-storage space, the multi-level caching strategy described in the embodiment of the present disclosure may also include: if the decoded data of the frame image cached in the first sub-storage space has reached the upper limit of the first sub-storage space, then the decoded data of the target frame image is cached in the second sub-storage space.

[0055] When the first storage space includes the first sub-storage space and the second sub-storage space, the multi-level cache space in the embodiment of the present disclosure is equivalent to being composed of at least three types of storage spaces, and the three types of storage spaces are: the first sub-storage space, the second sub-storage space, and the second storage space. In a specific example, the first sub-storage space, the second sub-storage space, and the second storage space can be: dedicated video memory, shared video memory, and internal memory, respectively.

[0056] In the embodiment of the present disclosure, before storing the decoded data in the multi-level cache space, the non-linear editing system can implement the designation of the storage space as the multi-level cache space through the following steps:

[0057] First, the nonlinear editing system obtains the size of the currently available volatile storage space (including memory and video memory) of the computing device - for example, the size of the currently available volatile storage space can be obtained from the global resource management center (Global Resource Center) of the computing device, that is, the size of the currently free volatile storage space; wherein the global resource management center can be a component in the computing device that has the authority to obtain information such as the usage of the volatile storage space;

[0058] Then, the non-linear editing system can determine the multi-level cache space based on the size of the currently available volatile storage space (for example, 100M for memory and 50M for video memory) and the preset resource usage ratio (for example, 60%) - specifically 60% of the currently available 100M for memory, that is, 60M for memory; and 60% of the currently available 50M for video memory, that is, 30M for video memory.

[0059] After the multi-level cache space is determined, the decoded data can be stored in the multi-level cache space for subsequent query. It should be noted that the decoded data stored in the multi-level cache space can have an identifier of the decoded data, such as the frame number of the image frame corresponding to the decoded data; or the decoded data can be stored in the multi-level cache space corresponding to the frame number of the corresponding image frame for subsequent query.

[0060] It should be noted that, in the embodiment of the present disclosure, the purpose of storing the decoded data obtained by decoding the material track fragment in the multi-level cache space is mainly to take into account that if the non-linear editing system detects that a preview operation for the material track fragment is triggered on the video editing track, then when there is decoded data corresponding to the image to be previewed in the multi-level cache space, the decoded data can be directly obtained from the multi-level cache space without the need to decode the material track fragment in real time to obtain the decoded data, thereby ensuring the data acquisition efficiency of the video editing process.

[0061] The storage of decoded data in the disclosed embodiment has the following characteristics:

[0062] 1. The nonlinear editing system stores the decoded data obtained by decoding the images in the "material track fragments presented on the video editing track", rather than the decoded data obtained by decoding the materials that have not yet been loaded on the video editing track;

[0063] 2. The decoded data stored in the multi-level cache space is the decoded data obtained by decoding during the video editing process, rather than the decoded data obtained by decoding in other processes, nor is it the decoded data obtained by decoding the image that has not been previewed during the video editing process - therefore, by adopting the decoded data storage method provided by the embodiment of the present disclosure, there is no need to deliberately decode the image in advance to obtain the decoded data, but the decoded data can be stored naturally along with the image preview process.

[0064] The above two features are both based on the consideration that in the "non-linear editing" scenario, users are likely to reposition and preview the material track segments at any time, for example, performing a seek operation as described in the background technology. In such a scenario and demand background, the decoded data that has been decoded before the seek operation is stored, and when the seek operation occurs, the stored decoded data can be directly obtained for rendering and previewing the image, thereby improving the efficiency of non-linear editing.

[0065] Step 13: The non-linear editing system responds to the preview operation triggered on the video editing track, and if the timeline position of the target frame image in the material track segment is previewed, the target storage space is determined from the multi-level cache space;

[0066] Among them, the preview operation triggered on the video editing track may include but is not limited to the seek operation triggered on the video editing track, that is: by dragging the timeline on the video editing track, or by dragging the material loaded on the video editing track, the timeline position and the material change relative positions along the time axis direction of the video editing track, and then locating the target playback position of the material according to the timeline position, and then rendering and at least presenting the image frame in the material located at the target playback position.

[0067] Of course, in addition to operations by dragging the timeline or dragging the material, other operations that can cause the timeline position and the material to change in relative position along the time axis direction of the video editing track, thereby re-positioning the target playback position of the material according to the timeline position, and then rendering and at least presenting the image frame in the material at the target playback position, can all be classified as the "preview operation" described in the embodiments of the present disclosure.

[0068] To help readers understand the relationship between the preview operation triggered on the video editing track and the timeline position, please refer to the appendix of the manual. Figure 1b The attached Figure 1b , which is a schematic diagram of presenting a material track segment on a video editing track of a non-linear editing system. As shown in the figure, the image frames contained in the material track segment can be previewed in the preview window. Below the playback window are control buttons that support playback control of the video file.

[0069] In actual scenarios, users may directly drag Figure 1b The timeline in the figure, the dragging operation is the preview operation described in step 13; alternatively, the user can also long press the "rewind play" button in the control button, and the long press operation of the button is also the preview operation described in step 13.

[0070] In actual scenarios, the user can perform the preview operation in any manner, and the embodiments of the present disclosure do not limit how to perform the preview operation.

[0071] As for the target frame image described in step 13, it has been explained above that the "target frame image" does not specifically refer to a certain frame image, but generally refers to the image decoded by the linear editing system in the material track segment. In step 13, if the preview operation makes the timeline position coincide with the position of a certain image in the material track segment, then the linear editing system generally sequentially obtains the decoded data of the image, the decoded data of other images that belong to the same material track segment as the image but are arranged after the image, and the decoded data of images in other material track segments that are located on other video editing tracks and are arranged after the material track segment to which the image belongs, and sequentially renders the preview images of each image based on the obtained decoded data.

[0072] If, triggered by the preview operation, the decoded data to be acquired by the linear editing system includes the decoded data of the target image described in step 1, and the decoded data of the target image exists in the multi-level cache space, then the linear editing system can determine the target storage space from the multi-level cache space, and then execute the subsequent step 14; if, triggered by the preview operation, the decoded data to be acquired by the linear editing system does not include the decoded data of the target image described in step 1, or, although it includes the decoded data of the target image described in step 1, the decoded data does not yet exist in the multi-level cache space, then the linear editing system no longer executes the subsequent step 14, but acquires the decoded data according to the conventional decoded data acquisition method - for example, acquiring the image frame from the image frame currently loaded into the memory for decoding, and obtaining the decoded data.

[0073] In the disclosed embodiment, whether the decoded data of the target image exists in the multi-level cache space can be determined by the linear editing system by querying the multi-level cache space according to the identifier of the target image.

[0074] In the embodiment of the present disclosure, the identifier of the target image may refer to the number of the target image in the material track segment to which the target image belongs, for example, it may be the frame number of the image frame.

[0075] In the disclosed embodiment, assuming that the decoded data of the target frame image has been stored in the multi-level cache space by executing step 12 and has not yet been consumed, then any of the following two situations can trigger the non-linear editing system to determine the target storage space storing the target frame image from the multi-level cache space: 1. The timeline position just overlaps with the position of the target frame image; 2. The timeline position overlaps with the position of the image arranged before the target frame image in the material track segment.

[0076] The above-mentioned target storage space refers to a storage space used to store decoded data of a target frame image, which may be a first storage space or a second storage space.

[0077] As mentioned above, in the embodiment of the present disclosure, a multi-level cache strategy is adopted when storing decoded data. According to the multi-level cache strategy, the decoded data will be preferentially stored in the first storage space with a relatively high read and write speed until the first storage space reaches the storage limit, and then the decoded data will be stored in the second storage space.

[0078] It can be understood that from the perspective of ensuring the efficiency of obtaining decoded data, if the decoded data can always be read from the first storage space with a higher read / write speed, the efficiency of obtaining decoded data can be guaranteed to the greatest extent. Therefore, in the embodiment of the present disclosure, if the decoded data stored in the first storage space is consumed, the decoded data stored in the second storage space is transferred to the free storage position obtained by release in the first storage space in sequence according to the First In First Out (FIFO) rule.

[0079] Such a data transfer method is adopted in consideration of the process of previewing the image frames of the material track fragments on the video editing track, which conforms to the feature of "previewing in sequence starting from the timeline position". Based on this feature, if the decoded data of the previewed image frame can be moved from the second storage space to the first storage space in sequence in a FIFO manner, then, in theory, the decoded data can always be read (i.e., consumed) from the first storage space with a higher reading and writing speed, thereby ensuring the efficiency of decoding data acquisition and further ensuring the overall efficiency of the NLE.

[0080] Corresponding to the above-mentioned sequential transfer mode of the decoded data, an optional implementation mode of determining the target storage space from the multi-level cache space in step 13 may include the following sub-steps 1 and 2:

[0081] Sub-step 1: query the decoded data of the target frame image from the target storage space according to the data query strategy;

[0082] Sub-step 2: Determine the storage space where the decoded data of the queried target frame image is located as the target storage space.

[0083] The data query strategy may include: querying different storage spaces included in the multi-level cache space in order of the read and write speed of the storage space from large to small.

[0084] According to the data query strategy, if it is found that the decoded data of the target frame image exists in the multi-level cache space, then it can be determined in which storage space the decoded data is located, that is, the target storage space is determined, and then step 14 is executed.

[0085] Step 14: If the decoded data of the target frame image is stored in the multi-level cache space, the non-linear editing system reads the decoded data of the target frame image from the target storage space, and renders the target frame image based on the decoded data of the target frame image and displays a preview image.

[0086] In an optional implementation, when the decoded data of the target frame image is stored in step 12, the identifier of the target frame image and the decoded data may be stored correspondingly. Thus, in step 14, according to the identifier of the target frame image, the decoded data stored corresponding to the identifier may be read from the target storage space as the decoded data of the target frame image.

[0087] After the decoded data of the target frame image is read, the target frame image can be rendered based on the decoded data and a preview image can be displayed.

[0088] In an optional implementation, the non-linear editing system may read the decoded data in the target storage space into the graphics processor memory (referred to as video memory), so that the graphics processor may render a preview image corresponding to the target frame image based on the decoded data.

[0089] Considering that among the decoded data stored in the multi-level cache space, there may be decoded data that has not been acquired and used (acquisition and use can be called consumption) by the non-linear editing system from the multi-level cache space in the recent period of time. Such decoded data is "cold data", and cold data continues to occupy storage resources, which is equivalent to causing a waste of storage resources. Therefore, in an optional implementation, the non-linear editing system can clear the decoded data that meets the data elimination rule from the multi-level cache space according to the data elimination rule set based on the Least Recently Used (LRU) algorithm.

[0090] The LRU algorithm refers to replacing the least used data. In the disclosed embodiment, the data elimination rule set based on the LRU algorithm may include: eliminating the least used data relatively, and replacing the least used data with the decoded data obtained by the most recent decoding.

[0091] By adopting the above elimination method, it can be ensured that the storage resources of the multi-level cache space are fully utilized, avoiding the problem of infrequently used "cold data" occupying storage resources and causing resource waste.

[0092] In an optional implementation, when the non-linear editing system obtains decoded data for rendering, it is specifically implemented by the non-linear editing system triggering the graphics processor. For the graphics processor, the efficiency of obtaining decoded data from the video memory for rendering is higher than the efficiency of obtaining decoded data from the memory for rendering. This is because if the graphics processor obtains decoded data from the memory of the computing device, it will read the obtained decoded data from the memory to the video memory, and then obtain the decoded data from the video memory for rendering processing - therefore, from the perspective of the graphics processor, when both the memory and the video memory can be used to store decoded data, the decoded data is stored in the video memory first, which is more conducive to the graphics processor to efficiently obtain the decoded data that has been decoded.

[0093] Based on the above considerations, in the embodiment of the present disclosure, the video memory is determined as a storage space with a high consumption priority in the multi-level cache space, and the internal memory is determined as a secondary storage space with a low consumption priority in the multi-level cache space. The "consumption priority" mentioned here can be equivalent to the "reading and writing speed" mentioned above, specifically, for example, it can refer to the reading and writing speed when the graphics processor reads and writes data.

[0094] Furthermore, if the first storage space includes dedicated video memory (i.e., the first sub-storage space) and shared video memory (i.e., the second sub-storage space), the dedicated video memory can be further determined as the storage space with the highest consumption priority in the multi-level cache space, and the shared video memory can be determined as the storage space with the second highest consumption priority in the multi-level cache space.

[0095] It should be noted that the consumption priority of the multi-level cache space is equivalent to the query priority followed by the non-linear editing system when querying the decoded data stored in the multi-level cache space. The storage space with a higher consumption priority will be queried more preferentially when the non-linear editing system queries the decoded data.

[0096] Based on the different types of storage spaces contained in the multi-level cache space having different consumption priorities, in an optional implementation, the non-linear editing system can query the decoding data of the target frame image from the target storage space according to the data query strategy; and determine the storage space where the decoding data of the queried target frame image is located as the target storage space.

[0097] The data query strategy includes: querying different storage spaces included in the multi-level cache space in order from large to small according to the reading and writing speed (ie, consumption priority) of the storage space.

[0098] By adopting the above method provided by the embodiment of the present disclosure, a multi-level cache space including at least a first storage space and a second storage space is adopted to store the decoded data generated by decoding the target frame image in the material track segment. Based on this, in response to the preview operation triggered on the video editing track, when the timeline position of the target frame image in the material track segment is previewed, if the decoded data of the target frame image is stored in the multi-level cache space, then the decoded data of the target frame image can be obtained from the target storage space of the multi-level cache space to render the target frame and display the preview image, without the need to obtain the decoded data of the target frame image by real-time decoding of the video file as in the prior art, thereby improving the efficiency of obtaining the decoded data, and further improving the overall efficiency of the NLE.

[0099] In order to enable readers to better understand the method provided by the embodiment of the present disclosure, the specific implementation process of the above-mentioned decoding data acquisition method in a non-linear editing system is introduced below in combination with actual scenarios.

[0100] Please refer to the attached instructions Figure 2 , is a schematic diagram of a specific implementation process of the decoding data acquisition method in practice. The execution subject of each step in the process may be, for example, a non-linear editing system. The process mainly includes the following steps:

[0101] Step 31: The non-linear editing system obtains the size of the currently available dedicated video memory, the size of the shared video memory, and the size of the internal memory from the global resource management center of the local device (such as the user's personal computer) where the non-linear editing system is installed;

[0102] Step 32: The nonlinear editing system sets / designates the dedicated video memory, the shared video memory, and the internal memory as a decoding buffer pool, and sets 60% of the currently available storage space sizes of the dedicated video memory, the shared video memory, and the internal memory as the storage space size of the decoding buffer pool according to the default resource usage ratio of 60%;

[0103] In the disclosed embodiment, the consumption priority of dedicated video memory, shared video memory, and internal memory may be in the order of: dedicated video memory>shared video memory>internal memory.

[0104] Step 33: the non-linear editing system stores the decoded data of the video frame obtained by decoding the decoder in the process of non-linear editing the video file into the decoding buffer pool;

[0105] Among them, when storing the decoded data of the video frames in the decoding cache pool, the decoded data of the video frames can be stored in the decoding cache pool in the order of consumption priority from high to low, and in accordance with the rule of filling up different types of storage space (including dedicated video memory, shared video memory, and internal memory) contained in the decoding cache pool in sequence.

[0106] The term "full storage" mentioned here refers to filling up 60% of the currently available storage space of the dedicated video memory, using the above example.

[0107] Step 34: The non-linear editing system receives a fast-rewind instruction during the non-linear editing of the video file;

[0108] Step 35: the non-linear editing system responds to the instruction and queries the decoding buffer pool according to the frame number of the video frame to be decoded corresponding to the instruction;

[0109] If it is queried that the decoded data of the video frame with the frame number exists in the decoded buffer pool, the decoded data of the video frame with the frame number is directly obtained from the decoded buffer pool, and then based on the decoded data of the video frame, the operation of rendering and presenting the corresponding preview image is performed; wherein, the decoded data of the video frame obtained from the decoded buffer pool by the non-linear editing system will be deleted from the decoded buffer pool as the decoded data of the consumed video frame, and then, the decoded data of the video frame stored in the decoded buffer pool will be transferred in a FIFO manner - the details can be seen in the relevant description in the previous embodiment, which will not be repeated here.

[0110] If no decoded data of the video frame with the frame number is found in the decode buffer pool, step 36 is executed;

[0111] Step 36: the non-linear editing system decodes the video file loaded in the memory of the computing device to obtain decoded data of the video frame with the frame number, and then performs an operation of rendering and presenting a corresponding image based on the video frame; in addition, jump to step 37;

[0112] Step 37: The non-linear editing system stores the decoded data of the video frame with the frame number obtained by performing the decoding in step 36 into the decoding buffer pool.

[0113] It should be noted that the non-linear editing system can run a thread such as "Recyle Thread", which is used to monitor the usage of decoded data of video frames stored in the decoding cache pool; if the thread monitors that the decoded data of a certain / certain video frames stored in the decoding cache pool has been consumed the least number of times from the storage time to the current time compared with the decoded data of other video frames - for example, less than a preset threshold Cache Threshold, then the non-linear editing system can clear the decoded data of video frames that meet the data elimination rules from the decoding cache pool according to the data elimination rules determined based on the LRU algorithm, that is, for example, eliminate the decoded data of video frames that have been consumed the number of times less than the preset threshold CacheThreshold.

[0114] By adopting the above-mentioned scheme provided by the embodiment of the present disclosure, a decoding buffer pool is used to realize the storage of decoded data of video frames decoded in the process of decoding and presenting video files by a non-linear editing system. Based on this, when an instruction is received during the process to trigger decoding of a video file to obtain decoded data of a specified video frame, the decoded data of the specified video frame can be obtained from the decoding buffer pool, without the need to use real-time decoding of video files to obtain decoded data of the specified video frame as in the prior art, thereby improving the efficiency of obtaining decoded data of the specified video frame in the process.

[0115] The specific implementation process of the above solution can also be further referred to as Figure 3 Schematic diagram shown.

[0116] Based on the same inventive concept as the previous embodiments, in order to solve the problem that in the prior art, in response to a preview operation triggered on a video editing track, real-time decoding of materials is used to obtain decoding data of image frames matching the target playback position, which leads to low decoding data acquisition efficiency and thus low overall efficiency of the NLE, the disclosed embodiments further provide a decoding data acquisition device.

[0117] The specific structural diagram of the device is shown in Figure 4 As shown, it includes the following functional units:

[0118] The decoding unit 41 is used to decode the target frame image in the material track segment presented on the video editing track to generate decoded data of the target frame image;

[0119] The storage unit 42 is used to store the decoded data of the target frame image in a multi-level cache space according to a multi-level cache strategy; wherein the multi-level cache space at least includes a first storage space and a second storage space, and the read and write speed of the first storage space is greater than the read and write speed of the second storage space; the multi-level cache strategy includes: if the decoded data of the frame image cached in the first storage space does not reach the upper limit of the first storage space, then the decoded data of the target frame image is cached in the first storage space; if the decoded data of the frame image cached in the first storage space has reached the upper limit of the first storage space, then the decoded data of the target frame image is cached in the second storage space;

[0120] A storage space determination unit 43 is used to respond to a preview operation triggered on a video editing track, and if the timeline position of the target frame image in the material track segment is previewed, determine a target storage space from the multi-level cache space, wherein the target storage space is used to store decoded data of the target frame image;

[0121] The data acquisition and display unit 44 is used to read the decoded data of the target frame image from the target storage space if the decoded data of the target frame image is stored in the multi-level cache space, and render the target frame image based on the decoded data and display a preview image.

[0122] In an optional implementation manner, the storage space determining unit 43 may be configured to:

[0123] According to the data query strategy, query the decoded data of the target frame image from the target storage space;

[0124] The storage space where the decoded data of the target frame image is found is determined as the target storage space.

[0125] The data query strategy includes: querying different storage spaces included in the multi-level cache space in sequence according to the order of the read and write speed of the storage space from large to small.

[0126] In an optional implementation, the device provided by the embodiment of the present disclosure may further include:

[0127] The data transfer unit is used to transfer the decoded data stored in the second storage space to the free storage positions released in the first storage space in sequence according to the first-in-first-out (FIFO) rule if the decoded data stored in the first storage space is consumed.

[0128] In an optional implementation, the first storage space is a dedicated storage space; the second storage space is a shared storage space; or, the first storage space is a video memory; and the second storage space is a memory.

[0129] In an optional implementation, the first storage space includes: a first sub-storage space and a second sub-storage space;

[0130] When the first storage space is a video memory, the first sub-storage space is a dedicated video memory, and the second sub-storage space is a shared video memory;

[0131] The multi-level cache strategy further includes: if the decoded data of the frame image cached in the first sub-storage space has reached the upper limit of the first sub-storage space, then caching the decoded data of the target frame image in the second sub-storage space.

[0132] In an optional implementation, the device provided by the embodiment of the present disclosure may further include:

[0133] The data clearing unit is used to clear the decoded data that meets the data elimination rule from the multi-level cache space according to the data elimination rule set based on the least recently used LRU algorithm.

[0134] The above-mentioned decoding data acquisition device provided by the embodiment of the present disclosure adopts a multi-level cache space including at least a first storage space and a second storage space to store the decoding data generated by decoding the target frame image in the material track segment. Based on this, in response to the preview operation triggered on the video editing track, when the timeline position of the target frame image in the material track segment is previewed, if the decoding data of the target frame image is stored in the multi-level cache space, then the decoding data of the target frame image can be obtained from the target storage space of the multi-level cache space to render the target frame and display the preview image, without the need to obtain the decoding data of the target frame image by real-time decoding of the video file as in the prior art, thereby improving the efficiency of obtaining the decoding data, and further improving the overall efficiency of the NLE.

[0135] Based on the same inventive concept as the previous embodiments, the embodiments of the present disclosure also provide an electronic device to solve the problem in the prior art that, in response to a preview operation triggered on a video editing track, real-time decoding of materials is used to obtain decoding data of image frames matching a target playback position, which results in low efficiency in obtaining decoding data and further leads to low overall efficiency of the NLE.

[0136] like Figure 5 As shown, the electronic device includes: a memory 51 and a processor 52. The memory 51 can be configured to store various other data to support operations on the electronic device. Examples of these data include instructions for any application or method for operating on the electronic device. The memory 51 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk or an optical disk.

[0137] The processor 52 is coupled to the memory 51 and is used to execute the program stored in the memory 51 to execute the decoding data acquisition method described in the embodiment of the present disclosure, or to execute the multimedia data storage method described in the embodiment of the present disclosure.

[0138] When the processor 52 executes the program in the memory 51, in addition to the above functions, it can also realize other functions, and the details can be referred to the description of the previous embodiments.

[0139] Further, such as Figure 5As shown, the electronic device also includes: a display 54, a communication component 53, a power component 55, an audio component 56 and other components. Figure 5 Only some components are shown schematically, which does not mean that the electronic device only includes Figure 5 Components shown.

[0140] Accordingly, an embodiment of the present disclosure further provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a computer, the steps or functions of the methods provided in the above embodiments can be implemented.

[0141] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0142] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0143] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present disclosure.

Claims

1. A method for obtaining decoded data, characterized in that: include: For a material track segment presented on a video editing track, decoding a target frame image in the material track segment to generate decoded data of the target frame image; According to the multi-level cache strategy, the decoded data of the target frame image is stored in the multi-level cache space; wherein the multi-level cache space at least includes a first storage space and a second storage space, and the read and write speed of the first storage space is greater than the read and write speed of the second storage space; the multi-level cache strategy includes: if the decoded data of the frame image cached in the first storage space does not reach the upper limit of the first storage space, then the decoded data of the target frame image is cached in the first storage space; if the decoded data of the frame image cached in the first storage space has reached the upper limit of the first storage space, then the decoded data of the target frame image is cached in the second storage space; In response to a preview operation triggered on a video editing track, if the timeline position of the target frame image in the material track segment is previewed, a target storage space is determined from the multi-level cache space, and the target storage space is used to store decoded data of the target frame image; If the decoded data of the target frame image is stored in the multi-level cache space, the decoded data of the target frame image is read from the target storage space, and the target frame image is rendered based on the decoded data and a preview image is displayed.

2. The method according to claim 1, characterized in that Determining a target storage space from the multi-level cache space includes: According to the data query strategy, query the decoded data of the target frame image from the target storage space; Determine the storage space where the decoded data of the target frame image is located as the target storage space; The data query strategy includes: querying different storage spaces included in the multi-level cache space in sequence according to the order of the read and write speed of the storage space from large to small.

3. The method according to claim 2, characterized in that The method further comprises: If the decoded data stored in the first storage space is consumed, the decoded data stored in the second storage space is sequentially transferred to the free storage positions released in the first storage space according to the first-in-first-out (FIFO) rule.

4. The method according to claim 1, characterized in that The first storage space is a dedicated storage space; the second storage space is a shared storage space; or, The first storage space is video memory; the second storage space is internal memory.

5. The method according to claim 1 or 4, characterized in that The first storage space includes: a first sub-storage space and a second sub-storage space; When the first storage space is a video memory, the first sub-storage space is a dedicated video memory, and the second sub-storage space is a shared video memory; The multi-level cache strategy further includes: if the decoded data of the frame image cached in the first sub-storage space has reached the upper limit of the first sub-storage space, then caching the decoded data of the target frame image in the second sub-storage space.

6. The method according to claim 1, characterized in that The method further comprises: According to the data elimination rule set based on the least recently used (LRU) algorithm, the decoded data satisfying the data elimination rule is cleared from the multi-level cache space.

7. A decoding data acquisition device, characterized in that: include: A decoding unit, configured to decode a target frame image in a material track segment presented on a video editing track, and generate decoded data of the target frame image; A storage unit, for storing the decoded data of the target frame image in a multi-level cache space according to a multi-level cache strategy; wherein the multi-level cache space at least includes a first storage space and a second storage space, and the read and write speed of the first storage space is greater than the read and write speed of the second storage space; the multi-level cache strategy includes: if the decoded data of the frame image cached in the first storage space does not reach the upper limit of the first storage space, then the decoded data of the target frame image is cached in the first storage space; if the decoded data of the frame image cached in the first storage space has reached the upper limit of the first storage space, then the decoded data of the target frame image is cached in the second storage space; a storage space determination unit, configured to respond to a preview operation triggered on a video editing track, and if the timeline position of the target frame image in the material track segment is previewed, determine a target storage space from the multi-level cache space, wherein the target storage space is used to store decoded data of the target frame image; A data acquisition and display unit is used to read the decoded data of the target frame image from the target storage space if the decoded data of the target frame image is stored in the multi-level cache space, and render the target frame image based on the decoded data and display a preview image.

8. The device according to claim 7, characterized in that The storage space determining unit is used to: According to the data query strategy, query the decoded data of the target frame image from the target storage space; Determine the storage space where the decoded data of the target frame image is located as the target storage space; The data query strategy includes: querying different storage spaces included in the multi-level cache space in sequence according to the order of the read and write speed of the storage space from large to small.

9. The device according to claim 8, characterized in that The device also includes: The data transfer unit is used to transfer the decoded data stored in the second storage space to the free storage positions released in the first storage space in sequence according to the first-in-first-out (FIFO) rule if the decoded data stored in the first storage space is consumed.

10. The device according to claim 7, characterized in that The first storage space is a dedicated storage space; the second storage space is a shared storage space; or, The first storage space is video memory; the second storage space is internal memory.

11. The device according to claim 7 or 10, characterized in that The first storage space includes: a first sub-storage space and a second sub-storage space; When the first storage space is a video memory, the first sub-storage space is a dedicated video memory, and the second sub-storage space is a shared video memory; The multi-level cache strategy further includes: if the decoded data of the frame image cached in the first sub-storage space has reached the upper limit of the first sub-storage space, then caching the decoded data of the target frame image in the second sub-storage space.

12. The device according to claim 7, characterized in that The device also includes: The data clearing unit is used to clear the decoded data that meets the data elimination rule from the multi-level cache space according to the data elimination rule set based on the least recently used LRU algorithm.

13. An electronic device, characterized in that: include: Memory and processor, wherein: The memory is used to store programs; The processor is coupled to the memory and is used to execute the program stored in the memory to execute the decoding data acquisition method described in any one of claims 1 to 6.

14. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a computer, can implement the method for obtaining decoded data as claimed in any one of claims 1 to 6.

Images