Thumbnail display method and device, electronic equipment, storage medium and program product

Abstract

The present disclosure relates to a thumbnail display method and device, electronic equipment, storage medium and program product. The method comprises: displaying a first track segment corresponding to a target video material on a video editing track, wherein a thumbnail of a first video frame in the target video material is presented in the first track segment; in response to a target operation, generating a thumbnail refresh request, wherein the target operation is used to trigger a display area of the target video material on the video editing track to change from the first track segment to a second track segment, and the thumbnail refresh request is used to obtain a thumbnail of a second video frame in the target video material; based on the thumbnail refresh request, generating the thumbnail of the second video frame by decoding the target video material; and displaying the thumbnail of the second video frame in the second track segment presented on the video editing track.

Description

Technical Field

[0001] This disclosure relates to the field of video processing technology, and in particular to a thumbnail display method, apparatus, electronic device, storage medium, and program product. Background Technology

[0002] With the rapid development of computer and mobile communication technologies, more and more users are happy to share their video creations on video platforms for other users to watch. Before sharing a video, users need to edit and process the video footage to obtain the video to be shared.

[0003] During video editing, it's necessary to refresh the thumbnails of video frames displayed on the editing track by performing targeted operations on the track, thus enabling editing of the video footage. Currently, the common approach is to request these thumbnails from the server via HTTP or other network interfaces before initially displaying them on the editing track. The server then returns these thumbnails for display. However, since the server can only return a fixed number of thumbnails at a time, refreshing the thumbnails on the editing track, especially when increasing the thumbnail density, involves repeatedly filling the space with the server-returned thumbnails.

[0004] Thus, although the above solution changes the density of thumbnails of video frames on the video editing track, the display of thumbnails on the video editing track does not increase the detail of the video footage (new video frame thumbnails), resulting in low efficiency for users to edit video footage based on thumbnails and a poor user experience. Summary of the Invention

[0005] To solve the above-mentioned technical problems, or at least partially solve them, this disclosure provides a thumbnail display method, apparatus, electronic device, storage medium, and program product.

[0006] A first aspect of this disclosure provides a thumbnail display method, the method comprising: displaying a first track segment corresponding to a target video material on a video editing track, wherein a thumbnail of a first video frame in the target video material is presented in the first track segment; generating a thumbnail refresh request in response to a target operation, the target operation being used to trigger a change in the display area of ​​the target video material on the video editing track from the first track segment to a second track segment, the thumbnail refresh request being used to obtain a thumbnail of a second video frame in the target video material; generating a thumbnail of the second video frame by decoding the target video material based on the thumbnail refresh request; and displaying the thumbnail of the second video frame in the second track segment presented on the video editing track.

[0007] In some embodiments of this disclosure, the target operation includes any one of the following: scaling down the video editing track, scaling up the video editing track, moving forward on the video editing track, and moving backward on the video editing track.

[0008] In some embodiments of this disclosure, before generating a thumbnail of the second video frame by decoding the target video material based on the thumbnail refresh request, the method further includes: displaying a thumbnail of a third video frame in the target video material in a second track segment, wherein the thumbnail of the third video frame is a cached thumbnail whose timestamp is closest to the timestamp of the thumbnail of the second video frame before the target operation; displaying the thumbnail of the second video frame in the second track segment presented on the video editing track includes: updating the thumbnail of the third video frame displayed in the second track segment to the thumbnail of the second video frame.

[0009] In some embodiments of this disclosure, generating a thumbnail refresh request in response to a target operation includes: generating the thumbnail refresh request via a first thread in response to the target operation; sending the thumbnail refresh request to a second thread via the first thread; and generating a thumbnail of a second video frame by decoding the target video material based on the thumbnail refresh request, including: decoding the target video material based on the thumbnail refresh request via the second thread to generate a thumbnail of the second video frame.

[0010] In some embodiments of this disclosure, the process of decoding the target video material based on the thumbnail refresh request using a second thread to generate a thumbnail of the second video frame includes: adding a target task to a task buffer queue based on the thumbnail refresh request using a second thread, the target task being used to indicate the acquisition of a thumbnail of the second video frame; retrieving the target task from the task buffer queue using a second thread and adding the target task to a task processing queue; retrieving the target task from the task processing queue using a second thread and decoding the target video material to generate a thumbnail of the second video frame; and adding the thumbnail of the second video frame to a queue to be rendered using a second thread. The process of displaying the thumbnail of the second video frame in a second track segment presented on the video editing track includes: retrieving the thumbnail of the second video frame from the queue to be rendered using a first thread and displaying the thumbnail of the second video frame in a second track segment.

[0011] In some embodiments of this disclosure, after adding the thumbnail of the second video frame to the queue to be rendered via a second thread, the method further includes: deleting the target task from the task processing queue via a second thread.

[0012] In some embodiments of this disclosure, adding the target task to the task buffer queue via the second thread based on the thumbnail refresh request includes: in response to the task buffer queue not being full, adding the target task to the task buffer queue via the second thread; in response to the task buffer queue being full, deleting the first task in the task buffer queue via the second thread and adding the target task to the task buffer queue.

[0013] In some embodiments of this disclosure, the method of generating a thumbnail of a second video frame by decoding the target video material based on the thumbnail refresh request includes: decoding the target video material to generate a thumbnail of a second video frame in response to the absence of a cached thumbnail of a second video frame based on the thumbnail refresh request; the method further includes: obtaining a cached thumbnail of a second video frame in response to the presence of a cached thumbnail of a second video frame.

[0014] A second aspect of this disclosure provides a thumbnail display device, comprising: a display module for displaying a first track segment corresponding to a target video material on a video editing track, wherein a thumbnail of a first video frame in the target video material is presented in the first track segment; a generation module for generating a thumbnail refresh request in response to a target operation, wherein the target operation is used to trigger a change in the display area of ​​the target video material on the video editing track from the first track segment to a second track segment, and the thumbnail refresh request is used to obtain a thumbnail of a second video frame in the target video material; and generating a thumbnail of the second video frame by decoding the target video material based on the thumbnail refresh request; the display module is further configured to display the thumbnail of the second video frame in the second track segment presented on the video editing track.

[0015] In some embodiments of this disclosure, the target operation includes any one of the following: scaling down the video editing track, scaling up the video editing track, moving forward on the video editing track, and moving backward on the video editing track.

[0016] In some embodiments of this disclosure, the display module is further configured to display a thumbnail of a third video frame in the target video material in the second track segment before generating a thumbnail of the second video frame by decoding the target video material based on the thumbnail refresh request. The thumbnail of the third video frame is a cached thumbnail whose timestamp is closest to the timestamp of the thumbnail of the second video frame before the target operation. Specifically, the display module is configured to update the thumbnail of the third video frame displayed in the second track segment to the thumbnail of the second video frame.

[0017] In some embodiments of this disclosure, the generation module is specifically configured to generate the thumbnail refresh request in response to the target operation via a first thread; send the thumbnail refresh request to a second thread via the first thread; and decode the target video material based on the thumbnail refresh request via the second thread to generate a thumbnail of the second video frame.

[0018] In some embodiments of this disclosure, the generation module is specifically configured to add a target task to a task buffer queue based on the thumbnail refresh request via a second thread, the target task being used to indicate the acquisition of a thumbnail of a second video frame; extract the target task from the task buffer queue via the second thread and add the target task to a task processing queue; obtain the target task from the task processing queue via the second thread and decode the target video material to generate a thumbnail of the second video frame; and add the thumbnail of the second video frame to a queue to be rendered via the second thread. The display module is specifically configured to obtain the thumbnail of the second video frame from the queue to be rendered via a first thread and display the thumbnail of the second video frame in a second track segment.

[0019] In some embodiments of this disclosure, the apparatus further includes a deletion module, configured to delete the target task from the task processing queue via a second thread after the thumbnail of the second video frame has been added to the queue to be rendered via a second thread.

[0020] In some embodiments of this disclosure, the generation module is specifically configured to add the target task to the task buffer queue in response to the second thread when the task cache queue is not full; and to delete the first task in the task cache queue and add the target task to the task buffer queue in response to the second thread when the task cache queue is full.

[0021] In some embodiments of this disclosure, the generation module is specifically used to decode the target video material based on the thumbnail refresh request and in response to a thumbnail of a second video frame that does not exist in the cache, and generate a thumbnail of the second video frame; the apparatus further includes: an acquisition module, used to acquire a cached thumbnail of the second video frame in response to a cached thumbnail of the second video frame.

[0022] A third aspect of this disclosure provides an electronic device including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the thumbnail display method as described in the first aspect.

[0023] A fourth aspect of this disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the thumbnail display method as described in the first aspect.

[0024] A fifth aspect of this disclosure provides a computer program product, wherein the computer program product includes a computer program that, when the computer program product is run on a processor, causes the processor to execute the computer program to implement the thumbnail display method as described in the first aspect.

[0025] A sixth aspect of this disclosure provides a chip including a processor and a communication interface coupled to the processor, the processor being used to execute program instructions to implement the thumbnail display method as described in the first aspect.

[0026] The technical solution provided in this disclosure has the following advantages compared with the prior art: A first track segment corresponding to the target video material is displayed on the video editing track, and a thumbnail of the first video frame in the target video material is presented in the first track segment; in response to a target operation, a thumbnail refresh request is generated, the target operation being used to trigger the display area of ​​the target video material on the video editing track to change from the first track segment to the second track segment, and the thumbnail refresh request being used to obtain a thumbnail of the second video frame in the target video material; based on the thumbnail refresh request, the target video material is decoded to generate a thumbnail of the second video frame; and the thumbnail of the second video frame is displayed in the second track segment presented on the video editing track. In this way, by generating a thumbnail refresh request in response to the target operation and decoding the target video material based on the thumbnail refresh request, when it is necessary to refresh the thumbnails of the video frames displayed on the video editing track, especially when increasing the display density of thumbnails on the video editing track, more thumbnails in the video material can be quickly obtained and displayed to improve the display details of the thumbnails, thereby improving the efficiency of users editing video material based on thumbnails and improving the user experience. Attached Figure Description

[0027] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0028] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 A flowchart illustrating a thumbnail display method provided in an embodiment of this disclosure;

[0030] Figure 2 One of the interface diagrams for the thumbnail display method provided in the embodiments of this disclosure;

[0031] Figure 3 A second schematic diagram of the interface for the thumbnail display method provided in this embodiment of the disclosure;

[0032] Figure 4 The third schematic diagram of the interface for the thumbnail display method provided in this embodiment of the disclosure;

[0033] Figure 5 The fourth schematic diagram of the interface for the thumbnail display method provided in this embodiment of the disclosure;

[0034] Figure 6 A structural block diagram of a thumbnail display device provided in an embodiment of this disclosure;

[0035] Figure 7 This is a structural block diagram of an electronic device provided in an embodiment of the present disclosure. Detailed Implementation

[0036] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0037] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0038] The terms "first," "second," etc., used in this disclosure and in the claims are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this disclosure can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0039] The electronic devices in this disclosure can be mobile electronic devices or non-mobile electronic devices. Mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc.; non-mobile electronic devices can be personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc.; this disclosure does not impose specific limitations.

[0040] The subject executing the thumbnail display method provided in this disclosure can be the aforementioned electronic device (including mobile electronic devices and non-mobile electronic devices), or it can be a functional module and / or functional entity in the electronic device that can implement the thumbnail display method. The specific implementation can be determined according to actual usage requirements, and this disclosure does not limit it.

[0041] The thumbnail display method provided by the present disclosure will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0042] like Figure 1 As shown in the figure, this disclosure provides a thumbnail display method, which may include the following steps 100 to 103.

[0043] 100. Display the first track segment corresponding to the target video material on the video editing track. The first track segment presents a thumbnail of the first video frame in the target video material.

[0044] The first track segment presents thumbnails of multiple first video frames from the target video material, and the thumbnail of each first video frame is any one of the multiple thumbnails of the first video frames.

[0045] The first track segment refers to the presentation status of the first video segment in the target video material within the display area of ​​the video editing track. The first track segment is a thumbnail of the first video frame generated based on a portion of the video frame images in the first video segment, and it indicates the positional information of the first video segment within the target video material.

[0046] 101. In response to the target operation, generate a thumbnail refresh request.

[0047] Specifically, the target operation is used to trigger the display area of ​​the target video material on the video editing track to change from the first track segment to the second track segment, and the thumbnail refresh request is used to obtain the thumbnail of the second video frame in the target video material.

[0048] The thumbnail refresh request can be used to obtain thumbnails of one or more second video frames. When the thumbnail refresh request is used to indicate the acquisition of thumbnails of multiple second video frames, the thumbnails of the multiple second video frames may include thumbnails of cached video frames or thumbnails of uncached video frames, which is not limited here.

[0049] The second track segment refers to the presentation status of the second video segment in the target video footage within the display area of ​​the video editing track. The second track segment is a thumbnail of a second video frame generated based on a portion of the video frame images from the second video segment; it indicates the positional information of the second video segment within the target video footage.

[0050] It can be understood that the first track segment and the second track segment represent the presentation status of video segments at different locations in the target video material on the display area of ​​the video editing track.

[0051] 102. Based on the thumbnail refresh request, the target video material is decoded to generate a thumbnail of the second video frame.

[0052] The thumbnail refresh request carries the timestamp of the thumbnail of the second video frame, so that the target video material can be decoded according to the timestamp of the thumbnail of the second video frame, the original video frame image corresponding to the timestamp of the thumbnail of the second video frame can be obtained, and then the thumbnail of the second video frame can be generated based on the original video frame image.

[0053] In some embodiments of this disclosure, step 102 specifically involves calling the thumbnail dynamic acquisition interface to decode the target video material based on the thumbnail refresh request, and generating a thumbnail of the second video frame.

[0054] The dynamic thumbnail retrieval interface can be implemented based on the WebAssembly SDK.

[0055] In this embodiment, it is not necessary to play the target video material. After decoding the target video material, frame extraction can be performed on the decoded target video material to obtain the original video frames, and a thumbnail of the second video frame can be generated based on the original video frames. Compared with the scheme of finding the original video frame corresponding to the thumbnail of the second video frame during the playback of the target video material and generating the thumbnail of the second video frame, processing overhead can be saved and the time spent obtaining the thumbnail can be reduced.

[0056] In this embodiment of the disclosure, a dynamic thumbnail acquisition interface is developed locally on the electronic device based on the WebAssembly SDK. This allows for the dynamic acquisition of video frame thumbnails based on thumbnail refresh requests. This eliminates the need for repeated filling of cached video frame thumbnails, pre-requests from the server, and network requests (HTTP requests or other network interface requests). It can be used in offline environments (avoiding the time overhead of network requests and not being limited by the server's fixed frame extraction length), thus improving the efficiency of video frame thumbnail acquisition. Furthermore, the performance advantages of WebAssembly are utilized for decoding (high decoding efficiency), further enhancing the efficiency of video frame thumbnail acquisition.

[0057] 103. Display a thumbnail of the second video frame in the second track clip presented on the video editing track.

[0058] In this embodiment of the disclosure, by generating a thumbnail refresh request in response to a target operation and decoding the video material based on the thumbnail refresh request, when it is necessary to refresh the thumbnails of the video frames displayed on the video editing track, especially when increasing the display density of thumbnails on the video editing track, more thumbnails in the video material can be quickly obtained and displayed to improve the display details of the thumbnails, thereby improving the efficiency of users editing video material based on thumbnails and improving the user experience.

[0059] In some embodiments of this disclosure, the target operation includes any one of the following: scaling down the video editing track, scaling up the video editing track, moving forward on the video editing track, and moving backward on the video editing track.

[0060] In some embodiments of this disclosure, when the target operation is a forward operation or a backward operation of the video editing track, a scaling down operation and / or a scaling up operation of the video editing track may be included before the target operation, which is not limited here.

[0061] In some embodiments of this disclosure, when the target operation is a scaling down operation of the video editing track, one or more of the following operations may be included before the target operation: a scaling up operation of the video editing track, a forward operation of the video editing track, and a backward operation of the video editing track. This is not limited here.

[0062] Specifically, when the video editing track is zoomed in: the precision and size of the video editing track increase, the duration of individual thumbnails decreases, and the density of thumbnails increases, facilitating more detailed editing and previewing of video footage. When the video editing track is zoomed out: the precision and size of the video editing track decrease, and the duration of individual thumbnails increases.

[0063] It is understandable that, in response to the requirement of obtaining thumbnails of video frames other than cached video frames in the target video material in response to the target operation, the thumbnails of the second video frame can be obtained and displayed through the above steps 100 to 103.

[0064] In this embodiment of the disclosure, in response to different target operations, the thumbnail of the second video frame can be dynamically obtained based on the thumbnail refresh request, thereby improving the display details of the thumbnail on the video editing track and enhancing the user experience.

[0065] For example, such as Figure 2 The diagram shows a schematic of thumbnails for video frames on a video editing track. In the diagram, the video frame thumbnails are simply referred to as thumbnails. The area enclosed by the dashed box represents the visible range (display area) of the video editing track. The thumbnails of the video frames displayed within the dashed box are thumbnails of the video frames presented in the track segment of video footage within the visible range of the video editing track. The duration of one video frame thumbnail is 1 second. Combined with... Figure 2 ,like Figure 3 As shown, when the video editing track is zoomed in, the thumbnail length of the video frames within the visible area of ​​the video editing track becomes shorter (the thumbnail duration becomes 0.8s), and the density increases. Combined with... Figure 3 ,like Figure 4 As shown, when the video editing track is rewound, thumbnails of video frames that are not within the visible range on the video editing track are dynamically loaded and displayed within the visible range of the video editing track. Combined with... Figure 3 ,like Figure 5 As shown, when the forward operation of the video editing track is performed, thumbnails located outside the visible area on the video editing track are dynamically loaded and displayed within the visible area of ​​the video editing track.

[0066] In some embodiments of this disclosure, before step 102 above, the thumbnail display method provided by the embodiments of this disclosure may further include the following step 104, and the above step 103 may be implemented through the following step 103a.

[0067] 104. Display a thumbnail of the third video frame in the target video footage in the second track segment.

[0068] The thumbnail of the third video frame is a cached thumbnail whose timestamp is closest to the timestamp of the thumbnail of the second video frame before the target operation.

[0069] The thumbnail of the third video frame can be a cached thumbnail whose timestamp is before the timestamp of the thumbnail of the second video frame and whose timestamp is closest to the timestamp of the thumbnail of the second video frame; the thumbnail of the third video frame can also be a cached thumbnail whose timestamp is after the timestamp of the thumbnail of the second video frame and whose timestamp is closest to the timestamp of the thumbnail of the second video frame.

[0070] The thumbnail of the third video frame can be a thumbnail of the first video frame or a thumbnail of other video frames; there is no limitation here.

[0071] 103a. Update the thumbnail of the third video frame displayed in the second track segment to the thumbnail of the second video frame.

[0072] In this embodiment of the disclosure, since it takes a certain amount of time to obtain the thumbnail of the second video frame, before displaying the thumbnail of the second video frame, the video editing track is first filled with the cached thumbnail of the third video frame whose timestamp is closest to the timestamp of the thumbnail of the second video frame. This can avoid displaying a loading status or a blank area on the video editing track before displaying the thumbnail of the second video frame, thereby improving the user experience.

[0073] In some embodiments of this disclosure, the target video material can be decoded based on a thumbnail refresh request by a thread that renders and displays the thumbnail to generate a thumbnail of the second video frame. Alternatively, the target video material can be decoded based on a thumbnail refresh request by an independent thread that is different from the thread that renders and displays the thumbnail to generate a thumbnail of the second video frame. No limitation is made here.

[0074] In some embodiments of this disclosure, step 101 can be implemented by steps 101a and 101b; and step 102 can be implemented by step 102a.

[0075] 101a. The thumbnail refresh request is generated by the first thread in response to the target operation.

[0076] 101b. The thumbnail refresh request is sent to the second thread through the first thread.

[0077] 102a. Based on the thumbnail refresh request, the second thread decodes the target video material and generates a thumbnail of the second video frame.

[0078] The first thread can be the thread that renders and displays thumbnails, while the second thread is different from the first thread.

[0079] In this embodiment of the disclosure, by using an independent second thread to decode the target video material based on the thumbnail refresh request and generate a thumbnail of the second video frame, the execution of the processing task of the first thread can be avoided. Moreover, by using an independent second thread to decode (i.e., asynchronous thumbnail generation), the efficiency of thumbnail acquisition can be improved.

[0080] In some embodiments of this disclosure, step 102a can be implemented by steps 102a1 to 102a4, and step 103 can be implemented by step 103b.

[0081] 102a1. The target task is added to the task buffer queue based on the thumbnail refresh request via the second thread.

[0082] The objective task is to instruct the acquisition of a thumbnail of the second video frame.

[0083] The task buffer queue is used to buffer thumbnail retrieval tasks. In some cases, a large number of thumbnail retrieval tasks may be triggered in a short period of time. Therefore, caching thumbnail retrieval tasks through the task buffer queue can prevent a large number of thumbnail tasks that cannot be processed in time from being overwritten, resulting in the loss of thumbnail tasks.

[0084] 102a2. The target task is retrieved from the task cache queue by the second thread and added to the task processing queue.

[0085] In this embodiment of the disclosure, the task processing queue is set to a maximum number of parallel tasks, thereby reducing the overhead of obtaining thumbnails by limiting the number of parallel processing tasks.

[0086] 102a3. The second thread retrieves the target task from the task processing queue, decodes the target video material, and generates a thumbnail of the second video frame.

[0087] 102a4. Add the thumbnail of the second video frame to the queue to be rendered via the second thread.

[0088] In some embodiments of this disclosure, generated thumbnails are cached in a queue to be rendered. This can prevent thumbnails from being lost due to their inability to be rendered and displayed in a timely manner. It can also count the number of thumbnails generated and compare it with the number of thumbnails requested by the thumbnail refresh request to determine the thumbnail generation status.

[0089] In some embodiments of this disclosure, the length of the queue to be rendered is usually quite long, and the queue to be rendered is generally not full. The length of the queue to be rendered can be determined based on the relationship between the generation time of the thumbnail of a video frame and the rendering and display time of the thumbnail of a video frame. No limitation is made here.

[0090] 103b. Obtain the thumbnail of the second video frame from the queue to be rendered through the first thread, and display the thumbnail of the second video frame on the video editing track.

[0091] In this embodiment of the disclosure, by setting up a task buffer queue, a task processing queue, and a queue to be rendered, the generation and rendering of the thumbnail of the second video frame can be guaranteed, thereby improving the thumbnail refresh efficiency.

[0092] In some embodiments of this disclosure, after step 102a3 described above, the thumbnail display method provided by the embodiments of this disclosure may further include step 102a5 described below.

[0093] 102a5. Delete the target task from the task processing queue through the second thread.

[0094] In some embodiments of this disclosure, after generating the thumbnail of the second video frame, deleting the target task from the task processing queue can facilitate the entry of tasks in the task cache queue into the task processing queue for execution.

[0095] In some embodiments of this disclosure, the length of the task cache queue can be set according to actual needs and is not limited here. For example, if all thumbnail retrieval tasks need to be executed, the length of the task cache queue can be set to a larger value; if when there are many thumbnail retrieval tasks, some earlier thumbnail retrieval tasks need to be overwritten or discarded to avoid processing expired tasks (thumbs of video frames that no longer need to be displayed on the video editing track, i.e., thumbnails of video frames outside the second track segment), the length of the task cache queue can be set to a smaller value.

[0096] In some embodiments of this disclosure, step 102a1 can be specifically implemented by the following steps 201 and 202.

[0097] 201. In response to the fact that the task cache queue is not full, the target task is added to the task cache queue through a second thread.

[0098] 202. In response to the task cache queue being full, the first task in the task cache queue is deleted by the second thread, and the target task is added to the task buffer queue.

[0099] In some embodiments of this disclosure, when there are many thumbnail retrieval tasks, some earlier thumbnails may no longer need to be rendered. In this case, when the task cache queue is full, the oldest thumbnail retrieval task in the task cache queue can be deleted, and the latest thumbnail retrieval task can be added to the task cache queue. This can reduce the thumbnail retrieval overhead and improve the execution efficiency of new thumbnail retrieval tasks.

[0100] In some embodiments of this disclosure, thumbnails of all video frames in the thumbnail refresh request can be obtained by decoding the video material, regardless of whether there is a cache.

[0101] In some embodiments of this disclosure, step 102 can be implemented by steps 301 and 302 as described below. After step 301, the thumbnail display method provided by the embodiments of this disclosure may further include step 303 as described below.

[0102] 301. Based on the thumbnail refresh request, determine whether there is a cached thumbnail of the second video frame.

[0103] 302. Based on the thumbnail refresh request, in response to the thumbnail of the second video frame that does not exist in the cache, decode the target video material and generate the thumbnail of the second video frame.

[0104] 303. In response to the existence of a cached thumbnail of the second video frame, obtain the cached thumbnail of the second video frame.

[0105] In some embodiments of this disclosure, when there is no cached thumbnail of the second video frame, the target video material is decoded and frame-stripped to generate a thumbnail of the second video frame. When there is a cached thumbnail of the second video frame, obtaining the cached thumbnail of the second video frame can improve the thumbnail acquisition efficiency and reduce the thumbnail acquisition overhead, thereby improving the user experience.

[0106] Figure 6 This is a structural block diagram of a thumbnail display device shown in an embodiment of the present disclosure, such as... Figure 6 As shown, it includes: a display module 601, used to display a first track segment corresponding to the target video material on the video editing track, wherein the first track segment displays a thumbnail of the first video frame in the target video material; a generation module 602, used to generate a thumbnail refresh request in response to a target operation, wherein the target operation is used to trigger the display area of ​​the target video material on the video editing track to change from the first track segment to the second track segment, and the thumbnail refresh request is used to obtain a thumbnail of the second video frame in the target video material; based on the thumbnail refresh request, the target video material is decoded to generate a thumbnail of the second video frame; the display module 601 is also used to display the thumbnail of the second video frame in the second track segment presented on the video editing track.

[0107] In some embodiments of this disclosure, the target operation includes any one of the following: scaling down the video editing track, scaling up the video editing track, moving forward on the video editing track, and moving backward on the video editing track.

[0108] In some embodiments of this disclosure, the display module 601 is further configured to display a thumbnail of a third video frame in the target video material in the second track segment before generating a thumbnail of the second video frame by decoding the target video material based on the thumbnail refresh request. The thumbnail of the third video frame is a cached thumbnail whose timestamp is closest to the timestamp of the thumbnail of the second video frame before the target operation. Specifically, the display module 601 is configured to update the thumbnail of the third video frame displayed in the second track segment to the thumbnail of the second video frame.

[0109] In some embodiments of this disclosure, the generation module 602 is specifically configured to generate the thumbnail refresh request in response to the target operation via a first thread; send the thumbnail refresh request to a second thread via the first thread; and decode the target video material based on the thumbnail refresh request via the second thread to generate a thumbnail of the second video frame.

[0110] In some embodiments of this disclosure, the generation module 602 is specifically configured to add a target task to a task buffer queue based on the thumbnail refresh request via a second thread, the target task being used to indicate the acquisition of a thumbnail of a second video frame; extract the target task from the task buffer queue via the second thread and add the target task to a task processing queue; obtain the target task from the task processing queue via the second thread and decode the target video material to generate a thumbnail of the second video frame; and add the thumbnail of the second video frame to a queue to be rendered via the second thread. The display module 601 is specifically configured to obtain the thumbnail of the second video frame from the queue to be rendered via a first thread and display the thumbnail of the second video frame in a second track segment.

[0111] In some embodiments of this disclosure, the apparatus further includes a deletion module, configured to delete the target task from the task processing queue via a second thread after the thumbnail of the second video frame has been added to the queue to be rendered via a second thread.

[0112] In some embodiments of this disclosure, the generation module 602 is specifically configured to add the target task to the task buffer queue in response to the second thread not being full; and to delete the first task in the task buffer queue and add the target task to the task buffer queue in response to the second thread being full.

[0113] In some embodiments of this disclosure, the generation module 602 is specifically used to decode the target video material based on the thumbnail refresh request and in response to a thumbnail of a second video frame that does not exist in the cache, and generate a thumbnail of the second video frame; the device further includes: an acquisition module, used to acquire a cached thumbnail of the second video frame in response to a cached thumbnail of the second video frame.

[0114] In this embodiment of the disclosure, each module can implement the thumbnail display method provided in the above method embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0115] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. It is used to exemplarily illustrate an electronic device that implements any thumbnail display method in the embodiments of the present disclosure and should not be construed as a specific limitation on the embodiments of the present disclosure.

[0116] like Figure 7 As shown, the electronic device 700 may include a processor (e.g., a central processing unit, a graphics processor, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the electronic device 700. The processor 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0117] Typically, the following devices can be connected to I / O interface 705: input devices 706 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 707 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 708 including, for example, magnetic tapes, hard disks, etc.; and communication devices 709. Communication device 709 allows electronic device 700 to communicate wirelessly or wiredly with other devices to exchange data. Although an electronic device 700 with various devices is shown, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0118] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 709, or installed from storage device 708, or installed from ROM 702. When the computer program is executed by processor 701, it can perform the functions defined in any thumbnail display method provided in embodiments of this disclosure.

[0119] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0120] In some implementations, the client and server can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol), and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0121] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0122] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: display a first track segment corresponding to the target video material on a video editing track, wherein the first track segment displays a thumbnail of a first video frame in the target video material; generate a thumbnail refresh request in response to a target operation, wherein the target operation is used to trigger a change in the display area of ​​the target video material on the video editing track from the first track segment to a second track segment, and the thumbnail refresh request is used to obtain a thumbnail of a second video frame in the target video material; generate a thumbnail of the second video frame by decoding the target video material based on the thumbnail refresh request; and display the thumbnail of the second video frame in the second track segment displayed on the video editing track.

[0123] In embodiments of this disclosure, computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include, but are not limited to, object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on a computer, partially on a computer, as a standalone software package, partially on a computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0124] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0125] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0126] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0127] In the context of this disclosure, a computer-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of computer-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0128] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0129] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0130] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A method for displaying thumbnails, characterized in that, The method includes: The first track segment corresponding to the target video material is displayed on the video editing track, and a thumbnail of the first video frame in the target video material is presented in the first track segment; In response to a target operation, a thumbnail refresh request is generated. The target operation is used to trigger the display area of ​​the target video material on the video editing track to change from the first track segment to the second track segment. The thumbnail refresh request is used to obtain a thumbnail of the second video frame in the target video material. Based on the thumbnail refresh request, the target video material is decoded to generate a thumbnail of the second video frame; A thumbnail of the second video frame is displayed in the second track segment presented on the video editing track.

2. The method according to claim 1, characterized in that, The target operation includes any one of the following: scaling down the video editing track, scaling up the video editing track, moving forward on the video editing track, or moving backward on the video editing track.

3. The method according to claim 1, characterized in that, Before generating the thumbnail of the second video frame by decoding the target video material based on the thumbnail refresh request, the method further includes: The thumbnail of the third video frame in the target video material is displayed in the second track segment. The thumbnail of the third video frame is a cached thumbnail whose timestamp is closest to the timestamp of the thumbnail of the second video frame before the target operation. Displaying a thumbnail of the second video frame in the second track segment presented on the video editing track includes: Update the thumbnail of the third video frame shown in the second track segment to the thumbnail of the second video frame.

4. The method according to claim 1, characterized in that, The step of generating a thumbnail refresh request in response to a target operation includes: The thumbnail refresh request is generated by the first thread in response to the target operation; The thumbnail refresh request is sent to the second thread via the first thread; The step of generating a thumbnail of the second video frame by decoding the target video material based on the thumbnail refresh request includes: The second thread decodes the target video material based on the thumbnail refresh request to generate a thumbnail of the second video frame.

5. The method according to claim 4, characterized in that, The step of decoding the target video material based on the thumbnail refresh request using the second thread to generate a thumbnail of the second video frame includes: The second thread adds the target task to the task buffer queue based on the thumbnail refresh request, and the target task is used to indicate the acquisition of the thumbnail of the second video frame; The second thread retrieves the target task from the task cache queue and adds the target task to the task processing queue. The second thread retrieves the target task from the task processing queue, decodes the target video material, and generates a thumbnail of the second video frame. The second thread adds the thumbnail of the second video frame to the queue to be rendered. Displaying a thumbnail of the second video frame in the second track segment presented on the video editing track includes: The first thread retrieves a thumbnail of the second video frame from the queue to be rendered, and displays the thumbnail of the second video frame in the second track segment.

6. The method according to claim 5, characterized in that, After adding the thumbnail of the second video frame to the rendering queue via the second thread, the method further includes: The target task is deleted from the task processing queue by the second thread.

7. The method according to claim 5, characterized in that, The step of adding the target task to the task buffer queue through the second thread based on the thumbnail refresh request includes: In response to the task buffer queue not being full, the target task is added to the task buffer queue via a second thread; In response to the task cache queue being full, the first task in the task cache queue is deleted by a second thread, and the target task is added to the task buffer queue.

8. The method according to any one of claims 1 to 7, characterized in that, The step of generating a thumbnail of the second video frame by decoding the target video material based on the thumbnail refresh request includes: Based on the thumbnail refresh request, in response to the thumbnail of the second video frame that does not have a cache, the target video material is decoded to generate a thumbnail of the second video frame. The method further includes: In response to the existence of a cached thumbnail of the second video frame, the cached thumbnail of the second video frame is obtained.

9. A thumbnail display device, characterized in that, include: The display module is used to display the first track segment corresponding to the target video material on the video editing track, and the first track segment presents a thumbnail of the first video frame in the target video material; A generation module is used to generate a thumbnail refresh request in response to a target operation. The target operation is used to trigger the display area of ​​the target video material on the video editing track to change from the first track segment to the second track segment. The thumbnail refresh request is used to obtain a thumbnail of the second video frame in the target video material. Based on the thumbnail refresh request, the target video material is decoded to generate a thumbnail of the second video frame; The display module is also used to display a thumbnail of the second video frame in the second track segment presented on the video editing track.

10. An electronic device, characterized in that, include: A memory and a processor, wherein the memory is used to store a computer program; and the processor is used to execute the thumbnail display method according to any one of claims 1 to 8 when the computer program is invoked.

11. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the thumbnail display method as described in any one of claims 1 to 8.

12. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the thumbnail display method according to any one of claims 1 to 8.

Images