Video playback method, device, electronic device and storage medium

Abstract

The embodiments of the present application provide a video playback method, device, electronic device, and storage medium. During the video playback process, the electronic device can determine the playback video frame from the playback video according to the dragging speed in response to the dragging operation issued by the user on the playback timeline; following the dragging operation, the determined playback video frame is played. Since the time interval between adjacent video frames in the playback video frame or the number of playback video frames will change with the dragging speed, the determined playback video frame will also change when the user drags the timeline at different speeds. The user can slow down the dragging speed in the area of ​​interest to make the time interval shorter or the number of playback video frames smaller, and the user can carefully view the playback video content; the user can speed up the dragging speed in the area of ​​no interest to make the time interval longer or the number of playback video frames larger, and the content of no interest can be quickly skipped. Therefore, the retrieval efficiency of video playback can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of video playback, and in particular to a video playback method, device, electronic device and storage medium. Background Art

[0002] Video playback is a method of re-watching a recorded video to find the target scene that the user wants to view. It has a wide range of application scenarios. For example, in a video recording scenario, if an item is stolen or a safety accident occurs, the video recorded by the camera can be replayed to determine the course of the incident. For another example, in an online class scenario, if a student does not keep up with a certain knowledge point in a live course, the recorded video of the course can be replayed.

[0003] At present, the video playback mode in the related art is speed playback, that is, playing the video at a speed different from the normal speed, for example, 2x speed, 4x speed, 8x speed, etc. At present, the maximum speed supported by mobile terminals is less than 64x speed, and the retrieval efficiency is low. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a video playback method, device, electronic device and storage medium to improve the retrieval efficiency of video playback. The specific technical solution is as follows:

[0005] In a first aspect, an embodiment of the present application provides a video playback method, the method comprising:

[0006] During video playback, in response to a user's dragging operation on a playback timeline of a currently played playback video, a playback video frame is determined from the playback video according to a dragging speed of the dragging operation, wherein the faster the dragging speed is, the longer the time interval between two adjacent video frames in the playback video frame is, or the time interval between two adjacent video frames in the playback video frame is equal, and the faster the dragging speed is, the more the number of the playback video frames is;

[0007] Following the dragging operation, the determined playback video frame is played.

[0008] Optionally, during the video playback process, in response to a drag operation issued by a user on the playback timeline of the currently playing playback video, before the step of determining a playback video frame from the playback video according to the drag speed of the drag operation, the method further includes:

[0009] Determine the current condensed video duration and the current frame extraction interval corresponding to the current time axis zoom level according to the correspondence between the preset time axis zoom level and the condensed video duration and the frame extraction interval;

[0010] Divide the total time period corresponding to the original video into multiple sub-time periods, wherein the duration of each sub-time period is the duration of the current condensed video;

[0011] The original video corresponding to each sub-time period is downloaded, and during the downloading process, the original video corresponding to each sub-time period is frame extracted according to the current frame extraction interval to obtain a playback video.

[0012] Optionally, the step of determining a playback video frame from the playback video according to the dragging speed of the dragging operation includes:

[0013] Determine the time interval between the timestamp corresponding to the drag operation of the current callback on the playback timeline and the timestamp of the last playback video frame; if the time interval is not less than the full frame rate interval, determine from the playback video the video frame corresponding to the timestamp of the current callback as the current playback video frame; or,

[0014] Determine, from the playback video, a video frame corresponding to a timestamp included in the playback timeline portion through which the drag operation passes; extract the determined video frame to obtain a playback video frame, so that in the playback video frame, a time interval between adjacent video frames is not less than a full frame rate interval;

[0015] The full frame rate interval is the time interval between adjacent video frames when playing the video at full frame rate.

[0016] Optionally, the step of determining the time interval between the timestamp corresponding to the drag operation of the current callback on the playback timeline and the timestamp of the last playback video frame; if the time interval is not less than the full frame rate interval, determining from the playback video that the video frame corresponding to the timestamp of the current callback is the current playback video frame, includes:

[0017] Obtaining, from a pre-established serial play queue, a timestamp corresponding to the drag operation of the current callback on the playback timeline, wherein the serial play queue is used to record the timestamp corresponding to the drag operation of the callback on the playback timeline;

[0018] Determine the time interval between the acquired timestamp and the timestamp of the last playback video frame;

[0019] If the time interval is not less than the full frame rate interval, based on the timestamps of each playback video frame included in the cached frame analysis results of the playback video frame, a video frame corresponding to the acquired timestamp is determined from the playback video as the current playback video frame.

[0020] Optionally, the step of downloading the original video corresponding to each sub-time period, and extracting frames of the original video corresponding to each sub-time period according to the current frame extraction interval during the downloading process to obtain the playback video includes:

[0021] Constructing a download task queue of the condensed videos corresponding to the multiple sub-time periods, wherein the download task queue includes a download task of the condensed videos corresponding to each sub-time period, and the order of the download tasks is determined based on the time interval between the sub-time period corresponding to the download task and the current time;

[0022] The download tasks in the download task queue are executed, and in the process of executing each download task, the original video corresponding to each sub-time period of the download is frame-drawn according to the current frame-drawing interval to obtain the condensed video corresponding to each sub-time period.

[0023] Optionally, the method further includes:

[0024] In the process of downloading the original video corresponding to each sub-time period, in response to a zoom level adjustment instruction issued by the user based on the playback time axis, determining whether the frame extraction intervals corresponding to the time axis zoom levels before and after the adjustment are the same according to the corresponding relationship;

[0025] If the frame extraction intervals corresponding to the timeline zoom levels before and after adjustment are different, the adjusted timeline zoom level is used as the current timeline zoom level, and the step of determining the current condensed video length and the current frame extraction interval corresponding to the current timeline zoom level according to the correspondence between the preset timeline zoom level, the condensed video length and the frame extraction interval is returned.

[0026] Optionally, the method further includes:

[0027] In the process of downloading the original video corresponding to each sub-time period, in response to a drag operation issued by the user based on the playback timeline, in the download task queue, the download task corresponding to the sub-time period containing the timestamp corresponding to the current position of the drag operation on the playback timeline is adjusted to be the first task except the download task being executed;

[0028] Based on the time interval between the sub-time period corresponding to the download task and the timestamp corresponding to the current position of the dragging operation on the playback timeline, the remaining download tasks are adjusted.

[0029] Optionally, the method further includes:

[0030] When it is detected that the timestamp corresponding to the current position of the drag operation on the playback timeline is close to the timestamp corresponding to the last video frame of the playback video corresponding to the current time period, pre-load the condensed video corresponding to the next time period;

[0031] The current time period is a sub-time period including a current timestamp of the playback time axis, and the next time period is a next sub-time period of the current time period in the dragging direction of the dragging operation.

[0032] Optionally, before the step of determining a playback video frame from the playback video according to the dragging speed of the dragging operation, the method further includes:

[0033] Determine whether there is a frame analysis result of the playback video corresponding to the current time period, wherein the current time period is a sub-time period including a timestamp corresponding to the current position of the playback time axis;

[0034] If there is no frame analysis result of the playback video corresponding to the current time period, determining whether the playback video corresponding to the current time period has been downloaded;

[0035] If the playback video corresponding to the current time period has not been downloaded, download the condensed video corresponding to the current time period, and perform frame analysis on the downloaded condensed video to obtain a frame analysis result;

[0036] If a frame analysis result corresponding to the current time period already exists, the step of determining a playback video frame from the playback video according to the dragging speed of the dragging operation is performed.

[0037] In a second aspect, an embodiment of the present application provides a video playback device, the device comprising:

[0038] a playback video frame determination module, used for, in the process of video playback, in response to a drag operation issued by a user on the playback time axis of the currently played playback video, determining the playback video frame from the playback video according to the drag speed of the drag operation, wherein the faster the drag speed is, the longer the time interval between two adjacent video frames in the playback video frame is, or the time interval between two adjacent video frames in the playback video frame is equal, and the faster the drag speed is, the more the number of the playback video frames is;

[0039] The playback video frame playing module is used to play the determined playback video frame following the dragging operation.

[0040] Optionally, the device further comprises:

[0041] A parameter determination module, used to determine the current condensed video duration and the current frame extraction interval corresponding to the current time axis zoom level according to the correspondence between the preset time axis zoom level and the condensed video duration and the frame extraction interval;

[0042] A time segmentation module, used to segment the total time segment corresponding to the original video into multiple sub-time segments, wherein the duration of each sub-time segment is the duration of the current condensed video;

[0043] The video download module is used to download the original video corresponding to each sub-time period, and during the downloading process, extract frames of the original video corresponding to each sub-time period according to the current frame extraction interval to obtain a playback video.

[0044] Optionally, the playback video frame determination module includes:

[0045] The first playback video frame determination submodule is used to determine the time interval between the timestamp corresponding to the drag operation of the current callback on the playback timeline and the timestamp of the previous playback video frame; if the time interval is not less than the full frame rate interval, determine from the playback video the video frame corresponding to the timestamp of the current callback as the current playback video frame;

[0046] The second playback video frame determination submodule is used to determine, from the playback video, a video frame corresponding to a timestamp included in the playback timeline portion through which the drag operation passes; extract frames from the determined video frame to obtain a playback video frame, so that the time interval between adjacent video frames in the playback video frame is not less than the full frame rate interval;

[0047] The full frame rate interval is the time interval between adjacent video frames when playing the video at full frame rate.

[0048] Optionally, the first playback video frame determination submodule includes:

[0049] A timestamp acquisition unit, used to acquire the timestamp corresponding to the drag operation of the current callback on the playback timeline from a pre-established serial playback queue, wherein the serial playback queue is used to record the timestamp corresponding to the drag operation of the callback on the playback timeline;

[0050] a time interval determination unit, used to determine the time interval between the acquired timestamp and the timestamp of the last playback video frame;

[0051] The current playback video frame determination unit is used to determine the video frame corresponding to the acquired timestamp from the playback video as the current playback video frame based on the timestamps of each playback video frame included in the cached frame analysis results of the playback video frames if the time interval is not less than the full frame rate interval.

[0052] Optionally, the video download module includes:

[0053] A download task queue construction submodule, used to construct a download task queue of condensed videos corresponding to the multiple sub-time periods, wherein the download task queue includes download tasks of condensed videos corresponding to each sub-time period, and the order of the download tasks is determined based on the time interval between the sub-time period corresponding to the download task and the current time;

[0054] The download task execution submodule is used to execute the download tasks in the download task queue, and in the process of executing each download task, the original video corresponding to each sub-time period of the download is framed according to the current frame extraction interval to obtain the condensed video corresponding to each sub-time period.

[0055] Optionally, the device further comprises:

[0056] A frame extraction interval determination module is used to determine whether the frame extraction intervals corresponding to the time axis zoom levels before and after the adjustment are the same according to the corresponding relationship in response to a zoom level adjustment instruction issued by a user based on the playback time axis during the process of downloading the original video corresponding to each sub-time period;

[0057] A return module is used to, if the frame extraction intervals corresponding to the timeline zoom levels before and after adjustment are different, use the adjusted timeline zoom level as the current timeline zoom level, and return the step of determining the current condensed video duration and the current frame extraction interval corresponding to the current timeline zoom level based on the correspondence between the preset timeline zoom level, the condensed video duration and the frame extraction interval.

[0058] Optionally, the device further comprises:

[0059] A first order determination module is used for, in the process of downloading the original video corresponding to each sub-time period, in response to a drag operation issued by a user based on a playback timeline, adjusting, in the download task queue, a download task corresponding to a sub-time period containing a timestamp corresponding to a current position of the drag operation on the playback timeline to be the first task other than the download task being executed;

[0060] The second sequence determination module is used to adjust the remaining download tasks based on the time interval between the sub-time period corresponding to the download task and the timestamp corresponding to the current position of the dragging operation on the playback timeline.

[0061] Optionally, the device further comprises:

[0062] A condensed video preloading module, configured to preload a condensed video corresponding to a next time period when it is detected that a timestamp corresponding to a current position of the dragging operation on the playback timeline is close to a timestamp corresponding to a last video frame of the playback video corresponding to the current time period;

[0063] The current time period is a sub-time period including a current timestamp of the playback time axis, and the next time period is a next sub-time period of the current time period in the dragging direction of the dragging operation.

[0064] Optionally, the device further comprises:

[0065] A frame analysis result determination module, used to determine whether there is a frame analysis result of the playback video corresponding to the current time period, wherein the current time period is a sub-time period including a timestamp corresponding to the current position of the playback time axis;

[0066] A video download determination module, used to determine whether the playback video corresponding to the current time period has been downloaded if there is no frame analysis result of the playback video corresponding to the current time period;

[0067] A frame analysis module, configured to download a condensed video corresponding to the current time period if the playback video corresponding to the current time period has not been downloaded, and perform frame analysis on the downloaded condensed video to obtain a frame analysis result;

[0068] An execution module is used to execute the step of determining a playback video frame from the playback video according to the dragging speed of the dragging operation if a frame analysis result corresponding to the current time period already exists.

[0069] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0070] Memory, used to store computer programs;

[0071] The processor is used to implement any method described in the first aspect when executing a program stored in the memory.

[0072] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any method described in the first aspect is implemented.

[0073] Beneficial effects of the embodiments of the present application:

[0074] In the solution provided by the embodiment of the present application, the electronic device can, during the video playback process, respond to the user's drag operation on the playback timeline of the currently played playback video, determine the playback video frame from the playback video according to the drag speed of the drag operation, wherein the faster the drag speed, the longer the time interval between two adjacent video frames in the playback video frame, or, if the time interval between two adjacent video frames in the playback video frame is equal, the faster the drag speed, the more the number of playback video frames; following the drag operation, play the determined playback video frame. Since the time interval between two adjacent video frames in the playback video frame or the number of playback video frames will change with the change of the drag speed, the playback video frame determined from the playback video will also change when the user drags the timeline at different speeds. Specifically, the user can slow down the dragging speed in the places where he is interested. In this case, the time interval between two adjacent video frames in the playback video frame is shorter, or the time interval between two adjacent video frames is equal but the number of playback video frames played is small, and the user can carefully check the content of the playback video; the user can speed up the dragging speed in the places where he is not interested. In this case, the time interval between two adjacent video frames in the playback video frame is longer, or the time interval between two adjacent video frames is equal but the number of playback video frames played is large, and the content that the user is not interested in can be quickly skipped. Therefore, the speed at which the user drags the playback timeline can determine the speed of change of the playback video playback screen, and can improve the retrieval efficiency of video playback. Of course, implementing any product or method of the present application does not necessarily require all of the advantages described above to be achieved at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0076] Figure 1 A flowchart of a video playback method provided in an embodiment of the present application;

[0077] Figure 2(a) is based on Figure 1 A first schematic diagram of a playback video playing interface of the illustrated embodiment;

[0078] Figure 2(b) is based on Figure 1 A second schematic diagram of the playback video playing interface of the illustrated embodiment;

[0079] Figure 2(c) is based on Figure 1 A third schematic diagram of the playback video playing interface of the illustrated embodiment;

[0080] Figure 3 Based on Figure 1 A flow chart of a playback video download method of the illustrated embodiment;

[0081] Figure 4 Based on Figure 1 A flow chart of a first method of determining a playback video frame in the illustrated embodiment;

[0082] Figure 5 Based on Figure 1 A flow chart of a second method for determining playback video frames according to the illustrated embodiment;

[0083] Figure 6 for Figure 4 A specific flow chart of the first method for determining playback video frames in the illustrated embodiment;

[0084] Figure 7 for Figure 1 A specific flow chart of step S101 in the illustrated embodiment;

[0085] Figure 8 for Figure 7 A specific flow chart of step S703 in the illustrated embodiment;

[0086] Fig. 9 for Figure 3 A specific flow chart of step S303 in the illustrated embodiment;

[0087] Fig.10 Based on Figure 1 A schematic diagram of a download task queue of the illustrated embodiment;

[0088] Fig.11 Based on Figure 1 A flow chart of re-determining the download task queue of the illustrated embodiment;

[0089] Figure 12 (a) is based on Figure 1 A first schematic diagram of adjusting the timeline zoom level in the illustrated embodiment;

[0090] Figure 12(b) is based on Figure 1 A second schematic diagram of adjusting the timeline zoom level in the illustrated embodiment;

[0091] Figure 12(c) is based on Figure 1 A third schematic diagram of adjusting the timeline zoom level in the illustrated embodiment;

[0092] Fig.13 Based on Figure 1 A schematic diagram of the download task queue changes in the illustrated embodiment;

[0093] Fig.14 Based on Figure 1 A flow chart of a download task sequence adjustment method of the illustrated embodiment;

[0094] Fig.15 Based on Figure 1 Another schematic diagram of the download task queue change in the illustrated embodiment;

[0095] Fig.16 Based on Figure 1 A flow chart of downloading and frame analysis of playback video in the illustrated embodiment;

[0096] Fig.17 A schematic diagram of the structure of a video playback device provided in an embodiment of the present application;

[0097] Fig.18 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0098] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field based on the present application belong to the scope of protection of the present application.

[0099] In order to improve the retrieval efficiency of video playback, the embodiments of the present application provide a video playback method, device, electronic device, computer-readable storage medium and computer program product. The following first introduces a video playback method provided by the embodiments of the present application.

[0100] The video playback method provided in the embodiment of the present application can be applied to any electronic device that needs video playback, for example, a mobile phone, a tablet computer, a laptop computer, etc., which is not specifically limited here. For the sake of clarity, it is referred to as an electronic device in the following text.

[0101] like Figure 1 As shown, a video playback method, the method comprising:

[0102] S101, during video playback, in response to a drag operation issued by a user on a playback timeline of a currently playing playback video, determining a playback video frame from the playback video according to a drag speed of the drag operation;

[0103] Among them, the faster the dragging speed is, the longer the time interval between two adjacent video frames in the playback video frames is, or the time interval between two adjacent video frames in the playback video frames is equal, and the faster the dragging speed is, the more the number of playback video frames is.

[0104] S102, following the dragging operation, playing the determined playback video frame.

[0105] It can be seen that in the embodiment of the present application, the electronic device can, during the video playback process, respond to the user's drag operation on the playback timeline of the currently playing playback video, determine the playback video frame from the playback video according to the drag speed of the drag operation, wherein the faster the drag speed, the longer the time interval between two adjacent video frames in the playback video frame, or, the time interval between two adjacent video frames in the playback video frame is equal, the faster the drag speed, the more the number of playback video frames; following the drag operation, play the determined playback video frame. Since the time interval between two adjacent video frames in the playback video frame or the number of playback video frames will change with the change of the drag speed, the playback video frame determined from the playback video will also change when the user drags the timeline at different speeds. Specifically, the user can slow down the dragging speed at the places of interest. In this case, the time interval between two adjacent video frames in the playback video frame is shorter, or the time interval between two adjacent video frames is equal but the number of playback video frames played is small, and the user can carefully check the content of the playback video; the user can speed up the dragging speed at the places of no interest. In this case, the time interval between two adjacent video frames in the playback video frame is longer, or the time interval between two adjacent video frames is equal but the number of playback video frames played is large, and the content that the user is not interested in can be quickly skipped. Therefore, the speed at which the user drags the playback timeline can determine the speed of change of the playback video playback screen, and can improve the retrieval efficiency of video playback.

[0106] When a video needs to be played back, the user can trigger the electronic device to display a playback video playback interface, which may include a playback area for the playback video and a playback timeline. The playback video may be an original video recorded by a camera without frame extraction, an original video downloaded from a server without frame extraction, or a condensed video obtained by performing frame extraction on the original video, which is not specifically limited here.

[0107] Each scale line of the playback time axis corresponds to a video frame in the playback video, each scale line has the same timestamp as the corresponding video frame, and the scale value of the playback time axis is the time interval between adjacent video frames in the playback video. For example, assuming that the total duration of the playback video is 5 minutes and the time interval between adjacent video frames in the playback video is 10 seconds, then the playback video includes video frames 1 to 30, and the playback time axis corresponding to the playback video includes scale lines 1 to 30, scale lines 1 to 30 correspond to video frames 1 to 30 respectively, and scale lines 1 to 30 have the same timestamp as video frames 1 to 30. Taking scale line 5 and video frame 5 as an example, the timestamp corresponding to video frame 5 is 00:00:50, and the timestamp corresponding to scale line 5 is also 00:00:50.

[0108] In one embodiment, when the dragging operation passes through a certain scale line on the playback timeline, the timestamp corresponding to the scale line can be displayed. For example, the schematic diagram of the playback video playback interface can be shown in Figure 2 (a)-Figure 2 (c). In Figure 2 (a), the playback video playback interface includes a playback area 201 for the playback video and a playback progress bar 202. The time interval between adjacent video frames in the playback video is 1 minute, and the playback video includes video frames 1-30. Correspondingly, the playback progress bar includes scale lines 1-30, and scale lines 1-30 correspond to video frames 1-30 respectively, and the time difference between the timestamps corresponding to adjacent scale lines is also 1 minute.

[0109] In FIG2 (a), the user's finger is at scale line 13, and the playback area 201 of the playback video displays video frame 13, and the timestamp corresponding to scale line 13 is displayed above scale line 13, that is, 00:13:00. Next, the user drags the finger from scale line 13 to scale line 14, and the playback video playback interface will change to FIG2 (b). In FIG2 (b), the playback area of ​​the playback video displays video frame 14, and the timestamp corresponding to scale line 14 is displayed above scale line 14, that is, 00:14:00. Then, the user continues to drag the playback timeline from scale line 14 to scale line 15, and the playback video playback interface will change to FIG2 (c). In FIG2 (c), the playback area of ​​the playback video displays video frame 15, and the timestamp corresponding to scale line 15 is displayed above scale line 15, that is, 00:15:00.

[0110] During video playback, if the user wants to view video frames at other time points of the playback video, a drag operation can be issued on the playback timeline of the currently playing playback video. For example, the user can drag the playback timeline displayed on the touch screen of the electronic device with a finger to issue a drag operation.

[0111] When the user issues a drag operation on the playback timeline of the currently played back video, the electronic device can respond to the drag operation issued by the user and determine the playback video frame from all video frames included in the playback video according to the drag speed of the drag operation, that is, execute step S101.

[0112] According to the drag speed of the drag operation, the method of determining the playback video frame can be one of the following two methods:

[0113] In the first mode, the faster the dragging speed of the user, the longer the time interval between two adjacent video frames in the playback video frame can be, and correspondingly, the slower the dragging speed of the user, the shorter the time interval between two adjacent video frames in the playback video frame can be. Specifically, if the dragging speed of the dragging operation issued by the user is faster, it means that the user is not very interested in the playback video frame currently dragged. In this case, the time interval between two adjacent video frames in the playback video frame can be made longer, so that the playback video is played quickly, thereby quickly skipping the part that the user is not interested in. If the dragging speed of the dragging operation issued by the user is slower, it means that the user is more interested in the playback video frame currently dragged. In this case, the time interval between two adjacent video frames in the playback video frame can be made shorter, so that the playback video is played slowly.

[0114] In the second mode, when the time interval between two adjacent video frames in the playback video frame is equal, the faster the dragging speed, the more the number of playback video frames, and correspondingly, the slower the dragging speed, the fewer the number of playback video frames. Specifically, if the dragging speed of the dragging operation issued by the user is fast, then the playback time axis portion passed by the dragging operation per unit time is longer, and the number of video frames corresponding to the timestamps included in the playback time axis portion is larger, then a larger number of playback video frames can be determined from the playback video. Correspondingly, if the dragging speed of the dragging operation issued by the user is slow, then the playback time axis portion passed by the dragging operation per unit time is shorter, and the number of video frames corresponding to the timestamps included in the playback time axis portion is smaller, then a smaller number of playback video frames can be determined from the playback video.

[0115] When the user is dragging the timeline, the dragging speed usually changes in real time. Accordingly, the electronic device will determine the time interval between two adjacent video frames in the playback video frame in real time according to the user's real-time dragging speed, and then determine the playback video frame from the playback video according to the time interval determined in real time, and the OSD (on-screen display) time in the playback video playback interface will also change in real time. Among them, the OSD time can be the timestamp corresponding to the current position of the playback timeline. In other words, assuming that the user's dragging speed is sometimes fast and sometimes slow, then correspondingly, the time interval of the playback video frame determined by the electronic device will also be sometimes small and sometimes large, and the change of the OSD time in the playback video playback interface will also be sometimes fast and sometimes slow.

[0116] Since the playback video frame is the video frame to be displayed by the electronic device when the user drags the playback timeline, after determining the playback video frame, the electronic device can follow the user's dragging operation and play the determined playback video frame, that is, execute step S102.

[0117] For example, corresponding to the first method described above, assuming that the user starts to drag the playback timeline at a drag speed of 0.5 cm / s, the electronic device can determine the playback video frame from the playback video at a time interval of 0.04 s, and when the user drags the playback timeline at a drag speed of 0.5 cm / s, the playback video frame determined at a time interval of 0.04 s is played. Next, assuming that the user's drag speed becomes 2 cm / s, the electronic device can determine the playback video frame from the playback video at a time interval of 0.2 s, and when the user drags the playback timeline at a drag speed of 2 cm / s, the playback video frame determined at a time interval of 0.2 s is played.

[0118] For another example, corresponding to the second method described above, assuming that the time interval between adjacent video frames in the playback video is 0.2s, and when the user starts to drag the playback timeline, the dragging speed is 0.6cm / s, then the electronic device can determine 10 playback video frames from the playback video, and when the user drags the playback timeline at a dragging speed of 0.6cm / s, play the determined 10 playback video frames, and the time interval between these 10 playback video frames is 0.2s. Next, assuming that the user's dragging speed becomes 1.5cm / s, then the electronic device can determine 30 playback video frames from the playback video, and when the user drags the playback timeline at a dragging speed of 1.5cm / s, play the determined 30 playback video frames, and the time interval between these 30 playback video frames is also 0.2s.

[0119] It can be seen that in the embodiment of the present application, since the time interval between two adjacent video frames in the playback video frame or the number of playback video frames will change with the change of the drag speed, the drag speed of the user dragging the time axis is different, and the playback video frame determined from the playback video will also change. Specifically, the user can slow down the drag speed in the place of interest. In this case, the time interval between two adjacent video frames in the playback video frame is short, or the time interval between two adjacent video frames is equal but the number of playback video frames played is small, and the user can carefully check the content of the playback video; the user can speed up the drag speed in the place of no interest. In this case, the time interval between two adjacent video frames in the playback video frame is long, or the time interval between two adjacent video frames is equal but the number of playback video frames played is large, and the content that the user is not interested in can be quickly skipped. Therefore, the speed of the user dragging the playback time axis can determine the speed of change of the playback video playback screen, and can improve the retrieval efficiency of video playback.

[0120] As an implementation method of the present application, Figure 3 As shown, in the above-mentioned process of video playback, in response to a drag operation issued by a user on the playback timeline of the currently played playback video, before the step of determining a playback video frame from the playback video according to the drag speed of the drag operation, the above-mentioned method may further include:

[0121] S301, determining the current condensed video duration and the current frame extraction interval corresponding to the current time axis zoom level according to the correspondence between the preset time axis zoom level and the condensed video duration and the frame extraction interval;

[0122] Since the contents of multiple adjacent video frames in the original video are usually similar, in order to improve the viewing efficiency of users and save download resources and storage resources, the original video can be extracted to obtain the playback video. Users usually have different needs at different stages after the playback video starts playing. Specifically:

[0123] When starting to play back a video, the user usually needs to identify the part of interest from a large range of the playback video, so the original video can be framed at a larger frame sampling interval to help the user efficiently locate the part of interest. After the user has located the part of interest, the user usually needs to find the video frame of interest in a small range of the playback video, so the original video can be framed at a smaller frame sampling interval to help the user find the video frame he wants to view.

[0124] In order to optimize download resources, the total time period corresponding to the original video can be divided into multiple different sub-time periods, and download tasks corresponding to each sub-time period can be created respectively. When the frame extraction interval is large, the number of video frames obtained after frame extraction is small, so the duration of each sub-time period can be set to be large. Conversely, when the frame extraction interval is small, the number of video frames obtained after frame extraction is large, so the duration of each sub-time period can be set to be small.

[0125] In summary, in order to meet the needs of users at different stages of playing and replaying videos and to optimize download resources, the correspondence between the timeline zoom level and the condensed video duration and the frame extraction interval can be preset. The condensed video duration is the duration of each of the above-mentioned sub-time periods, the condensed video is obtained by extracting frames from the original video, and the replay video includes the condensed video corresponding to each sub-time period. The electronic device can determine the current condensed video duration and the current frame extraction interval corresponding to the current timeline zoom level according to the correspondence between the preset timeline zoom level and the condensed video duration and the frame extraction interval.

[0126] For example, the above correspondence may be shown in Table 1:

[0127] Table 1

[0128]

[0129] Assuming that the user wants to find suspicious objects by replaying the video, the user can set the timeline zoom level to 1 hour when the replay video just starts playing. In this way, it can be determined that the current condensed video duration and the current frame extraction interval corresponding to the current timeline zoom level are 6 hours and 60 seconds respectively. In this case, since the frame extraction interval of the original video is 60 seconds, the user can roughly find the part with suspicious objects from a large range of the replay video.

[0130] After the user locates the part where the suspicious object exists, the timeline zoom level can be adjusted to 5 minutes. In this way, it can be determined that the current condensed video duration and the current frame extraction interval corresponding to the current timeline zoom level are 3 hours and 10 seconds respectively. In this case, for the original video with a frame extraction interval of 10 seconds, the user can accurately find the video frame containing the suspicious object in a small range.

[0131] S302, dividing the total time period corresponding to the original video into multiple sub-time periods;

[0132] After determining the duration of the current condensed video, the total time period corresponding to the original video can be divided into multiple sub-time periods. For example, assuming that the total time period corresponding to the original video is 24 hours and the current condensed video duration is 6 hours, the total time period can be divided into 4 sub-time periods of 6 hours each. For another example, assuming that the total time period corresponding to the original video is 18 hours and the current condensed video duration is 3 hours, the total time period can be divided into 6 sub-time periods of 3 hours each.

[0133] S303, downloading the original video corresponding to each sub-time period, and extracting frames of the original video corresponding to each sub-time period according to the current frame extraction interval during the downloading process to obtain a playback video.

[0134] The electronic device can obtain the playback video by using one of the following two implementation modes:

[0135] In the first implementation, since the playback video is obtained by extracting frames from the original video, the electronic device can download the original video corresponding to each sub-time period, and extract frames from the original video corresponding to each sub-time period according to the current frame extraction interval during the download process, thereby obtaining the playback video. The electronic device can silently start downloading the original video in the background when starting to play the playback video, and can also download the original video in advance before playing the playback video, which is not specifically limited here.

[0136] For example, assuming that the total time period corresponding to the original video is divided into sub-time period 1 to sub-time period 3, and the current frame extraction interval is 60 seconds, then the original videos corresponding to sub-time period 1 to sub-time period 3 can be downloaded in sequence, and during the download process, the original videos can be extracted at a frame extraction interval of 60 seconds to obtain the playback video.

[0137] In the second implementation, the electronic device may send a play request for the time point corresponding to the drag operation on the playback timeline to the streaming media device, wherein the streaming media device stores the playback video in advance, and the streaming media device may specifically be another mobile terminal, a camera, etc. Next, the streaming media device may return the playback video corresponding to the time point to the electronic device, and the electronic device may obtain the playback video.

[0138] It can be seen that in the embodiment of the present application, the electronic device can determine the current condensed video duration and the current frame extraction interval corresponding to the current time axis zoom level according to the correspondence between the preset time axis zoom level and the condensed video duration and the frame extraction interval; divide the total time period corresponding to the original video into multiple sub-time periods, wherein the duration of each sub-time period is the current condensed video duration; download the original video corresponding to each sub-time period, and extract the original video corresponding to each sub-time period according to the current frame extraction interval during the download process to obtain the playback video. Since the user can adjust the time axis zoom level according to their own needs at different stages of playing the playback video, the efficiency of the user finding the key frames of interest through the playback video can be improved. Since the total time period corresponding to the original video can be divided into multiple sub-time periods according to the current condensed video duration, and then the original video corresponding to each sub-time period is downloaded separately, and the original video can be extracted during the download process, the playback video of the corresponding specification can be downloaded according to the time axis zoom level, thereby improving the download efficiency.

[0139] As an implementation manner of the embodiment of the present application, the step of determining the playback video frame from the playback video according to the dragging speed of the dragging operation may include one of the following two methods:

[0140] Determine the time interval between the timestamp corresponding to the drag operation of the current callback on the playback timeline and the timestamp of the last playback video frame; if the time interval is not less than the full frame rate interval, determine from the playback video the video frame corresponding to the timestamp of the current callback as the current playback video frame; or, from the playback video, determine the video frame corresponding to the timestamp included in the portion of the playback timeline through which the drag operation passes; extract the determined video frames to obtain playback video frames, so that in the playback video frames, the time interval between adjacent video frames is not less than the full frame rate interval.

[0141] The flowchart of the first method for determining the playback video frame can be as follows: Figure 4 As shown, the following steps may be specifically included:

[0142] S401, determining the time interval between the timestamp corresponding to the drag operation of the current callback on the playback timeline and the timestamp of the last playback video frame; if the time interval is not less than the full frame rate interval, executing step S402; if the time interval is less than the full frame rate interval, executing step S403;

[0143] When the user's dragging operation passes through the playback timeline, the electronic device will receive the playback timeline callback's timestamp corresponding to the playback timeline that the dragging operation passes through. For example, assuming that the timestamps corresponding to scale lines 1-3 are 00:00:05, 00:00:10, and 00:00:15, respectively, then as the dragging operation passes through scale lines 1-3 in sequence, the electronic device will receive the playback timeline callback's timestamps 00:00:05, 00:00:10, and 00:00:15 in sequence. Among them, the time interval between two adjacent callbacks is relatively short, specifically 5 microseconds, 10 microseconds, 15 microseconds, etc., which is not specifically limited here.

[0144] Electronic devices have a full frame rate limitation, which is the maximum frame rate that an electronic device can achieve when playing a video normally. If the user's dragging speed is too fast, then within a unit of time, the number of video frames corresponding to the timestamps included in the playback timeline portion where the dragging operation passes will also be large. In this case, if the electronic device follows the dragging operation and plays all the video frames corresponding to the timestamps included in the playback timeline portion where the dragging operation passes, it is likely that the electronic device will not be able to meet the full frame rate limitation, causing playback to freeze.

[0145] In order to avoid playback freezes, the electronic device needs to control the time interval between the determined timestamp of the playback video frame and the timestamp of the adjacent playback video frame to be no less than the full frame rate interval, wherein the full frame rate interval is the time interval between adjacent video frames when playing the condensed video at full frame rate.

[0146] Specifically, each time the electronic device receives a callback drag operation corresponding to a timestamp on the playback timeline, it can determine the time interval between the timestamp of the current callback and the timestamp of the last playback video frame. In this way, it can be determined whether to use the video frame as the playback video frame for the video frame corresponding to the timestamp of the current callback. When determining the playback video frame for the first time, the first playback video frame can be the video frame corresponding to the first timestamp included in the playback timeline through which the user's drag operation passes.

[0147] S402, determining from the playback video that the video frame corresponding to the timestamp of the current callback is the current playback video frame;

[0148] If the time interval is not less than the full frame rate interval, it means that the time interval between the timestamp of the current callback and the timestamp of the last playback video frame is long enough to meet the full frame rate limit of the electronic device, so the video frame corresponding to the timestamp of the current callback can be determined from the playback video as the current playback video frame.

[0149] S403: Do not determine the video frame corresponding to the timestamp of the current callback as the current playback video frame.

[0150] If the time interval is not less than the full frame rate interval, it means that the time interval between the timestamp of the current callback and the timestamp of the last playback video frame is short and cannot meet the full frame rate limit of the electronic device, so the video frame corresponding to the timestamp of the current callback is not determined as the current playback video frame.

[0151] For example, assuming that the timestamp corresponding to the drag operation of the current callback on the playback timeline is 00:00:01.10, the timestamp of the previous playback video frame is 00:00:01.00, and the full frame rate interval is 20 milliseconds, then it can be calculated that the time interval between the timestamp corresponding to the drag operation of the current callback on the playback timeline and the timestamp of the previous playback video frame is 10 milliseconds, which is less than the full frame rate interval, so the electronic device does not determine the video corresponding to the timestamp of the current callback as the playback video frame.

[0152] Next, the user continues the dragging operation, and the next returned dragging operation corresponds to a timestamp of 00:00:01.20 on the playback timeline. The time interval between the timestamp of 00:00:01.00 of the previous playback video frame is 20 milliseconds, which is not less than the full frame rate interval. Therefore, the electronic device can determine the video frame corresponding to the next returned timestamp as the playback video frame.

[0153] The flowchart of the second method for determining the playback video frame can be as follows: Figure 5 As shown, the following steps may be specifically included:

[0154] S501, determining, from the playback video, a video frame corresponding to a timestamp included in the playback timeline portion through which the drag operation passes;

[0155] Since the playback timeline portion through which the user drags usually includes multiple timestamps, the video frames corresponding to the timestamps included in the playback timeline portion through which the drag operation passes usually also include multiple timestamps. The electronic device can determine the video frames corresponding to the timestamps included in the playback timeline portion through which the drag operation passes from the playback video.

[0156] S502: extract frames from the determined video frames to obtain playback video frames, so that the time interval between adjacent video frames in the playback video frames is not less than the full frame rate interval.

[0157] If the number of determined video frames is large, playing all the determined video frames may cause playback jams, so the determined video frames may be extracted to obtain playback video frames, so that the time interval between adjacent video frames in the playback video frames is not less than the full frame rate interval. In one embodiment, the specific method of extracting frames may be interval extraction.

[0158] For example, assuming that the playback timeline portion of the operation includes timestamps 05:00:00.00, 05:00:00.05, 05:00:00.10, 05:00:00.15 and 05:00:00.20, and the full frame rate interval is 10 milliseconds, then the video frames with timestamps 05:00:00.05 and 05:00:00.15 can be extracted, and the resulting playback video frames are the video frames corresponding to timestamps 05:00:00.00, 05:00:00.10 and 05:00:00.20.

[0159] For another example, suppose that in the playback video, the time interval between adjacent video frames is 60 seconds, and the user drags the playback timeline over a distance corresponding to 1 hour within 1 second. Then the timestamps included in the playback timeline portion of the dragging operation correspond to video frames 1 to 60. Playing 60 video frames within 1 second may cause the electronic device to freeze, so some of the 60 video frames must be filtered out. Assuming that the time interval between adjacent video frames is 15 milliseconds and the full frame rate interval is 30 milliseconds, the electronic device can extract the odd-numbered video frames from video frames 1 to 60 to obtain video frames 1, 3, ..., 59.

[0160] It can be seen that in the embodiment of the present application, the electronic device can use any of the above two implementations to determine the playback video frame from the playback video, so that the time interval between the determined timestamps of adjacent playback video frames is not less than the full frame rate interval, thereby allowing the electronic device to meet the full frame rate limit. The above implementation can quickly determine the playback video frame from the playback video, and can avoid playback freezes when the user drags the playback timeline too fast.

[0161] As an implementation method of the present application, Figure 6 As shown, the above step of determining the time interval between the timestamp corresponding to the drag operation of the current callback on the playback timeline and the timestamp of the last playback video frame; if the time interval is not less than the full frame rate interval, determining from the playback video the video frame corresponding to the timestamp of the current callback as the current playback video frame may include:

[0162] S601, obtaining a timestamp corresponding to the drag operation of the current callback on the playback timeline from a pre-established serial playback queue;

[0163] During the playback of a video, the user's dragging speed may be fast or slow, and the video frame interval of the playback video is fixed. If the user drags faster, the playback timeline portion that the dragging operation passes through per unit time includes more video frames corresponding to the timestamp. If all these video frames are played, playback may be jittery.

[0164] In order to improve the smoothness of playing the playback video when the user quickly drags the playback timeline, a serial playback queue can be established in advance. Among them, the serial playback queue is used to record the timestamp corresponding to the callback drag operation on the playback timeline. When the user performs a drag operation, the electronic device can obtain the timestamp corresponding to the current callback drag operation on the playback timeline from the serial playback queue.

[0165] S602, determining the time interval between the acquired timestamp and the timestamp of the last playback video frame; if the time interval is not less than the full frame rate interval, executing step S603; if the time interval is less than the full frame rate interval, executing step S604;

[0166] After the electronic device obtains the timestamp of the current callback from the serial play queue, it can determine the time interval between the obtained timestamp and the timestamp of the last playback video frame. In the case of determining the playback video frame for the first time, the first playback video frame can be the video frame corresponding to the first timestamp included in the playback timeline passed by the user's drag operation.

[0167] S603, according to the timestamps of the playback video frames included in the cached frame analysis results of the playback video frames, determining the video frame corresponding to the acquired timestamp from the playback video as the current playback video frame;

[0168] The electronic device may pre-cache the frame analysis results of each video frame in the playback video, and the frame analysis results may include a timestamp. The frame analysis results may be cached by the electronic device after performing frame analysis on each video frame in the playback video, or may be cached by the electronic device after another image processing device performs frame analysis on each video frame in the playback video and sends the frame analysis results to the electronic device.

[0169] If the time interval is not less than the full frame rate interval, it means that the time interval between the acquired timestamp and the timestamp of the previous playback video frame is long enough to meet the full frame rate limitation of the electronic device. Therefore, the electronic device can determine the video frame corresponding to the acquired timestamp from the playback video based on the timestamps of each playback video frame included in the frame analysis results of the cached playback video frames, as the current playback video frame.

[0170] S604: Do not determine the video frame corresponding to the acquired timestamp as the currently played back video frame.

[0171] If the time interval is less than the full frame rate interval, it means that the time interval between the acquired timestamp and the timestamp of the last playback video frame is short and cannot meet the full frame rate limitation of the electronic device, so the electronic device does not determine the video frame corresponding to the acquired timestamp as the current playback video frame.

[0172] For example, assuming that the drag operation currently callback in the serial playback queue corresponds to a timestamp of 10:30:00.30 on the playback timeline, the timestamp of the previous playback video frame is 10:30:00.20, and the full frame rate interval is 20 milliseconds, then the electronic device can obtain the timestamp 10:30:00.30. Since the time interval between the obtained timestamp 10:30:00.30 and the timestamp 10:30:00.20 of the previous playback video frame is less than the full frame rate interval, the video frame corresponding to the timestamp 10:30:00.30 is not determined as the current playback video frame.

[0173] Next, as the drag operation continues, the electronic device obtains the next callback drag operation from the serial playback queue and the timestamp corresponding to the playback timeline is 10:30:00.40. Since the time interval between the obtained timestamp 10:30:00.40 and the timestamp 10:30:00.20 of the previous playback video frame is not less than the full frame rate interval, the video frame corresponding to the timestamp 10:30:00.40 can be determined as the current playback video frame.

[0174] It can be seen that in the embodiment of the present application, the electronic device can obtain the timestamp corresponding to the drag operation of the current callback on the playback timeline from the pre-established serial playback queue, wherein the serial playback queue is used to record the timestamp corresponding to the drag operation of the callback on the playback timeline; determine the time interval between the obtained timestamp and the timestamp of the previous playback video frame; if the time interval is not less than the full frame rate interval, determine the video frame corresponding to the obtained timestamp from the playback video according to the timestamps of each playback video frame included in the frame analysis results of the cached playback video frame, as the current playback video frame. In this way, the serial playback queue can be used to accurately record the timestamp of each callback, so that the time interval between the determined timestamps of adjacent playback video frames is not less than the full frame rate interval, which can avoid the occurrence of playback jams when the user drags the playback timeline too fast.

[0175] As an implementation method of the present application, Figure 7 As shown, the step of determining the playback video frame from the playback video according to the dragging speed of the dragging operation may include:

[0176] S701, determining the number of video frames corresponding to the timestamps included in the playback timeline portion through which the drag operation passes;

[0177] If the portion of the playback timeline through which the dragging operation passes includes a large number of video frames corresponding to timestamps and the dragging operation is short, then in this case, the frame rate to be played when all these video frames are played may be greater than the full frame rate, causing playback stuttering.

[0178] In order to avoid playback jams, it is necessary to control the frame rate of the electronic device playing the playback video to be no greater than the full frame rate. If the frame rate to be played is no greater than the full frame rate, the electronic device can play all of these video frames, and there will be no playback jams. If the frame rate to be played is greater than the full frame rate, the electronic device needs to extract these video frames at full frame rate intervals, and then play the extracted video frames. Since the frame rate is the number of video frames played per second, in order to calculate the frame rate to be played, the electronic device can determine the number of video frames corresponding to the timestamps included in the portion of the playback timeline that the drag operation passes through.

[0179] In one embodiment, since each scale line included in the playback timeline corresponds to a video frame in the playback video, the number of scale lines included in the portion of the playback timeline through which the drag operation passes can be used as the number of video frames corresponding to the timestamps included in the portion of the playback timeline through which the drag operation passes.

[0180] S702, calculating a quotient between the number of video frames and the duration of the dragging operation as a frame rate to be played;

[0181] After determining the number of video frames, the electronic device can calculate the quotient between the number of video frames and the duration of the drag operation to obtain the frame rate to be played. For example, assuming that the number of video frames determined by the electronic device is 20 frames and the drag duration is 1 second, the frame rate to be played can be determined to be 20fps.

[0182] S703, determining a video frame interval according to a magnitude relationship between the frame rate to be played and the full frame rate;

[0183] If the frame rate to be played is not greater than the full frame rate, it means that the playback video will not be stuck if it is played at the frame rate to be played, so the video frame interval can be determined according to the frame rate to be played, and then the playback video can be played. If the frame rate to be played is greater than the full frame rate, it means that the playback video will be stuck if it is played at the frame rate to be played, so the video frame interval can be determined according to the full frame rate, and then the playback video can be played.

[0184] S704: Determine a playback video frame from the playback video according to the video frame interval.

[0185] After determining the video frame interval, the playback video frame can be determined from the playback video according to the video frame interval. Specifically, the electronic device can determine the video frame satisfying that the time interval between adjacent video frames is not less than the video frame interval as the playback video frame.

[0186] It can be seen that in the embodiment of the present application, the electronic device can determine the number of video frames corresponding to the timestamps included in the playback timeline portion through which the drag operation passes; calculate the quotient between the number of video frames and the duration of the drag operation as the frame rate to be played; determine the video frame interval according to the size relationship between the frame rate to be played and the full frame rate; and determine the playback video frames from the playback video according to the video frame interval. Since the playback video frames are determined according to the video frame interval, and the video frame interval can meet the full frame rate requirements, the playback video frames determined from the playback video can also meet the full frame rate requirements when playing, which can avoid the occurrence of playback freezes when the user quickly drags the playback timeline.

[0187] As an implementation method of the present application, Figure 8 As shown, the step of determining the video frame interval according to the size relationship between the frame rate to be played and the full frame rate may include:

[0188] S801, determining the size relationship between the frame rate to be played and the full frame rate; if the frame rate to be played is greater than the full frame rate, executing step S802; if the frame rate to be played is not greater than the full frame rate, executing step S803;

[0189] If the frame rate to be played is greater than the full frame rate, it means that the playback of the video will be stuck when playing it at the frame rate to be played, so the video frame interval can be determined according to the full frame rate. If the frame rate to be played is not greater than the full frame rate, it means that the playback of the video will not be stuck when playing it at the frame rate to be played, so the video frame interval can be determined according to the frame rate to be played. Therefore, in order to determine the video frame interval, the electronic device can determine the size relationship between the frame rate to be played and the full frame rate.

[0190] S802, determining the full frame rate interval as the video frame interval;

[0191] If the frame rate to be played is greater than the full frame rate, the full frame rate interval can be determined as the video frame interval. Specifically, the reciprocal of the full frame rate can be calculated as the video frame interval. For example, assuming that the full frame rate is 20fps and the frame rate to be played is 40fps, the video frame interval can be determined according to the full frame rate, and the determined video frame interval is 50 milliseconds.

[0192] S803: Determine the time interval corresponding to the frame rate to be played as the video frame interval.

[0193] If the frame rate to be played is not greater than the full frame rate, the time interval corresponding to the frame rate to be played can be determined as the video frame interval. Specifically, the reciprocal of the frame rate to be played can be calculated as the video frame interval. For example, assuming that the full frame rate is 25fps and the frame rate to be played is 10fps, the video frame interval can be determined according to the frame rate to be played, and the determined video frame interval is 100 milliseconds.

[0194] It can be seen that in the embodiment of the present application, the electronic device can determine the size relationship between the frame rate to be played and the full frame rate; if the frame rate to be played is greater than the full frame rate, the full frame rate interval is determined as the video frame interval; if the frame rate to be played is not greater than the full frame rate, the time interval corresponding to the frame rate to be played is determined as the video frame interval. In this way, the video frame interval can be determined quickly and accurately, and in the subsequent steps, the playback video frame determined according to the video frame interval can meet the full frame rate requirement, which can avoid the occurrence of playback jams when the user quickly drags the playback timeline.

[0195] As an implementation manner of the embodiment of the present application, the step of determining the playback video frame from the playback video according to the video frame interval may include one of the following two implementation manners:

[0196] Determine the time interval between the timestamp corresponding to the drag operation of the current callback on the playback timeline and the timestamp of the last playback video frame; if the time interval is not less than the video frame interval, determine from the playback video the video frame corresponding to the timestamp of the current callback as the current playback video frame; or, from the playback video, determine the video frame corresponding to the timestamp included in the portion of the playback timeline through which the drag operation passes; extract the determined video frames to obtain playback video frames, so that in the playback video frames, the time interval between adjacent video frames is not less than the video frame interval.

[0197] The difference between the above two implementations and the implementations described in the above embodiments is that in the first implementation of this embodiment, it is determined whether the time interval is not less than the video frame interval. In the second implementation, the time interval between adjacent video frames in the playback video frame is not less than the video frame interval, and the remaining steps are the same and will not be repeated here.

[0198] As an implementation method of the present application, Fig. 9 As shown, the step of downloading the original video corresponding to each sub-time period, and extracting frames of the original video corresponding to each sub-time period according to the current frame extraction interval during the download process to obtain the playback video may include:

[0199] S901, constructing a download task queue of condensed videos corresponding to the multiple sub-time periods;

[0200] In order to download the condensed video download tasks corresponding to each sub-time period in a certain order, a download task queue of the condensed video corresponding to multiple sub-time periods can be constructed. The download task queue may include the download tasks of the condensed video corresponding to each sub-time period, and the order of the download tasks is determined based on the time interval between the sub-time period corresponding to the download task and the current time, and the current time is the time in the physical world.

[0201] The order of download tasks can be determined based on:

[0202] In the first implementation, the download task corresponding to the sub-time period that is closer to the time interval between the current time and the current time can be arranged in a more forward position. Specifically, the current time can be taken as the center, and half of the duration of the current condensed video can be intercepted before and after the current time to obtain the current sub-time period, and the download task corresponding to the current sub-time period can be used as the first download task in the download task queue. Next, the electronic device can cyclically perform the following steps until the total time period corresponding to the original video is completely divided into sub-time periods: first, a sub-time period with a duration of the current condensed duration is intercepted before the current sub-time period, and the download task corresponding to the sub-time period is used as the next download task in the download task queue, and then a sub-time period with a duration of the current condensed duration is intercepted after the current sub-time period, and the download task corresponding to the sub-time period is used as the next download task in the download task queue. A duration threshold can be pre-set, and a time period with a duration less than the duration threshold can be merged with an adjacent time period into a sub-time period. Among them, the duration threshold is less than the duration of the current condensed video, and can be specifically half, one-third, etc. of the current condensed duration, which is not specifically limited here.

[0203] For example, a schematic diagram of the download task queue can be as follows Fig.10 As shown, assuming that the current time is 13:00, the current condensed video duration is 3 hours, and the total time period corresponding to the original video is 00:00-24:00, then 13:00 can be taken as the center, and 1.5 hours can be cut forward and backward respectively to obtain the current sub-time period 11:30-14:30, and the download task corresponding to the current sub-time period can be taken as the first download task.

[0204] Next, you can first intercept 3 hours before the current sub-time period to obtain the sub-time period 08:30-11:30, and use the download task corresponding to the sub-time period as the second download task. Then, you can intercept 3 hours after the current sub-time period to obtain the sub-time period 14:30-17:30, and use the download task corresponding to the sub-time period as the third download task. The electronic device can cyclically perform the above steps to sequentially determine the sub-time periods 05:30-08:30, 17:30-20:30, 02:30-05:30, 20:30-24:00, and 00:00-02:30, and determine the download tasks corresponding to the determined sub-time periods as the fourth to eighth download tasks, respectively.

[0205] Among them, 20:30-24:00 includes a time period of 20:30-23:30 with a duration of 3 hours and a time period of 23:30-24:00 with a duration of less than 3 hours. The time threshold of 1.5 hours can be pre-set to merge the time period of 23:30-24:00 with a duration of less than 1.5 hours and the adjacent time period of 20:30-23:30 into a sub-time period of 20:30-24:00. Although the time period of 00:00-02:30 is less than 3 hours, the duration of this time period is greater than the time threshold, so the download task corresponding to this time period can be used as a separate sub-time period.

[0206] In a second implementation, the electronic device may place the download task corresponding to the sub-time period that is closer to the current time at a later position. Specifically, the following steps may be performed cyclically until the total time period corresponding to the original video is completely divided into sub-time periods: from the time points that do not belong to any sub-time period, determine the time point with the largest time interval from the current time, use the time point as the endpoint, determine the sub-time period with a duration equal to the duration of the current condensed video, and use the determined order of each sub-time period as the order of the download task corresponding to the sub-time period in the download task queue.

[0207] For example, assuming that the current time is 22:00, the current condensed video duration is 6 hours, and the total time period corresponding to the original video is 00:00-24:00, then the time point 00:00 with the largest time interval with the current time can be determined, and the sub-time period 00:00-06:00 can be determined. Next, the sub-time periods 06:00-12:00, 12:00-18:00 and 18:00-24:00 can be determined in turn. The above four sub-time periods are respectively used as the first to fourth download tasks in the determined order.

[0208] S902, executing the download tasks in the download task queue, and in the process of executing each download task, extracting frames for the original video corresponding to each sub-time period of the download according to the current frame extraction interval to obtain a condensed video corresponding to each sub-time period.

[0209] After the download task queue is constructed, each download task in the download task queue can be executed in sequence according to the order of the download tasks. In the process of executing each download task, the original video corresponding to each sub-time period of the download can be framed according to the current frame extraction interval to obtain a condensed video corresponding to each sub-time period.

[0210] It can be seen that in the embodiment of the present application, the electronic device can construct a download task queue of condensed videos corresponding to multiple sub-time periods, wherein the download task queue includes download tasks of condensed videos corresponding to each sub-time period, and the order of the download tasks is determined based on the time interval between the sub-time period corresponding to the download task and the current time; the download tasks in the download task queue are executed, and in the process of executing each download task, the original video corresponding to each downloaded sub-time period is frame-drawn according to the current frame-drawing interval to obtain the condensed video corresponding to each sub-time period. Since the download task queue can be constructed in a certain order, the download task corresponding to each sub-time period can be quickly downloaded.

[0211] As an implementation method of the present application, Fig.11 As shown, the above method may also include:

[0212] S1101, in the process of downloading the original video corresponding to each sub-time period, in response to a zoom level adjustment instruction issued by the user based on the playback timeline, determining whether the frame extraction intervals corresponding to the timeline zoom levels before and after the adjustment are the same according to the corresponding relationship; if the frame extraction intervals corresponding to the timeline zoom levels before and after the adjustment are different, executing step S1102; if the frame extraction intervals corresponding to the timeline zoom levels before and after the adjustment are the same, executing step S1103;

[0213] The user may issue a zoom level adjustment instruction based on the playback timeline. Specifically, the zoom level adjustment instruction may be used to indicate increasing or decreasing the timeline zoom level. For example, assuming that the timeline zoom levels include 1 hour, 30 minutes, and 15 minutes, and the current timeline zoom level is 30 minutes, if the user issues a zoom level adjustment instruction for indicating increasing the timeline zoom level, the current timeline zoom level will be adjusted to 1 hour, and if the user issues a zoom level adjustment instruction for decreasing the timeline zoom level, the current timeline zoom level will be adjusted to 15 minutes.

[0214] Since increasing the timeline zoom level and decreasing the timeline zoom level have opposite effects, the action directions of the user operation for indicating increasing or decreasing the timeline zoom level may also be opposite. For example, the user operation for indicating decreasing the timeline zoom level may be to place two fingers in the playback timeline area and touch the screen to expand outward, and the user operation for indicating increasing the timeline zoom level may be to place two fingers in the playback timeline area and touch the screen to shrink inward.

[0215] For another example, the user operation for indicating an increase in the timeline zoom level may be that the user touches the screen in the playback timeline area and swipes upward, and the user operation for indicating a decrease in the timeline zoom level may be that the user touches the screen in the playback timeline area and swipes downward.

[0216] When the timeline zoom level changes, the frame extraction interval corresponding to the timeline zoom level may also change. Since each tick mark on the playback timeline corresponds to a video frame in the playback video, and each tick mark has the same timestamp as the corresponding video frame, the time interval between the timestamps corresponding to adjacent tick marks is the frame extraction interval. When the frame extraction interval changes, each tick mark on the playback timeline can be adjusted according to the changed frame extraction interval. Specifically, if the frame extraction interval becomes larger, the tick marks on the playback timeline will become sparse; if the frame extraction interval becomes smaller, the tick marks on the playback timeline will become dense.

[0217] For example, a schematic diagram of adjusting the time axis zoom level may be shown in FIG. 12( a ) to FIG. 12( c ), assuming that the corresponding relationship between the time axis zoom level and the frame extraction interval is as shown in Table 2:

[0218] Table 2

[0219]

[0220] In FIG. 12 (a), the time axis zoom level is 30 minutes, the corresponding frame extraction interval is 60 seconds, and the original video duration is 30 minutes. Therefore, the playback video includes 30 video frames, and the timestamps of each video frame are 00:01:00, 00:02:00, 00:03:00, ... 00:30:00. Accordingly, the first playback time axis 1201 includes 30 scale lines, and the timestamps corresponding to each scale line are 00:01:00, 00:02:00, 00:03:00, ... 00:30:00. The currently displayed picture is the picture corresponding to the timestamp 00:14:00.

[0221] Assuming that in the case of FIG. 12 (a), the user's two fingers touch the screen inwards in the area near the first playback timeline 1201, then the schematic diagram of the playback timeline will change from FIG. 12 (a) to FIG. 12 (b). In FIG. 12 (b), the timeline zoom level is the next level of 30 minutes, that is, 1 hour, and the corresponding frame extraction interval is 120 seconds, then the playback video includes 15 video frames, and the timestamps of each video frame are 00:02:00, 00:04:00, 00:06:00...00:30:00. Correspondingly, the second playback timeline 1202 includes 15 scale lines, and the timestamps corresponding to each scale line are 00:02:00, 00:04:00, 00:06:00...00:30:00, respectively, so the scale lines become sparse.

[0222] Assuming that in the case of FIG. 12 (a), the user's two fingers touch the screen in the area near the first playback timeline 1201 and expand outward, then the schematic diagram of the playback timeline will change from FIG. 12 (a) to FIG. 12 (c). In FIG. 12 (c), the timeline zoom level is the previous level of 30 minutes, that is, 15 minutes, and the corresponding frame extraction interval is 30 seconds, then the playback video includes 60 video frames, and the timestamps of each video frame are 00:00:30, 00:01:00, 00:01:30...00:30:00. Correspondingly, the third playback timeline 1203 includes 60 scale lines, and the timestamps corresponding to each scale line are 00:00:30, 00:01:00, 00:01:30...00:30:00, so the scale lines become dense.

[0223] In order to be able to extract frames for the original video according to the changed frame extraction interval after the frame extraction interval corresponding to the timeline zoom level changes, the electronic device, in the process of downloading the original video corresponding to each sub-time period, can respond to the user's zoom level adjustment instruction based on the playback timeline. It can determine whether the frame extraction intervals corresponding to the timeline zoom levels before and after the adjustment are the same according to the correspondence between the preset timeline zoom level, the condensed video length and the frame extraction interval.

[0224] S1102, taking the adjusted timeline zoom level as the current timeline zoom level, and returning to the step of determining the current condensed video duration and the current frame extraction interval corresponding to the current timeline zoom level according to the correspondence between the preset timeline zoom level, the condensed video duration, and the frame extraction interval;

[0225] If the frame extraction intervals corresponding to the timeline zoom levels before and after the adjustment are different, the electronic device can use the adjusted timeline zoom level as the current timeline zoom level, and return to the above-mentioned step of determining the current condensed video duration and the current frame extraction interval corresponding to the current timeline zoom level according to the correspondence between the preset timeline zoom level and the condensed video duration and the frame extraction interval, thereby determining the current frame extraction interval and the current condensed video duration corresponding to the current timeline zoom level, and then readjusting the download task queue according to the current frame extraction interval and the current condensed video duration. In one embodiment, the electronic device may not stop the currently executing download task, that is, it may complete the currently executing download task, and then execute each download task in the adjusted download task queue.

[0226] For example, when the timeline zoom level is adjusted, a schematic diagram of the download task queue changes can be as follows: Fig.13 As shown, the left picture is the download task queue before adjustment, the timeline zoom level is 15 minutes, the condensed video duration corresponding to this timeline zoom level is 3 hours, the corresponding frame extraction interval is 30 seconds, and the order of the download tasks is 06:00-09:00, 03:00-06:00, 09:00-12:00 and 00:00-03:00.

[0227] Assume that the user adjusts the timeline zoom level to 5 minutes. The condensed video duration corresponding to this timeline zoom level is 2 hours, and the corresponding frame extraction interval is 20 seconds. The right picture shows the adjusted download task queue. The electronic device can keep the order of the download tasks corresponding to 06:00-09:00 being executed unchanged, and redetermine the download task queue according to the adjusted condensed video duration. In the redetermined download task queue, the order of the download tasks is 06:00-09:00, 04:00-06:00, 09:00-12:00, 02:00-04:00, and 00:00-02:00.

[0228] S1103, continue downloading the original video corresponding to each sub-time period.

[0229] Since the frame extraction interval usually changes simultaneously with the condensed video duration, if the frame extraction interval corresponding to the timeline zoom level before and after the adjustment is the same, it means that the condensed video duration before and after the adjustment is also the same. In this case, the download task queue can be kept unchanged and the original video corresponding to each sub-time period can continue to be downloaded.

[0230] It can be seen that in the embodiment of the present application, the electronic device can, in the process of downloading the original video corresponding to each sub-time period, respond to the zoom level adjustment instruction issued by the user based on the playback timeline, and determine whether the frame extraction intervals corresponding to the timeline zoom levels before and after the adjustment are the same according to the corresponding relationship; if the frame extraction intervals corresponding to the timeline zoom levels before and after the adjustment are different, the adjusted timeline zoom level is used as the current timeline zoom level, and the step of determining the current condensed video duration and the current frame extraction interval corresponding to the current timeline zoom level according to the corresponding relationship between the preset timeline zoom level and the condensed video duration and the frame extraction interval is returned. Since the download task queue can be re-determined when the frame extraction intervals corresponding to the timeline zoom levels before and after the adjustment are different, it can be ensured that the download task queue is always adapted to the current timeline zoom level, which can improve the download efficiency. In addition, when the user drags the playback timeline corresponding to different timeline zoom levels, the played video frames will show regular arithmetic changes in the OSD time. That is to say, when the timeline zoom level is different, the corresponding frame extraction interval may also be different. Accordingly, the time interval between the scale line corresponding to each video frame on the playback timeline and the scale line corresponding to the adjacent video frame on the playback timeline will also change adaptively.

[0231] As an implementation method of the present application, Fig.14 As shown, the above method may also include:

[0232] S1401, in the process of downloading the original video corresponding to each sub-time period, in response to a drag operation issued by a user based on a playback timeline, in the download task queue, adjusting the download task corresponding to the sub-time period containing a timestamp corresponding to the current position of the drag operation on the playback timeline to be the first task other than the download task being executed;

[0233] During the execution of a download task, if the user issues a drag operation based on the playback timeline, the order of the download tasks in the download task queue can be adjusted. Specifically, the download task corresponding to the sub-time period containing the timestamp corresponding to the current position of the drag operation on the playback timeline can be adjusted to be the first task except for the download task being executed.

[0234] S1402: Adjust the remaining download tasks based on the time interval between the sub-time period corresponding to the download task and the timestamp corresponding to the current position of the dragging operation on the playback timeline.

[0235] For the remaining download tasks, the order of the download tasks in the download task queue can be determined based on the time interval between the sub-time period corresponding to the download task and the timestamp corresponding to the current position of the drag operation on the playback timeline. In one embodiment, with the timestamp corresponding to the current position of the drag operation on the playback timeline as the center, the first sub-time period before the timestamp, the first sub-time period after the timestamp, the second sub-time period before the timestamp, the second sub-time period after the timestamp, etc. are determined in sequence, and the download tasks corresponding to the determined sub-time periods are respectively used as the second, third, fourth, fifth, etc. download tasks in addition to the download task being executed.

[0236] For example, in the case of dragging the playback timeline, a schematic diagram of the download task queue changes can be as follows: Fig.15 The left picture shows the download task queue before dragging. Assuming the current time is 13:00, the download task corresponding to the sub-time period including the current time can be taken as the first download task, and then the order of the download tasks is determined as 11:30-14:30, 08:30-11:30, 14:30-17:30, 05:30-08:30, 17:30-20:30, 02:30-05:30, 20:30-24:00 and 00:00-02:30.

[0237] Assume that the user drags the playback timeline, and the timestamp corresponding to the current position of the dragging operation on the playback timeline is 18:00, then the download task queue after dragging will be as shown in the right figure. The download task of the condensed video corresponding to the current downloading time of 11:30-14:30 can continue to be executed, and the sub-time period 17:30-20:30 including the timestamp 18:00 corresponding to the current position of the dragging operation on the playback timeline will be used as the first download task except for the download task being executed.

[0238] Next, the order of other download tasks can be determined. Specifically, the download task corresponding to the first sub-time period 14:30-17:30 before 18:00 can be determined as the second download task except the download task being executed, and the download task corresponding to the first sub-time period 20:30-24:00 after 18:00 can be determined as the third download task except the download task being executed, and the cycle continues until the order of all other download tasks is re-determined, and the final order of the download tasks is the download tasks corresponding to 11:30-14:30, 17:30-20:30, 14:30-17:30, 20:30-24:00, 08:30-11:30, 05:30-08:30, 02:30-05:30, and 00:00-02:30.

[0239] It can be seen that in the embodiment of the present application, the electronic device can, in the process of downloading the original video corresponding to each sub-time period, in response to the drag operation issued by the user based on the playback timeline, adjust the download task corresponding to the sub-time period containing the timestamp corresponding to the current position of the drag operation on the playback timeline in the download task queue to be the first task other than the download task being executed; and adjust the remaining download tasks based on the time interval between the sub-time period corresponding to the download task and the timestamp corresponding to the current position of the drag operation on the playback timeline. Since the order of the download tasks in the download task queue can be adjusted after the user drags the playback timeline, the adjusted download task queue can be adapted to the drag operation, thereby improving the download efficiency.

[0240] As an implementation of the embodiment of the present application, the above method may also include:

[0241] When it is detected that the timestamp corresponding to the current position of the drag operation on the playback timeline is close to the timestamp corresponding to the last video frame of the playback video corresponding to the current time period, the condensed video corresponding to the next time period is preloaded.

[0242] If, during the process of dragging, the timestamp corresponding to the current position of the dragging operation on the playback timeline is close to the timestamp corresponding to the last video frame of the playback video corresponding to the current time period, it means that the dragging operation is about to enter the part of the playback timeline corresponding to the next time period of the current time period.

[0243] In order to improve the smoothness of playing the playback video and reduce the freeze in the transition between segments caused by the segmentation of the condensed video, when it is detected that the timestamp corresponding to the current position of the drag operation on the playback timeline is close to the timestamp corresponding to the last video frame of the playback video corresponding to the current time period, the electronic device can preload the condensed video corresponding to the next time period. The current time period is a sub-time period including the current timestamp of the playback timeline, the last video frame is the last video frame in the drag direction of the drag operation among the video frames of the playback video corresponding to the current time period, and the next time period is the next sub-time period of the current time period in the drag direction of the drag operation.

[0244] In one embodiment, a time interval threshold may be pre-set. If the time interval between the timestamp corresponding to the current position of the drag operation on the playback timeline and the timestamp corresponding to the last video frame of the playback video corresponding to the current time period is not greater than the time interval threshold, it is determined that the timestamp corresponding to the current position of the drag operation on the playback timeline and the timestamp corresponding to the last video frame of the playback video corresponding to the current time period are close to each other.

[0245] For example, assuming that the playback video includes sub-time period 1 to sub-time period 4, sub-time period 1 is 00:00-03:00, sub-time period 2 is 03:00-06:00, sub-time period 3 is 06:00-09:00, and sub-time period 4 is 09:00-12:00, and the time interval threshold is 10 minutes.

[0246] If the user starts dragging the playback timeline from sub-time period 2, and the dragging direction is from 00:00 to 12:00, then the current time period is sub-time period 2, and the last video frame of the playback video corresponding to the current time period is 06:00. At a certain moment, the timestamp corresponding to the current position of the drag operation on the playback timeline is 05:50, and the timestamp and the last video frame of the playback video corresponding to the current time period are not greater than the time interval threshold. Since the next sub-time period of the current time period is sub-time period 3 in the dragging direction of the drag operation, the condensed video corresponding to sub-time period 3 can be preloaded at this moment.

[0247] If the user starts dragging the playback timeline from sub-time period 4, and the dragging direction is from 12:00 to 00:00, then the current time period is sub-time period 4, and the last video frame of the playback video corresponding to the current time period is 09:00. At a certain moment, the timestamp corresponding to the current position of the drag operation on the playback timeline is 09:10, and the timestamp and the last video frame of the playback video corresponding to the current time period are not greater than the time interval threshold. Since the next sub-time period of the current time period is sub-time period 3 in the dragging direction of the drag operation, the condensed video corresponding to sub-time period 3 can be preloaded at this moment.

[0248] It can be seen that in the embodiment of the present application, the electronic device can preload the condensed video corresponding to the next time period when it is detected that the timestamp corresponding to the current position of the drag operation on the playback timeline is close to the timestamp corresponding to the last video frame of the playback video corresponding to the current time period; wherein the current time period is a sub-time period containing the current timestamp of the playback timeline, and the next time period is the next sub-time period of the current time period in the dragging direction of the drag operation. Since the condensed video of the next time period will be preloaded when the user drags the playback timeline to slide out of a time period and enter the next time period, the smoothness of the playback video playback can be improved, and the freeze during the transition between segments caused by the segmentation of the condensed video can be reduced, so that smooth dragging can be achieved, that is, the playback screen can smoothly transition when dragging the playback timeline.

[0249] As an implementation method of the present application, Fig.16 As shown, before the step of determining the playback video frame from the playback video according to the dragging speed of the dragging operation, the method may further include:

[0250] S1601, determining whether there is a frame analysis result of the playback video corresponding to the current time period; if there is no frame analysis result of the playback video corresponding to the current time period, executing step S1602; if there is a frame analysis result corresponding to the current time period, executing step S1605;

[0251] After downloading the playback video, the electronic device can perform frame analysis on the playback video and cache the frame analysis results. Since it is necessary to determine the playback video frame based on the timestamp included in the frame analysis results during the video playback process, before determining the playback video frame, the electronic device can determine whether there is a frame analysis result of the playback video corresponding to the current time period. The current time period is a sub-time period containing the timestamp corresponding to the current position of the playback time axis.

[0252] S1602, determining whether the playback video corresponding to the current time period has been downloaded; if the playback video corresponding to the current time period has not been downloaded, executing step S1603; if the playback video corresponding to the current time period has been downloaded, executing step S1604;

[0253] If there is no frame analysis result of the playback video corresponding to the current time period, there may be the following two situations: one situation is that the electronic device has downloaded the playback video corresponding to the current time period, but has not performed frame analysis on the playback video corresponding to the current time period; the other situation is that the electronic device has not downloaded the playback video corresponding to the current time period. Therefore, in the case that there is no frame analysis result of the playback video corresponding to the current time period, the electronic device can determine whether the playback video corresponding to the current time period has been downloaded.

[0254] S1603, downloading the condensed video corresponding to the current time period, and performing frame analysis on the downloaded condensed video to obtain a frame analysis result;

[0255] If the playback video corresponding to the current time period has not been downloaded, the electronic device may download the condensed video corresponding to the current time period, and perform frame analysis on the downloaded condensed video to obtain a frame analysis result.

[0256] S1604, performing frame analysis on the playback video corresponding to the current time period to obtain a frame analysis result;

[0257] If the playback video corresponding to the current time period has been downloaded, the electronic device can perform frame analysis on the playback video corresponding to the current time period to obtain a frame analysis result.

[0258] S1605, executing the step of determining a playback video frame from the playback video according to the dragging speed of the dragging operation.

[0259] If the frame analysis result corresponding to the current time period already exists, the electronic device can perform the above step of determining the playback video frame from the playback video according to the dragging speed of the dragging operation, that is, perform step S101 in the above embodiment. In one embodiment, when the condensed video of a certain time period has been cached and the frame analysis result of the condensed video of the time period exists, if it is detected that the user drags the playback timeline within the time period of the playback timeline, the condensed video corresponding to the time period can be started to play.

[0260] It can be seen that in the embodiment of the present application, the electronic device can determine whether there is a frame analysis result of the playback video corresponding to the current time period, wherein the current time period is a sub-time period containing a timestamp corresponding to the current position of the playback timeline; if there is no frame analysis result of the playback video corresponding to the current time period, determine whether the playback video corresponding to the current time period has been downloaded; if the playback video corresponding to the current time period has not been downloaded, download the condensed video corresponding to the current time period, and perform frame analysis on the downloaded condensed video to obtain the frame analysis result; if there is a frame analysis result corresponding to the current time period, perform the step of determining the playback video frame from the playback video according to the dragging speed of the dragging operation. Because the electronic device will also determine whether there is a frame analysis result before playing the playback video frame, and perform frame analysis if there is no frame analysis result, it can ensure that the subsequent steps of playing the playback video frame are executed normally.

[0261] In the technical solution of this application, the operations involved in obtaining, storing, using, processing, transmitting, providing and disclosing user personal information are all carried out with the user's authorization.

[0262] Corresponding to the above-mentioned video playback method, the embodiment of the present application further provides a video playback device. The following is an introduction to a video playback device provided by the embodiment of the present application.

[0263] like Fig.17 As shown, a video playback device, the device comprising:

[0264] The playback video frame determination module 1701 is used to determine the playback video frame from the playback video according to the dragging speed of the dragging operation in response to the dragging operation issued by the user on the playback time axis of the currently played playback video during the video playback process, wherein the faster the dragging speed is, the longer the time interval between two adjacent video frames in the playback video frame is, or the time interval between two adjacent video frames in the playback video frame is equal, and the faster the dragging speed is, the more the number of the playback video frames is;

[0265] The playback video frame playing module 1702 is used to play the determined playback video frame following the dragging operation.

[0266] It can be seen that in the embodiment of the present application, the electronic device can, during the video playback process, respond to the user's drag operation on the playback timeline of the currently playing playback video, determine the playback video frame from the playback video according to the drag speed of the drag operation, wherein the faster the drag speed, the longer the time interval between two adjacent video frames in the playback video frame, or, the time interval between two adjacent video frames in the playback video frame is equal, the faster the drag speed, the more the number of playback video frames; following the drag operation, play the determined playback video frame. Since the time interval between two adjacent video frames in the playback video frame or the number of playback video frames will change with the change of the drag speed, the playback video frame determined from the playback video will also change when the user drags the timeline at different speeds. Specifically, the user can slow down the dragging speed at the places of interest. In this case, the time interval between two adjacent video frames in the playback video frame is shorter, or the time interval between two adjacent video frames is equal but the number of playback video frames played is small, and the user can carefully check the content of the playback video; the user can speed up the dragging speed at the places of no interest. In this case, the time interval between two adjacent video frames in the playback video frame is longer, or the time interval between two adjacent video frames is equal but the number of playback video frames played is large, and the content that the user is not interested in can be quickly skipped. Therefore, the speed at which the user drags the playback timeline can determine the speed of change of the playback video playback screen, and can improve the retrieval efficiency of video playback.

[0267] As an implementation of the embodiment of the present application, the above device may further include:

[0268] A parameter determination module, used to determine the current condensed video duration and the current frame extraction interval corresponding to the current time axis zoom level according to the correspondence between the preset time axis zoom level and the condensed video duration and the frame extraction interval;

[0269] A time segmentation module, used to segment the total time segment corresponding to the original video into multiple sub-time segments, wherein the duration of each sub-time segment is the duration of the current condensed video;

[0270] The video download module is used to download the original video corresponding to each sub-time period, and extract frames of the original video corresponding to each sub-time period according to the current frame extraction interval during the download process to obtain the playback video. As an implementation of an embodiment of the present application, the playback video frame determination module 1701 may include:

[0271] The first playback video frame determination submodule is used to determine the time interval between the timestamp corresponding to the drag operation of the current callback on the playback timeline and the timestamp of the previous playback video frame; if the time interval is not less than the full frame rate interval, determine from the playback video the video frame corresponding to the timestamp of the current callback as the current playback video frame;

[0272] The second playback video frame determination submodule is used to determine, from the playback video, a video frame corresponding to a timestamp included in the playback timeline portion through which the drag operation passes; extract frames from the determined video frame to obtain a playback video frame, so that the time interval between adjacent video frames in the playback video frame is not less than the full frame rate interval;

[0273] The full frame rate interval is the time interval between adjacent video frames when playing the video at full frame rate.

[0274] As an implementation of the embodiment of the present application, the first playback video frame determination submodule may include:

[0275] A timestamp acquisition unit, used to acquire the timestamp corresponding to the drag operation of the current callback on the playback timeline from a pre-established serial playback queue, wherein the serial playback queue is used to record the timestamp corresponding to the drag operation of the callback on the playback timeline;

[0276] a time interval determination unit, used to determine the time interval between the acquired timestamp and the timestamp of the last playback video frame;

[0277] The current playback video frame determination unit is used to determine the video frame corresponding to the acquired timestamp from the playback video as the current playback video frame based on the timestamps of each playback video frame included in the cached frame analysis results of the playback video frames if the time interval is not less than the full frame rate interval.

[0278] As an implementation of the embodiment of the present application, the video download module may include:

[0279] A download task queue construction submodule, used to construct a download task queue of condensed videos corresponding to the multiple sub-time periods, wherein the download task queue includes download tasks of condensed videos corresponding to each sub-time period, and the order of the download tasks is determined based on the time interval between the sub-time period corresponding to the download task and the current time;

[0280] The download task execution submodule is used to execute the download tasks in the download task queue, and in the process of executing each download task, the original video corresponding to each sub-time period of the download is framed according to the current frame extraction interval to obtain the condensed video corresponding to each sub-time period.

[0281] As an implementation of the embodiment of the present application, the above device may further include:

[0282] A frame extraction interval determination module is used to determine whether the frame extraction intervals corresponding to the time axis zoom levels before and after the adjustment are the same according to the corresponding relationship in response to a zoom level adjustment instruction issued by a user based on the playback time axis during the process of downloading the original video corresponding to each sub-time period;

[0283] A return module is used to, if the frame extraction intervals corresponding to the timeline zoom levels before and after adjustment are different, use the adjusted timeline zoom level as the current timeline zoom level, and return the step of determining the current condensed video duration and the current frame extraction interval corresponding to the current timeline zoom level based on the correspondence between the preset timeline zoom level, the condensed video duration and the frame extraction interval.

[0284] As an implementation of the embodiment of the present application, the above device may further include:

[0285] A first order determination module is used for, in the process of downloading the original video corresponding to each sub-time period, in response to a drag operation issued by a user based on a playback timeline, adjusting, in the download task queue, a download task corresponding to a sub-time period containing a timestamp corresponding to a current position of the drag operation on the playback timeline to be the first task other than the download task being executed;

[0286] The second sequence determination module is used to adjust the remaining download tasks based on the time interval between the sub-time period corresponding to the download task and the timestamp corresponding to the current position of the dragging operation on the playback timeline.

[0287] As an implementation of the embodiment of the present application, the above device may further include:

[0288] A condensed video preloading module, configured to preload a condensed video corresponding to a next time period when it is detected that a timestamp corresponding to a current position of the dragging operation on the playback timeline is close to a timestamp corresponding to a last video frame of the playback video corresponding to the current time period;

[0289] The current time period is a sub-time period including a current timestamp of the playback time axis, and the next time period is a next sub-time period of the current time period in the dragging direction of the dragging operation.

[0290] As an implementation of the embodiment of the present application, the above device may further include:

[0291] A frame analysis result determination module, used to determine whether there is a frame analysis result of the playback video corresponding to the current time period, wherein the current time period is a sub-time period including a timestamp corresponding to the current position of the playback time axis;

[0292] A video download determination module, used to determine whether the playback video corresponding to the current time period has been downloaded if there is no frame analysis result of the playback video corresponding to the current time period;

[0293] A frame analysis module, configured to download a condensed video corresponding to the current time period if the playback video corresponding to the current time period has not been downloaded, and perform frame analysis on the downloaded condensed video to obtain a frame analysis result;

[0294] An execution module is used to execute the step of determining a playback video frame from the playback video according to the dragging speed of the dragging operation if a frame analysis result corresponding to the current time period already exists.

[0295] The present application also provides an electronic device, such as Fig.18 As shown, including:

[0296] Memory 1801, used for storing computer programs;

[0297] The processor 1802 is used to implement the video playback method described in any of the above embodiments when executing the program stored in the memory 1801.

[0298] In addition, the electronic device may further include a communication bus and / or a communication interface, and the processor 1802, the communication interface, and the memory 1801 communicate with each other via the communication bus.

[0299] It can be seen that in the embodiment of the present application, the electronic device can, during the video playback process, respond to the user's drag operation on the playback timeline of the currently playing playback video, and determine the playback video frame from the playback video according to the drag speed of the drag operation, wherein the faster the drag speed, the longer the time interval between two adjacent video frames in the playback video frame, or the faster the drag speed, the more the number of playback video frames, and the time interval between two adjacent video frames in the playback video frame is equal; following the drag operation, the determined playback video frame is played. Since the time interval between two adjacent video frames in the playback video frame or the number of playback video frames will change with the change of the drag speed, the playback video frame determined from the playback video will also change when the user drags the timeline at different speeds. Specifically, the user can slow down the dragging speed at the places of interest. In this case, the time interval between two adjacent video frames in the playback video frame is shorter, or the time interval between two adjacent video frames is equal but the number of playback video frames played is small, and the user can carefully check the content of the playback video; the user can speed up the dragging speed at the places of no interest. In this case, the time interval between two adjacent video frames in the playback video frame is longer, or the time interval between two adjacent video frames is equal but the number of playback video frames played is large, and the content that the user is not interested in can be quickly skipped. Therefore, the speed at which the user drags the playback timeline can determine the speed of change of the playback video playback screen, and can improve the retrieval efficiency of video playback.

[0300] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0301] The communication interface is used for communication between the above electronic device and other devices.

[0302] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0303] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0304] In another embodiment provided in the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned video playback methods are implemented.

[0305] In another embodiment provided by the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any of the video playback methods in the above embodiments.

[0306] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a solid-state hard disk (SSD), etc.

[0307] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0308] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, electronic device, computer-readable storage medium, and computer program product embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0309] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A video playback method, characterized in that: The method comprises: Determine the current condensed video duration and the current frame extraction interval corresponding to the current time axis zoom level according to the correspondence between the preset time axis zoom level and the condensed video duration and the frame extraction interval; Divide the total time period corresponding to the original video into multiple sub-time periods, wherein the duration of each sub-time period is the duration of the current condensed video; Downloading the original video corresponding to each sub-time period, and extracting frames of the original video corresponding to each sub-time period according to the current frame extraction interval during the downloading process to obtain a playback video; During video playback, in response to a user's dragging operation on a playback timeline of a currently playing playback video, the playback video is frame-drawn according to a dragging speed of the dragging operation to determine a playback video frame, wherein the faster the dragging speed, the longer the time interval between two adjacent video frames in the playback video frame; Following the dragging operation, playing the determined playback video frame; The method further comprises: In the process of downloading the original video corresponding to each sub-time period, in response to a zoom level adjustment instruction issued by the user based on the playback time axis, determining whether the frame extraction intervals corresponding to the time axis zoom levels before and after the adjustment are the same according to the corresponding relationship; If the frame extraction intervals corresponding to the timeline zoom levels before and after adjustment are different, the adjusted timeline zoom level is used as the current timeline zoom level, and the step of determining the current condensed video length and the current frame extraction interval corresponding to the current timeline zoom level according to the correspondence between the preset timeline zoom level, the condensed video length and the frame extraction interval is returned.

2. The method according to claim 1, characterized in that: The step of extracting frames of the playback video according to the dragging speed of the dragging operation to determine the playback video frames includes: Determine the time interval between the timestamp corresponding to the drag operation of the current callback on the playback timeline and the timestamp of the last playback video frame; if the time interval is not less than the full frame rate interval, determine from the playback video the video frame corresponding to the timestamp of the current callback as the current playback video frame; or, Determine, from the playback video, a video frame corresponding to a timestamp included in the playback timeline portion through which the drag operation passes; extract the determined video frame to obtain a playback video frame, so that in the playback video frame, a time interval between adjacent video frames is not less than a full frame rate interval; The full frame rate interval is the time interval between adjacent video frames when playing the video at full frame rate.

3. The method according to claim 2, characterized in that The step of determining the time interval between the timestamp corresponding to the drag operation of the current callback on the playback timeline and the timestamp of the last playback video frame; if the time interval is not less than the full frame rate interval, determining from the playback video that the video frame corresponding to the timestamp of the current callback is the current playback video frame, comprises: Obtaining, from a pre-established serial play queue, a timestamp corresponding to the drag operation of the current callback on the playback timeline, wherein the serial play queue is used to record the timestamp corresponding to the drag operation of the callback on the playback timeline; Determine the time interval between the acquired timestamp and the timestamp of the last playback video frame; If the time interval is not less than the full frame rate interval, based on the timestamps of each playback video frame included in the cached frame analysis results of the playback video frame, a video frame corresponding to the acquired timestamp is determined from the playback video as the current playback video frame.

4. The method according to claim 1, characterized in that: The step of downloading the original video corresponding to each sub-time period, and extracting frames of the original video corresponding to each sub-time period according to the current frame extraction interval during the downloading process to obtain the playback video includes: Constructing a download task queue of the condensed videos corresponding to the multiple sub-time periods, wherein the download task queue includes a download task of the condensed videos corresponding to each sub-time period, and the order of the download tasks is determined based on the time interval between the sub-time period corresponding to the download task and the current time; The download tasks in the download task queue are executed, and in the process of executing each download task, the original video corresponding to each sub-time period of the download is frame-drawn according to the current frame-drawing interval to obtain the condensed video corresponding to each sub-time period.

5. The method according to claim 1, characterized in that The method further comprises: In the process of downloading the original video corresponding to each sub-time period, in response to a drag operation issued by the user based on the playback timeline, in the download task queue, the download task corresponding to the sub-time period containing the timestamp corresponding to the current position of the drag operation on the playback timeline is adjusted to be the first task except the download task being executed; Based on the time interval between the sub-time period corresponding to the download task and the timestamp corresponding to the current position of the dragging operation on the playback timeline, the remaining download tasks are adjusted.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: When it is detected that the timestamp corresponding to the current position of the drag operation on the playback timeline is close to the timestamp corresponding to the last video frame of the playback video corresponding to the current time period, pre-load the condensed video corresponding to the next time period; The current time period is a sub-time period including a current timestamp of the playback time axis, and the next time period is a next sub-time period of the current time period in the dragging direction of the dragging operation.

7. The method according to any one of claims 1 to 5, characterized in that: Before the step of extracting frames of the playback video according to the dragging speed of the dragging operation to determine the playback video frames, the method further includes: Determine whether there is a frame analysis result of the playback video corresponding to the current time period, wherein the current time period is a sub-time period including a timestamp corresponding to the current position of the playback timeline, and the frame analysis result includes the timestamp; If there is no frame analysis result of the playback video corresponding to the current time period, determining whether the playback video corresponding to the current time period has been downloaded; If the playback video corresponding to the current time period has not been downloaded, download the condensed video corresponding to the current time period, and perform frame analysis on the downloaded condensed video to obtain a frame analysis result; If a frame analysis result corresponding to the current time period already exists, the step of determining a playback video frame from the playback video according to the dragging speed of the dragging operation is performed.

8. A video playback device, characterized in that: The device comprises: A playback video frame determination module is used to, in the process of video playback, in response to a user's dragging operation on the playback time axis of the currently playing playback video, extract frames of the playback video according to the dragging speed of the dragging operation to determine the playback video frame, wherein the faster the dragging speed, the longer the time interval between two adjacent video frames in the playback video frame; A playback video frame playing module, used to play the determined playback video frame following the dragging operation; The device also includes: A parameter determination module, used to determine the current condensed video duration and the current frame extraction interval corresponding to the current time axis zoom level according to the correspondence between the preset time axis zoom level and the condensed video duration and the frame extraction interval; A time segmentation module, used to segment the total time segment corresponding to the original video into multiple sub-time segments, wherein the duration of each sub-time segment is the duration of the current condensed video; The video download module is used to download the original video corresponding to each sub-time period, and extract frames of the original video corresponding to each sub-time period according to the current frame extraction interval during the download process to obtain a playback video; The device also includes: A frame extraction interval determination module is used to determine whether the frame extraction intervals corresponding to the time axis zoom levels before and after the adjustment are the same according to the corresponding relationship in response to a zoom level adjustment instruction issued by a user based on the playback time axis during the process of downloading the original video corresponding to each sub-time period; A return module is used to, if the frame extraction intervals corresponding to the timeline zoom levels before and after adjustment are different, use the adjusted timeline zoom level as the current timeline zoom level, and return the step of determining the current condensed video duration and the current frame extraction interval corresponding to the current timeline zoom level based on the correspondence between the preset timeline zoom level, the condensed video duration and the frame extraction interval.

9. The device according to claim 8, characterized in that The playback video frame determination module includes: The first playback video frame determination submodule is used to determine the time interval between the timestamp corresponding to the drag operation of the current callback on the playback timeline and the timestamp of the previous playback video frame; if the time interval is not less than the full frame rate interval, determine from the playback video the video frame corresponding to the timestamp of the current callback as the current playback video frame; The second playback video frame determination submodule is used to determine, from the playback video, a video frame corresponding to a timestamp included in the playback timeline portion through which the drag operation passes; extract frames from the determined video frame to obtain a playback video frame, so that the time interval between adjacent video frames in the playback video frame is not less than the full frame rate interval; Wherein, the full frame rate interval is the time interval between adjacent video frames when playing the video at full frame rate; The first playback video frame determination submodule includes: A timestamp acquisition unit, used to acquire the timestamp corresponding to the drag operation of the current callback on the playback timeline from a pre-established serial playback queue, wherein the serial playback queue is used to record the timestamp corresponding to the drag operation of the callback on the playback timeline; a time interval determination unit, used to determine the time interval between the acquired timestamp and the timestamp of the last playback video frame; a current playback video frame determination unit, configured to determine, if the time interval is not less than the full frame rate interval, a video frame corresponding to the acquired timestamp from the playback video according to the timestamps of each playback video frame included in the cached frame analysis results of the playback video frames, as the current playback video frame; The video download module includes: A download task queue construction submodule, used to construct a download task queue of condensed videos corresponding to the multiple sub-time periods, wherein the download task queue includes download tasks of condensed videos corresponding to each sub-time period, and the order of the download tasks is determined based on the time interval between the sub-time period corresponding to the download task and the current time; A download task execution submodule is used to execute the download tasks in the download task queue, and in the process of executing each download task, extract frames of the original video corresponding to each sub-time period of the download according to the current frame extraction interval to obtain a condensed video corresponding to each sub-time period; The device also includes: A first order determination module is used for, in the process of downloading the original video corresponding to each sub-time period, in response to a drag operation issued by a user based on a playback timeline, adjusting, in a download task queue, a download task corresponding to a sub-time period containing a timestamp corresponding to a current position of the drag operation on the playback timeline to be the first task other than the download task being executed; A second order determination module, configured to adjust the remaining download tasks based on a time interval between a sub-time period corresponding to a download task and a timestamp corresponding to a current position of the drag operation on the playback timeline; The device also includes: A condensed video preloading module, configured to preload a condensed video corresponding to a next time period when it is detected that a timestamp corresponding to a current position of the dragging operation on the playback timeline is close to a timestamp corresponding to a last video frame of the playback video corresponding to the current time period; The current time period is a sub-time period including a current timestamp of the playback time axis, and the next time period is a next sub-time period of the current time period in the dragging direction of the dragging operation; The device also includes: A frame analysis result determination module, used to determine whether there is a frame analysis result of the playback video corresponding to the current time period, wherein the current time period is a sub-time period including a timestamp corresponding to the current position of the playback time axis, and the frame analysis result includes a timestamp; A video download determination module, used to determine whether the playback video corresponding to the current time period has been downloaded if there is no frame analysis result of the playback video corresponding to the current time period; A frame analysis module, configured to download a condensed video corresponding to the current time period if the playback video corresponding to the current time period has not been downloaded, and perform frame analysis on the downloaded condensed video to obtain a frame analysis result; An execution module is used to execute the step of determining a playback video frame from the playback video according to the dragging speed of the dragging operation if a frame analysis result corresponding to the current time period already exists.

10. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-7 when executing a program stored in a memory.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images