Video playing method, electronic equipment, storage medium and chip

Abstract

The invention provides a video playing method, electronic equipment, a storage medium and a chip, and relates to the technical field of terminal equipment, and the method comprises the steps: decoding a video file to obtain an I frame, a P frame and a B frame in a common playing process of a video, and recoding the P frame and the B frame into the I frame; then, caching an I frame of the video file and an I frame obtained by recoding; in the frame playing scene, according to the detected frame playing moment, the I frame corresponding to the moment is obtained from the cache, and then the obtained I frame is decoded, so that decoding of grouped video frames is avoided, and the fluency of frame playing is improved; besides, under the condition that the I frame corresponding to the frame playing moment is not cached, the video frame group where the I frame corresponding to the frame playing moment is located can be decoded through the idle decoder in the at least two decoders, and due to the fact that decoding is carried out through the idle decoder, the decoding efficiency is improved, and the fluency of frame playing can also be improved.

Description

Technical Field

[0001] This application relates to the technical field of terminal devices, and in particular, to a video playing method, an electronic device, a storage medium, and a chip. Background Art

[0002] The gallery application in an electronic device can provide a picture preview interface, and a user can view each picture stored in the electronic device through the picture preview interface of the gallery application; the gallery application also provides a video playing interface, and a user can play each video stored in the electronic device through the video playing interface in the gallery application.

[0003] The video playing interface can also display the playing progress of the video. For example, the playing progress of the video can be displayed in ways such as a progress bar, a progress ring, a progress display frame, etc.; the dragging operation of the user on the progress bar, the progress ring, or the progress display frame can trigger the electronic device to perform frame playing on the video. However, in the frame playing scenario of the video, the display process of the video picture is relatively stuck and not smooth. Summary of the Invention

[0004] This application provides a video playing method, an electronic device, a storage medium, and a chip, which can reduce the stuck situation of the video picture in the frame playing scenario of the video and improve the smoothness of the video picture in the frame playing scenario.

[0005] To achieve the above object, the first aspect of this application adopts the following technical solution:

[0006] The first aspect of this application provides a video playing method, including:

[0007] An electronic device displays a first interface of a first video, the first interface includes a first area and a first control, and a second control is included in the first area;

[0008] After the electronic device receives a first operation on the second control, it displays a first segment of video frames obtained by decoding the first encoded data of the first video in the first area;

[0009] During the process of displaying the first segment of video frames in the first area, the electronic device controls the first control to move from a first position to a second position along a first trajectory;

[0010] When the first control moves to the second position, the electronic device receives a first dragging operation on the first control;

[0011] During the process of moving the first control from the second position to the third position through the first dragging operation, the electronic device displays a video frame obtained by decoding the second encoded data of the first video in the first area. The third position is on the first trajectory and is located between the first position and the second position. The efficiency of the electronic device in decoding the second encoded data is higher than that in decoding the first encoded data.

[0012] In this application, during the process of displaying the video frame of the first video triggered by the first operation of the second control, the video frame to be displayed can be obtained by decoding the first encoded data of the first video; during the process of displaying the video frame of the first video triggered by the dragging operation (denoted as frame playback), the video frame to be displayed is obtained by decoding the second encoded data; since the efficiency of the electronic device in decoding the second encoded data is higher than that in decoding the first encoded data, the decoding efficiency in the frame playback scenario can be improved, thereby improving the smoothness of the video picture in the frame playback scenario.

[0013] As another implementation manner of the first aspect, during the process of displaying the first segment of video frames in the first area, the method further includes:

[0014] The electronic device caches the second encoded data of the first segment of video frames.

[0015] During the process of displaying the first segment of video frames obtained by decoding the first encoded data of the first video in the playback order triggered by the second control, the second encoded data of the first segment of video frames can be cached, and the decoding efficiency of the second encoded data is higher than that of the first encoded data; during the subsequent process of displaying video frames through the first dragging operation on the first control, if the second encoded data of the video frame to be displayed has been cached, the second encoded data can be directly decoded to display the decoded video frame. Since the decoding efficiency of the second encoded data is higher than the first decoding efficiency, the video frames are displayed more quickly during subsequent playback, improving the smoothness of the video picture in the frame playback scenario.

[0016] As an example, some of the encoded data in the first encoded data need to refer to other encoded data during decoding, resulting in low decoding efficiency; each encoded data in the second encoded data does not need to refer to other encoded data during decoding, resulting in high decoding efficiency.

[0017] As another example, some of the encoded data in the first encoded data need to refer to other encoded data during decoding. For example, the decoding of P frames and I frames needs to refer to the encoded data of I frames; the second encoded data can also be the respective YUV data obtained by decoding the first encoded data.

[0018] As an implementation manner of the first aspect, the frame types in the first encoded data include key frames and non-key frames, and the frame types in the second encoded data are all key frames.

[0019] In the present application, key frames are I frames, and non-key frames are P frames and B frames. Since the I frames, P frames, and B frames in the first coded data are all re-encoded as I frames, the I frames corresponding to each time stamp can be stored; therefore, each I frame can be decoded individually during decoding, without the need to decode video frames in groups, so the decoding speed can be increased, and the smoothness of the video picture of the frame playback scene can be improved.

[0020] As another implementation of the first aspect, the method further includes:

[0021] When the first control is dragged to a fourth position, the electronic device obtains the third coded data of the first video, when the first control is at the fourth position, the playback progress of the first video is a first moment, the coded data corresponding to the first moment is fourth coded data in the third coded data, the frame type in the third coded data is the same as the frame type in the first coded data, and the fourth position is not on the first track;

[0022] The electronic device decodes the third encoded data by a first decoder;

[0023] After the electronic device completes decoding of the fourth encoded data through the first decoder, the electronic device displays a video frame obtained by decoding the fourth encoded data in the first area.

[0024] In the present application, the fourth position is not on the first track. When the first control is in the fourth position, the playback progress is the first moment, which means that the video frame corresponding to the first moment is not cached. Therefore, it is necessary to obtain the third encoded data corresponding to the video frame group at the first moment, and then decode the third encoded data to obtain the fourth encoded data corresponding to the first moment. The frame type in the third encoded data is the same as that in the first encoded data. Compared with the process of obtaining the second encoded data from the cache for decoding and displaying, the decoding efficiency of the process of decoding the third encoded data and displaying the fourth decoded data is also low.

[0025] As another implementation of the first aspect, the method further includes:

[0026] In a case where the first control is dragged from the fourth position to the fifth position, the electronic device obtains fifth coded data of the first video, when the first control is at the fifth position, the playback progress of the first video is a second moment, the coded data corresponding to the second moment is sixth coded data in the fifth coded data, the frame type in the fifth coded data is the same as the frame type in the first coded data, and the first video frame in the fifth coded data and the last video frame in the third coded data are adjacent video frames;

[0027] The electronic device decodes the fifth coded data by a second decoder;

[0028] After the electronic device completes decoding of the fifth encoded data through the second decoder, the electronic device displays the video frame obtained by decoding the sixth encoded data in the first area.

[0029] In the present application, in order to improve the decoding efficiency, at least two decoders are set for frame playback, and two consecutive groups of video frames are decoded by different decoders. Even if the first decoder has not completed decoding the first group of video frames (for example, the third encoded data), it does not affect the second decoder to decode and display the second group of video frames (for example, the fifth encoded data) in a timely manner, thereby improving the smoothness of the video screen in the frame playback scene.

[0030] As another implementation of the first aspect, in a process of displaying, in the first area, a first video frame obtained by decoding first coded data of the first video in a playback order, the process includes:

[0031] For each first coded data, the electronic device decodes the first coded data through a third decoder to obtain YUV data;

[0032] The electronic device displays, in the first area, a video frame corresponding to the YUV data obtained by decoding the first encoded data.

[0033] As another implementation of the first aspect, the process in which the electronic device caches the second encoded data of the first video frame includes:

[0034] For each first coded data, the electronic device encodes the YUV data obtained from the first coded data to obtain second coded data corresponding to the first coded data;

[0035] The electronic device caches second coded data corresponding to the first coded data and a timestamp of the first coded data in a first storage space.

[0036] In the present application, since YUV data needs to be decoded during the previous normal playback process, the decoded YUV data is re-encoded, and each re-encoded encoded data and timestamp are cached; no additional decoding operation is required, thereby improving efficiency.

[0037] As another implementation of the first aspect, the first coded data includes coded data with a timestamp of a third moment, and before the electronic device encodes the YUV data obtained from the first coded data at the third moment, the method further includes:

[0038] The electronic device determines that the second encoded data of the third moment of the first video does not exist in the first storage space.

[0039] In the present application, in order to avoid repeated caching, the re-encoding and caching operation can be performed when the second encoded data of the first video at the third moment does not exist in the first storage space.

[0040] As another implementation of the first aspect, in a process of dragging the first control to a third position through the first drag operation, the electronic device displays a video frame obtained by decoding the second encoded data in the first area, including:

[0041] When the first control is dragged to the sixth position through the first drag operation, the electronic device displays a video frame obtained by decoding the seventh encoded data in the second encoded data in the first area, and when the first control is at the sixth position, the playback progress of the first video is the fourth moment, the seventh encoded data is the encoded data corresponding to the fourth moment, and the sixth position is on the first trajectory.

[0042] As another implementation of the first aspect, when the first control is dragged to a sixth position through the first drag operation, the electronic device displays, in the first area, a video frame obtained by decoding seventh coded data in the second coded data, including:

[0043] When the first control is dragged to the sixth position through the first drag operation, the electronic device determines that the playback progress of the first video is the fourth moment;

[0044] The electronic device searches the first storage space to see whether the seventh coded data corresponding to the fourth moment is stored;

[0045] The electronic device searches the first storage space for the seventh coded data;

[0046] The electronic device acquires the seventh coded data from the first storage space;

[0047] The electronic device decodes the seventh encoded data;

[0048] The electronic device displays, in the first area, a video frame obtained by decoding the seventh encoded data.

[0049] In this application, before decoding and displaying the cached encoded data, it is first necessary to determine whether the first storage space stores the video frame to be displayed, so that the corresponding video frame can be successfully obtained from the first storage space.

[0050] As another implementation of the first aspect, when the first control is dragged to the fourth position by the first drag operation, the electronic device obtains the third encoded data of the first video, including:

[0051] When the first control is dragged to the fourth position by the first drag operation, the electronic device determines that the playback progress of the first video is the first moment;

[0052] The electronic device does not query the encoded data corresponding to the first moment in the first storage space;

[0053] The electronic device obtains the third encoded data of the first moment of the first video.

[0054] In this application, during frame playback, when the encoded data corresponding to the frame playback moment is not queried in the first storage space, decoding is performed in groups by the first encoder or the second encoder to improve the smoothness of frame playback through frame concurrent decoding.

[0055] As another implementation of the first aspect, the encoded data corresponding to the first moment is the encoded data whose difference between the timestamp and the first moment is less than or equal to the first value, and the first value is half of the difference between the timestamps of two adjacent video frames.

[0056] In this application, after determining a certain moment according to the control position, this moment may be a moment between the playback moments corresponding to two adjacent video frames, that is, there is no video frame with a timestamp of this moment. Therefore, one of the video frames corresponding to the two timestamps adjacent to this moment can be used as the video frame corresponding to this moment. Of course, the first value (half of the difference between the timestamps of two adjacent video frames) can also be used as a reference, and the encoded data with a difference between the timestamp and this moment less than the first value can be used as the encoded data corresponding to this moment.

[0057] As another implementation of the first aspect, the first interface further includes a second area, and the first control is located within the second area and can move within the second area;

[0058] The first region and the second region do not overlap, or the second region is within the first region.

[0059] In a second aspect, an electronic device is provided, including a processor configured to call a computer program stored in a memory to implement the method according to any one of the first aspects of this application.

[0060] In a third aspect, a chip is provided, including a processor coupled to a memory, and the processor executes a computer program stored in the memory to enable an electronic device to implement the method according to any one of the first aspects of this application.

[0061] In a fourth aspect, a computer-readable storage medium is provided, storing a computer program, and when the computer instructions run on an electronic device, the electronic device is enabled to implement the method according to any one of the first aspects of this application.

[0062] In a fifth aspect, an embodiment of this application provides a computer program product, and when the computer program product runs on a device, the electronic device is enabled to execute the method according to any one of the first aspects of this application.

[0063] It can be understood that the beneficial effects of the above second to fifth aspects can be referred to the relevant descriptions in the above first aspect, and will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 It is a schematic diagram of a hardware structure of an electronic device provided by an embodiment of this application;

[0065] Figure 2 It is a schematic diagram of an interface during a video playback process provided by an embodiment of this application;

[0066] Figure 3 It is another schematic diagram of an interface during a video playback process provided by an embodiment of this application;

[0067] Figure 4 It is a schematic diagram of a video playback interface provided by an embodiment of this application;

[0068] Figure 5 It is another schematic diagram of an interface during a video playback process provided by an embodiment of this application;

[0069] Figure 6 It is a schematic diagram of the playback order and decoding order of a set of video frames provided by an embodiment of this application;

[0070] Figure 7 It is a schematic diagram of the playback order and decoding order after re-encoding a video frame into an I frame provided by an embodiment of this application;

[0071] Figure 8Schematic diagram of the frame playback process of the video provided by the embodiment of the present application;

[0072] Figure 9 Schematic diagram of the technical architecture of the video playback provided by the embodiment of the present application;

[0073] Figure 10 Timing diagram of the preparation work before normal playback and frame playback provided by the embodiment of the present application;

[0074] Figure 11 Timing diagram of the normal playback process of a video provided by the embodiment of the present application;

[0075] Figure 12 Timing diagram of the frame playback process of a video provided by the embodiment of the present application;

[0076] Figure 13 Timing diagram of the frame playback process of another video provided by the embodiment of the present application;

[0077] Figure 14 Timing diagram of ending frame playback and continuing normal playback provided by the embodiment of the present application;

[0078] Figure 15 Timing diagram of exiting the video playback interface provided by the embodiment of the present application. Detailed implementation manners

[0079] In the following description, specific details such as specific system architectures and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details.

[0080] It should be understood that when used in the specification of the present application and the appended claims, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0081] It should also be understood that in the embodiments of the present application, "one or more" means one, two, or more than two; " / ", which describes the association relationship of associated objects, indicates that three relationships can exist; for example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after.

[0082] In addition, in the description of the specification and the appended claims of the present application, terms such as "first", "second", "third", "fourth", etc. are only used for distinguishing descriptions and should not be construed as indicating or implying relative importance.

[0083] The reference to "one embodiment" or "some embodiments" etc. described in the specification of the present application means that in one or more embodiments of the present application, specific features, structures or characteristics described in connection with that embodiment are included. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments" etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "comprising", "including", "having" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0084] A video playback method provided by an embodiment of the present application can be applied to an electronic device, and the electronic device can be an electronic device such as a tablet computer, a mobile phone, a wearable device, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc. The specific type of the electronic device is not limited in the embodiments of the present application.

[0085] Figure 1 A schematic structural diagram of an electronic device is shown. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 1211, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone interface 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. Among them, the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0086] It can be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than those illustrated, or combine certain components, or split certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0087] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors. For example, the processor 110 is used to execute the video playback method in the embodiments of the present application.

[0088] A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory may store the instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can directly call it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0089] The internal memory 1211 can be used to store computer-executable program codes, and the executable program codes include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 1211. The internal memory 1211 can include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function (such as an image playback function, etc.). The touch sensor 180K, also known as the "touch panel". The touch sensor 180K can be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also known as a "touch control screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In some other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 100, at a different position from the display screen 194. For example, click operations on controls provided in the embodiments of the present application, drag operations on progress display frames, etc. can all be recognized by the touch sensor.

[0090] The electronic device 100 realizes the display function through the GPU, the display screen 194, and the application processor, etc. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 can include one or more GPUs, which execute program instructions to generate or change display information. For example, the process of rendering YUV data in the embodiments of the present application can be realized through the GPU.

[0091] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can adopt a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light-emitting diode or an active matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 can include 1 or N display screens 194, where N is a positive integer greater than 1. For example, in the embodiments of the present application Figure 2 or Figure 3The interfaces shown are all displayed by the display.

[0092] The video codec is used to compress or decompress digital videos. The electronic device 100 can support one or more video codecs. In this way, the electronic device 100 can play or record videos in multiple encoding formats. For example, the video codec can implement decoding of video frames in the H.264 format provided in the embodiments of the present application, or re-encode them into H.264 format video frames that are all key frames.

[0093] The embodiments of the present application do not particularly limit the specific structure of the execution subject of a video playback method. As long as it can communicate according to a video playback method provided in the embodiments of the present application by running the code recording a video playback method of the embodiments of the present application. For example, the execution subject of a video playback method provided in the embodiments of the present application can be a functional module in the electronic device that can call and execute a program, or a communication device applied to the electronic device, such as a chip.

[0094] The user can view each picture stored in the electronic device and play each video stored in the electronic device through the gallery application in the electronic device.

[0095] Refer to Figure 2 , which is a schematic diagram of the interface for playing a video stored in the electronic device through the gallery application provided in the embodiments of the present application.

[0096] Refer to Figure 2 In (a) of, it is a grid diagram of pictures and videos stored in the electronic device shown by the gallery application of the electronic device; this grid diagram can display pictures and videos. Among them, the cover and duration of the video are displayed in the grid where the video is located, and the thumbnail of the picture is displayed in the grid where the picture is located.

[0097] This grid diagram can be the interface diagram displayed by the gallery application after the user clicks the icon of the gallery application on the system desktop to open the gallery application, or it can also be the interface diagram displayed by the gallery application triggered through more steps of operations after the user clicks the icon of the gallery application on the system desktop to open the gallery application.

[0098] When this grid diagram displays the pictures and / or videos stored in the electronic device, it can be presented in multiple ways. For example, Figure 2 The way shown in (a) of is to display the pictures and videos in the photo group in reverse chronological order; in practical applications, the gallery application can also set multiple object groups. For example, through Figure 2The control 10 shown in (a) in [description] can view multiple groups in the photo album: image group, video group, screenshot and screen recording group, etc. Users can also display pictures through the image group in the gallery application, display videos through the video group in the gallery application, and display screenshot pictures and screen recording videos through the screenshot and screen recording group in the gallery application. Each group can also display the corresponding pictures and / or videos in the form of the grid diagram shown above.

[0099] The embodiments of the present application do not limit the number of steps of user operations required to start from the system desktop to display the grid diagram of the gallery application. Similarly, the embodiments also do not limit the way of displaying pictures and videos in the grid diagram.

[0100] Refer to Figure 2 in (b), after the user clicks on the cover of video A in the grid diagram shown in (a) in [description], the video playback interface of video A displayed by the electronic device in response to this operation. This video playback interface includes a video playback area 11 and a progress display area 12. Figure 2 in (a), after the user clicks on the cover of video A in the grid diagram shown in (a) in [description], the video playback interface of video A displayed by the electronic device in response to this operation. This video playback interface includes a video playback area 11 and a progress display area 12.

[0101] Among them, the video playback area 11 is used to display the video images during the video playback. After entering the video playback interface from the grid diagram interface, a control 111 is displayed in the video playback area. When the user clicks on this control 111, the electronic device responds to the click operation on this control 111 and starts playing the video images from the 0th second in this video playback area 11; in the video pause playback state, the control 111 is also displayed in the video playback area 11. The user can click on this control 111, and the electronic device responds to the click operation on this control 111, and the video playback area 11 starts playing the video images from the current paused moment.

[0102] The progress display area 12 is used to display the video playback progress. The progress display area 12 includes a progress display frame 121 and a progress indicator line 122. The position of the progress indicator line 122 can remain unchanged. The user can drag the progress display frame 121 located below the progress indicator line 122 so that the progress display frame 121 moves left and right without the progress indicator line 122 moving; the position of the progress indicator line 122 above the progress display frame on the progress display frame represents the current video playback progress.

[0103] As an example, the total playing duration of the current video is 15 seconds, and the total length of the progress display frame of the current video is the length represented by L pixels. Based on these two parameters, the playing duration corresponding to a unit length (or the length corresponding to a unit time) can be obtained. Therefore, if the leftmost edge position of the progress display frame (the position corresponding to the first pixel) is set to correspond to the 0th second of the video, different positions of the progress indicator line on the progress display frame correspond to different playing moments. For ease of subsequent description, the embodiments of the present application describe it in the form of "the moment corresponding to the progress indicator line" or "the moment pointed to by the progress indicator line".

[0104] Due to the space limitation of the progress display area 12, only some video frames in the progress display frame 121 or some regions of some video frames can be displayed in the progress display area 12. As Figure 2 shown in (b) of, the progress display frame 121 displayed in the progress display area 12 includes: thumbnails of 3 complete video frames and a partial region of the thumbnail of a video frame.

[0105] Generally, the picture played in the video playing area 11 is the picture of the video frame at the moment pointed to by the progress indicator line 122 in the progress display area 12.

[0106] In a specific implementation, there is a situation where the moment pointed to by the progress indicator line 122 is the moment between the playing moments corresponding to two adjacent video frames, that is, there is no video frame in the video file corresponding to the moment pointed to by the progress indicator line 122.

[0107] As an example, among the video frames arranged in ascending order of playing moment (denoted as timestamp), the playing moment of the i-th video frame is ti, and the playing moment of the (i + 1)-th video frame is t(i + 1); the moment t pointed to by the progress indicator line 122 is greater than ti and less than t(i + 1). In this case, the video frame corresponding to the timestamp with the shortest time interval from the moment t pointed to by the progress indicator line 122 can be used as the video frame at the moment pointed to by the progress indicator line 122.

[0108] Of course, in practical applications, if there is no video frame in the video file with the same timestamp as the moment pointed to by the progress indicator line, the previous video frame (the video frame corresponding to ti) of the moment t pointed to by the progress indicator line 122 can also be used as the video frame at the moment pointed to by the progress indicator line 122; or the next video frame (the video frame corresponding to t(i + 1)) of the moment t pointed to by the progress indicator line 122 can be used as the video frame at the moment pointed to by the progress indicator line 122.

[0109] In practical applications, there are many ways to display the playback progress in the progress display area 12. For example, the video playback progress can also be displayed in the form of a progress bar, or in the form of a progress ring. This application provides two playback modes for video playback: normal playback and frame playback. The embodiments of this application only use Figure 2 the progress display area shown in (b) in

[0110] as an example to describe the difference between normal playback and frame playback. Figure 2 Referring to (c) in Figure 2 after the user clicks the control 111 in the interface shown in (b) in Figure 2 a screenshot during the process when the electronic device plays video A in response to this operation. This screenshot shows that it has played to the 5th second of video A, and the video frame corresponding to the 5th second is displayed in the video playback area 11; at the same time, the progress display frame 121 in the progress display area 12 moves its position so that the moment corresponding to the progress indicator line 122 is the 5th second. Among them, the playback process from Figure 2 the (b) in

[0111] to the (c) in

[0112] is the normal playback process of the video.

[0113] Combined with Figure 2 the (b) to Figure 2In (c), it shows the normal playing process of Video A: starting from the video frame at the 0th second of Video A, each video frame is displayed in sequence according to the playing moment represented by the timestamp; Figure 2 In (c) is a schematic diagram of the interface showing the video frame at the 5th second. Refer to Figure 2 In (d), the user drags Figure 2 the progress display frame of the interface shown in (c).

[0114] In response to Figure 2 the drag operation shown in (d), the electronic device displays a picture corresponding to the drag operation in the video playing area.

[0115] As an example, refer to Figure 2 In (e), when the user drags the progress display frame so that the time pointed to by the progress indicator line is the 14th second, the video frame displayed in the video playing area is the video frame corresponding to the 14th second. The video frame corresponding to the 14th second can be the video frame with a timestamp of the 14th second in Video A; it can also be the video frame with the closest timestamp to the 14th second; it can also be a video frame with the closest time interval to the 14th second before the 14th second; or a video frame with the closest time interval to the 14th second after the 14th second.

[0116] It should be noted that in subsequent embodiments, the specific meaning of the description method of "the video frame corresponding to a certain moment" can refer to the detailed explanation of "the video frame corresponding to the 14th second", and will not be repeated hereinafter. From Figure 2 In (d) to Figure 2 the playing process shown in (e) is the frame playing process of the video. Generally, the video frame corresponding to a certain moment is the video frame whose timestamp difference from that moment is less than or equal to the first value (half of the difference between the timestamps of two adjacent video frames).

[0117] During the process of the user dragging the progress display frame 121, a relative position is formed between the progress indicator line 122 and the progress display frame 121, and the position of the progress indicator line 122 on the progress display frame 121 is related to the moment pointed to by the progress indicator line 122.

[0118] In the specific implementation process, assume that before the user moves in the area where the progress display frame 121 on the screen is located, the time indicated by the progress indicator line 122 is t1. When the user moves left or right in the area where the progress display frame on the screen is located, the electronic device can detect the real-time contact position of the user at a certain time interval; the electronic device determines the moving direction and moving distance of the user according to the continuous real-time contact positions. The electronic device determines the time variation Δt according to the moving distance of the user; determines the operation method for calculating the real-time time t2 according to the moving direction of the user. For example, if the moving direction is left, the starting time t1 minus the time variation Δt gives the real-time time t2; if the moving direction is right, the starting time t1 plus the time variation Δt gives the real-time time t2. After obtaining the real-time time t2, the electronic device searches for the video frame corresponding to the real-time time t2 and displays the video frame corresponding to t2 in the video playback area;

[0119] On the other hand, the electronic device obtains the moving direction of the progress display frame according to the moving direction determined by the real-time contact position, and determines the moving distance of the progress display frame according to the moving distance of the real-time contact position; the electronic device moves the progress display frame in the progress display area according to the moving direction and moving distance of the progress display frame, so that the time indicated by the progress indicator line is t2. As another example, the electronic device also determines the moving direction (if t2 is greater than t1, move left; if t2 is less than t1, move right) and moving distance of the progress display frame according to the starting time t1 and the real-time time t2; the electronic device moves the progress display frame in the progress display area according to the moving direction and moving distance of the progress display frame, so that the time indicated by the progress indicator line is t2.

[0120] It should be noted that the above process is only an example. In actual applications, there can be multiple ways to determine the time indicated by the progress indicator line 122, the position of the progress display frame 121, and the video frame displayed in the video playback area 11 when the user drags the progress display frame 121.

[0121] Based on the above frame playback process, it can be understood that the process of the user dragging the progress display frame is an uncertain process. It may move left, move right, may be dragged quickly, or may be dragged slowly. Correspondingly, the video frame picture displayed in the video playback area 11 has randomness.

[0122] As an example, before the user moves the area where the progress display frame on the screen is located, the start time indicated by the progress indicator line is t1; during the process of the user dragging the progress display frame, the electronic device detects the real-time contact position of the user at a certain time interval, so as to obtain the real-time times corresponding to multiple consecutive time periods: t2, t3, t7, t4, t3. In the video playback area, the video frames are played in the following order: starting from the video frame corresponding to the t1 moment as the starting video frame, playing the video frame corresponding to the t2 moment, the video frame corresponding to the t3 moment, the video frame corresponding to the t7 moment, the video frame corresponding to the t4 moment, and the video frame corresponding to the t3 moment in sequence. It can be understood that the process of the electronic device playing the video in the video playback area triggered by the user clicking on the control 111 is normal playback; the process of the electronic device displaying the video frame at the moment pointed to by the progress indicator line in the video playback area triggered by the user dragging the progress display frame is frame playback; in addition, during the normal playback process of video A, if the user drags the progress display frame and then releases it from the area where the progress display frame is located, the electronic device continues to play video A in the normal playback mode.

[0123] Figure 2 from (d) to Figure 2 as shown in (e) in Figure 2 on the basis of the interface shown in (c) in

[0124] As another embodiment of the present application, in Figure 2 on the basis of the interface shown in (c) in

[0125] Referring to Figure 3 in (a), this figure corresponds to Figure 2 in (c), and video A is played to the 5th second in the normal playback mode.

[0126] Referring to Figure 3 as shown in (b), when playing video A to the 5th second in the normal playback mode, if the user drags the progress display frame to the right, it will trigger the electronic device to display the video frames before the 5th second of video A in the frame playback mode.

[0127] Referring to Figure 3 as shown in (c), when the user drags the progress display frame so that the time pointed to by the progress indicator line is the 2nd second, the electronic device displays the video frame of the 2nd second of video A in the video playback area.

[0128] Referring to Figure 3As shown in (d), when the user drags the progress display frame to the 2nd second and then lifts the finger to release the progress display frame, the electronic device will start playing Video A in the normal playback mode from the 2nd second.

[0129] In specific implementation, the embodiment of the present application also provides another schematic diagram of the interface for playing the video stored in the electronic device through the gallery application.

[0130] The embodiment of the present application provides a video recording mode: getting multiple results from one recording. That is, when recording a video, photos can also be generated simultaneously (for example, photos automatically captured by the electronic device according to the picture content, or photos manually taken by the user through the photo-taking control on the video recording interface), so as to obtain the photos taken during the video recording process. The recorded video can be associated with the photos taken during the video recording process. Similarly, when viewing the video through the gallery application, in the video playback interface, the photos taken during the video recording process can also be displayed in the progress display area.

[0131] Refer to Figure 4 in (a), and this interface can refer to the description in Figure 2 (a).

[0132] Refer to Figure 4 in (b), which is the video playback interface of Video A displayed by the electronic device in response to the operation when the user clicks on the cover of Video A in the grid diagram shown in Figure 4 (a). This video playback interface is similar to the video playback interface shown in Figure 2 , and the difference lies in the content displayed in the progress display area 12.

[0133] The progress display frame 121 in the progress display area 12 is in a compressed state, and the progress display frame in the compressed state can display two video frames, three video frames, etc. Refer to Figure 2 in (b), the progress display frame 121 in the compressed state includes the thumbnail of the first frame of the video and the thumbnail of the middle frame. Of course, in practical applications, the thumbnails of other frames of the video can also be displayed, such as the thumbnail of the first frame and the thumbnail of the last frame. The thumbnail 123 of the photo obtained during the video recording process is displayed on the right side of the progress display frame 121.

[0134] The user can perform operations (such as clicking or double-clicking, etc.) above the progress display frame to trigger the progress display frame to switch from the compressed state to the expanded state. The progress display frame in the expanded state can include more video frame thumbnails than the progress display frame in the compressed state. The number of video frames displayed in the progress display frame in the expanded state is related to the duration of the video. For example, the longer the duration of the video, the more video frames the progress display frame includes.

[0135] Refer to Figure 4As shown in (c), after the user clicks on the progress display frame 121 in the compressed state shown in (b) of Figure 4 a schematic diagram of the progress display frame in the expanded state shown by the electronic device in response to this operation. Due to space limitations, after the progress display frame is expanded, the photos obtained during the video recording process are hidden.

[0136] Figure 4 The interface shown in (c) of Figure 2 is the same as the interface shown in (b) of Figure 4 Therefore, the normal playback process and the frame playback process based on the interface shown in (c) of Figure 2 can refer to the description from (b) to Figure 2 (e) of

[0137] Refer to Figure 5 , which is a schematic diagram of another interface for playing a video stored in an electronic device through a gallery application described in the embodiments of the present application.

[0138] Figure 5 (a) of Figure 5 and (b) of Figure 4 can refer to the description in (a) of Figure 4 and (b) of

[0139] In Figure 5 the interface shown in (b), the progress display frame is in the compressed state. When the user clicks on the control 111 in the interface shown in Figure 5 (b), the electronic device is triggered to play video A in the normal playback mode.

[0140] Refer to Figure 5 (c), which is an interface diagram during the process of playing video A after the user clicks on the control 111 in the interface shown in Figure 5 (b). This interface diagram corresponds to an interface during the normal playback process when the progress display frame is in the compressed state. This interface diagram indicates that it has played to the 5th second of video A, and the video screen corresponding to the 5th second is displayed in the video playback area; at the same time, the progress display frame in the progress display area moves its position so that the moment corresponding to the progress indicator line is the 5th second. Among them, the playback process from Figure 5 (b) to Figure 5 (c) shown is the normal playback process of the video.

[0141] Refer to Figure 5 (d), where the user drags the progress display frame based on the interface shown in Figure 5 (c).

[0142] Refer to Figure 5 (e), where the electronic device responds to Figure 4In the dragging operation shown in (d), the video frame corresponding to the time when the progress indicator line points in the video playback area is the video frame at the 14th second. Among them, the playback moment corresponding to the progress indicator line in the progress display area is the 14th second, and the video playback area displays the video frame corresponding to the 14th second.

[0143] It should be noted that if the progress display frame is in a compressed state and the user drags the progress display frame, the progress display frame will remain in the compressed state. Of course, the user can also, as Figure 4 shown, first trigger the progress display frame to change from the compressed state to the expanded state through an operation, and then drag the progress display frame in the expanded state, then the progress display frame will remain in the expanded state.

[0144] Of course, in practical applications, Figure 4 and Figure 5 in the scenarios shown, the operation of the user dragging the progress display frame can be a left drag or a right drag, and the present application will not elaborate on this.

[0145] In addition, each operation shown in the embodiments of the present application is an example. In practical applications, other operations can also be set according to specific situations. For example, some click operations can also be double-click operations or other air gestures, etc. Some dragging operations can also be air gestures, etc. Similarly, some operation examples in the subsequent embodiments can also be other operations.

[0146] In the above embodiments, whether it is normal playback or frame playback, before displaying the video frame in the video playback area, the video frame needs to be decoded. Usually, the video frames are decoded in groups. A group of video frames forms a group of pictures (GOP). Each group of video frames includes a key frame (I frame), at least one forward reference frame (P frame), and at least two bidirectional reference frames (B frames).

[0147] Referring to Figure 6 , it is a schematic diagram of the playback order and decoding order of a group of video frames provided by the embodiments of the present application. A group of video frames can also be denoted as a group of pictures (GOP).

[0148] Each group of video frames includes, in the playback order: I frame, B frame, B frame, P frame, B frame, B frame, and P frame. Among them, the I frame, as a key frame, can be decoded to obtain a complete picture, and the decoding speed is relatively slow; the P frame records the changes relative to the previous frame (non-B frame), and the picture of the P frame can be decoded only with the information of the previous frame. Of course, with the information of the previous P frame (which may also be an I frame) and the information of the subsequent P frame, the picture of the B frame can be decoded. The P frame and the B frame can improve the picture smoothness while reducing the data volume of the video file.

[0149] Whether it is an I-frame, a P-frame, or a B-frame, they are all video frames. When decoding a video frame at a certain moment, it is not directly decoding the video frame at that moment, but decoding the video frame group where the video frame at that moment is located. When decoding the video frames within a video frame group, they are decoded one by one in the decoding order.

[0150] As an example, if it is necessary to decode the video frame at time t1, and the video frame at time t1 is the 6th video frame in the playback order in a group of video frames: a B-frame; then the decoding order is:

[0151] The first decoding: the video frame with the playback order of 1: an I-frame;

[0152] The second decoding: the video frame with the playback order of 4: a P-frame;

[0153] The third decoding: the video frame with the playback order of 2: a B-frame;

[0154] The fourth decoding: the video frame with the playback order of 3: a B-frame;

[0155] The fifth decoding: the video frame with the playback order of 7: a P-frame;

[0156] The sixth decoding: the video frame with the playback order of 5: a B-frame;

[0157] The seventh decoding: the video frame with the playback order of 6: a B-frame;

[0158] Therefore, when decoding the video frame at time t1, it is necessary to reach the seventh decoding to decode the video frame at time t1.

[0159] Currently, in the ordinary playback scenario, the video frames in each video frame group are parsed, decoded, rendered, and displayed in sequence according to the playback order of the video frame group. Of course, when decoding the video frames in each video frame group, they are decoded in sequence according to the Figure 6 shown decoding order for each video frame in the video frame group. Usually, when displaying the video frames in the previous video frame group, the video frames in the next video frame group have all been decoded. Therefore, in the ordinary playback scenario, each video frame can be displayed at the playback moment corresponding to the timestamp of the video frame in the video frame group, and the video picture is relatively smooth during the process of playing the video.

[0160] In the frame playback scenario, as mentioned above, the process of the user dragging the progress bar to display frames is an uncertain process. The user may drag the progress bar to the left to display frames, or to the right to display frames. The dragging speed may be fast or slow. Correspondingly, the video frame displayed in the video playback area 11 is random. Therefore, during the process of the user dragging the progress bar to display frames, the electronic device needs to determine the moment pointed to by the progress indicator line based on the real-time detected contact position. Then, it parses, decodes, renders, and displays the video frame at the moment pointed to by the progress indicator line. As mentioned above, before decoding the video frame at this moment, it may be necessary to decode other video frames in the same video frame group to decode the video frame at this moment. Therefore, the frame playback scenario is prone to jitter and unsmoothness.

[0161] In some other applications of the frame playback scenario, in order to quickly decode, render, and display the video frame at the moment (for example, t1) pointed to by the progress indicator line, it is also possible to only decode the first decoded video frame in the video frame group where the video frame at t1 is located, and use this first decoded video frame as the video frame at t1. Of course, when decoding, other frames (such as P frames and B frames) in this video frame group are no longer decoded. However, although this method improves the decoding speed, it sacrifices the non-key frames used to improve smoothness, which will also result in discontinuous and jittery video images during frame playback.

[0162] In view of this, the embodiments of the present application provide a method for frame playback of a video. Since during the normal playback of a video, each video frame (I frame, P frame, and B frame) in each group of video frames needs to be decoded, it is possible to re-encode the P frames and B frames decoded during the normal playback process into I frames during the normal playback of the video. This is equivalent to re-encoding the I frames, P frames, and B frames decoded during the normal playback into a video frame consisting entirely of I frames, and then caching the video frame consisting entirely of I frames. During subsequent frame playback, directly obtain the I frame corresponding to the time stamp from the cache, and then decode and render the I frame corresponding to the time stamp, without the need to additionally decode other video frames, thereby improving the smoothness of the video image during frame playback.

[0163] Of course, in practical applications, if some key frames of the video file have been cached in the cache space, these key frames of the video file can also be obtained from the cache space for decoding and playback during normal playback.

[0164] Refer to Figure 7 As shown, for Figure 6The playback order and decoding order after all the video frames shown are encoded as I-frames. Among them, the playback order remains unchanged; for the decoding order, since each video frame is an I-frame, this key frame corresponding to the moment t1 can be directly decoded, which is equivalent to the decoding order of each video frame being the first, improving the decoding speed during frame playback, and the video picture during frame playback is relatively smooth.

[0165] In the embodiments of the present application, this frame playback method is denoted as frame buffer playback; it can be understood that the implementation of frame buffer playback requires prior normal playback to cache the re-encoded I-frames of the same video file in the cache space.

[0166] However, in practical applications, when playing a video frame at a certain moment during frame playback, the cache space may not cache the I-frame at that moment; for example, normal playback has not been performed before frame playback; or although normal playback has been performed before playing a video frame at a certain moment during frame playback, only a section of the video is played during the normal playback process, and the video frame at that moment has not been played in the normal playback mode. For example, in Figure 2 from (b) to Figure 2 in (c), the video frames from the 0th second to the 5th second are played in the normal playback mode, so the key frames from the 0th second to the 5th second are cached in the cache space. But in Figure 2 from (d) to Figure 2 in (e) of [], some video frames between the 5th second and the 14th second need to be displayed in the frame playback mode, and the key frames between the 5th second and the 14th second of this video file are not cached in the cache space.

[0167] In view of this, the embodiments of the present application also provide another frame playback method. The electronic device sets at least two decoders independent of the normal playback process for the frame playback process. When the cache space does not cache the video frame at a certain moment pointed to by the frame playback scenario, the idle decoder among the at least two set decoders is used to decode the video frame group where the video frame at that moment is located, avoiding insufficient decoding ability from affecting the smoothness of the picture. In the embodiments of the present application, this frame playback method is denoted as frame concurrent decoding playback.

[0168] Referring to Figure 8 , it is a schematic flowchart of the frame playback method of the video provided by the embodiments of the present application. This flowchart involves normal playback, frame buffer playback, and frame concurrent decoding playback.

[0169] During normal playback, the video file is parsed to obtain a video stream and an audio stream; the audio stream is decoded to obtain audio frames; the video stream is decoded to obtain video frames; the decoded video frames and audio frames are time-synchronized; the decoded video frames are video-rendered; the decoded audio frames are audio-rendered. Of course, at the end, audio playback and video display are also required.

[0170] During normal playback, after the video is decoded to obtain video frames (including I-frames, P-frames, and B-frames), the P-frames and B-frames among them are re-encoded into I-frames; the original I-frames and the re-encoded I-frames are cached.

[0171] During frame playback, first, the moment pointed to by the progress indicator line is determined according to the drag operation; it is checked whether there is a video frame corresponding to this moment in the cache space. Among them, the specific meaning of the video frame corresponding to a certain moment can refer to the explanation in the above "video frame corresponding to the 14th second".

[0172] If there is a video frame corresponding to this moment in the cache space, the frame cache playback process is executed: the video frame corresponding to this moment is obtained from the cache space; the video frame corresponding to this moment is decoded; after decoding, the video frame is rendered and subsequently displayed.

[0173] If there is no video frame corresponding to this moment in the cache space, the frame concurrent decoding playback process is executed: the video file is parsed, and it is determined by scheduling to use the idle decoder in GOP decoder one or GOP decoder two for decoding; then the determined decoder is used for decoding; then the video frame corresponding to this moment is obtained from the corresponding decoder by scheduling; finally, the video frame is rendered and subsequently displayed.

[0174] For the detailed processes of the above normal playback, frame cache playback, and frame concurrent decoding playback, reference can be made to the descriptions in the subsequent embodiments, which will not be elaborated here. Refer to Figure 9 , which is the technical architecture diagram of the playback engine corresponding to the video frame playback method provided in the embodiments of the present application.

[0175] There are multiple applications in the application layer of the electronic device, such as a gallery application, a file management application, a video playback application, an audio playback application, etc. The embodiments of the present application only list some applications related to audio and video playback. Of course, in actual applications, only some of the applications listed in the application layer may be used, and other applications not listed in the application layer may also be used to implement the video playback process.

[0176] There is a basic middle platform in the application framework layer, and there is a unified playback engine in this basic middle platform. This unified playback engine is used to provide a basis for upper-layer applications to implement audio and video playback.

[0177] The unified playback engine has an interface layer, which is used to transfer data between upper-layer applications and other modules in the unified playback engine.

[0178] The ordinary player is used to implement the ordinary playback process described in the above embodiments. The ordinary player includes: a basic interface module, an ordinary playback module, an audio rendering module, and a video rendering module.

[0179] The basic interface module is used to provide a basic interface for the video playback process. This basic interface can be used by video playback applications without their own interfaces. In the embodiments of the present application, the gallery application can not use this basic interface, but use the video playback interface provided by the gallery application itself;

[0180] The audio rendering module is used to call the audio rendering engine to render the audio stream in the video file during the ordinary playback process;

[0181] The video rendering module is used to call the graphics rendering engine to render the video stream in the video file during the ordinary playback process;

[0182] The ordinary playback module is used to execute the playback of audio and video files and call other modules (such as the codec module) to implement the ordinary playback process, etc. In practical applications, the audio rendering module and the video rendering module can also be set inside the ordinary playback module.

[0183] The frame player is used to implement the frame playback process. The frame player provides two playback modes: the frame buffer playback mode and the frame concurrent decoding playback mode.

[0184] The frame buffer playback mode involves a frame extraction module and a cache cleaning module; the frame extraction module is used to implement the extraction of non-key frames for re-encoding and caching the extracted non-key frames; the cache cleaning module is used to clean the relevant information (such as linked lists) in the cache at an appropriate time;

[0185] The frame concurrent decoding playback mode involves a GOP parser and a GOP decoder, which are used to implement the parsing of video files and the decoding of video frames.

[0186] The decision module can be a module independent of the ordinary player and the frame player, or can be a module included in the frame player, and is used to decide whether to perform frame concurrent decoding playback or frame buffer playback. In the embodiments of the present application, the decision module is taken as an example in the frame player.

[0187] The media parsing module includes: a metadata module and a thumbnail generation module; the metadata module is used to parse the video duration; the thumbnail generation module is used to generate multiple thumbnails as progress display frames according to the video duration in a multi-path scenario;

[0188] The data extraction module includes: a subtitle extraction module, a video data extraction module, and an audio data extraction module. The subtitle extraction module is used to extract subtitle data of a video file; the video data extraction module is used to extract video stream data of a video file, and the audio data extraction module is used to extract audio stream data of a video file.

[0189] The encoding and decoding module includes: a video encoding module, a video decoding module, an audio decoding module, and a software and hardware adaptation module.

[0190] Among them, the video encoding module, the video decoding module, and the audio decoding module can all be called by a common player and a frame player, and simultaneously call the encoding and decoding engine below to implement corresponding functions.

[0191] The software and hardware automatic adaptation module is used to implement corresponding functions using software modules when hardware resources are limited.

[0192] The application framework layer also includes an audio rendering engine, a graphics rendering engine, and an encoding and decoding engine;

[0193] The audio rendering engine is used to be called to provide audio rendering; the graphics rendering engine is used to be called to provide screen rendering; the encoding and decoding engine is used to be called to decode and encode video frames.

[0194] Refer to Figure 10 , which is a timing diagram of the preparation process before normal playback and frame playback of the electronic device provided in the embodiment of the present application. This timing diagram can be corresponding to (a) in Figure 2 and (b) in Figure 2 . Of course, video A in this example can be the video corresponding to "one recording, multiple uses", or it can not be the video corresponding to "one recording, multiple uses".

[0195] S101, the gallery application displays a grid view, and the grid view includes the cover of video A.

[0196] The grid view displayed by the gallery application can not only include the cover of video A, but also include the covers of other videos, and of course, can also include thumbnail images of pictures. Figure 2 The grid view shown in the interface shown in (a) in

[0197] S102, the gallery application receives a click operation on the cover of video A.

[0198] In the embodiment of the present application, when the electronic device displays the interface shown in (a) in Figure 2 , the user clicks on the cover of any video shown in the grid view in the interface shown in (a) in Figure 2 . In response to this operation, the electronic device displays the video playback interface of this video through the gallery application. This click operation can refer toFigure 2 The click operation of the user's finger in the interface shown in (a) of

[0199] It should be noted that in actual applications, after other modules of the electronic device receive the click operation, they may transmit the click operation to the gallery application. For the sake of simplicity of description, the intermediate steps are omitted in the embodiments of the present application. Taking the gallery application receiving the click operation on the cover of Video A as an example.

[0200] S103, after the gallery application receives the click operation on the cover of Video A, it displays the video playback interface of Video A.

[0201] The video playback interface of Video A can refer to Figure 2 the interface shown in (b) of , and the video playback interface of Video A includes a video playback area and a progress display area for displaying the playback progress.

[0202] It should be noted that in response to the click operation, on the one hand, the electronic device is executing the process of displaying the video playback interface of Video A; on the other hand, it is also executing the process corresponding to the subsequent steps. Therefore, there is no strict sequence between step S103 and step S104 (and subsequent steps).

[0203] S104, after the gallery application receives the click operation on the cover of Video A, it sends a creation request for the frame concurrent decoding playback module and the frame buffer playback module to the decision-making module.

[0204] After the decision-making module receives the creation request for the frame concurrent decoding playback module and the frame buffer playback module, on the one hand, it creates the frame buffer playback module in the frame player; on the other hand, it creates the frame concurrent decoding playback module in the frame player.

[0205] In actual applications, S104 can also be split into two steps, one is to send a creation request for the frame concurrent decoding playback module to the decision-making module, and the other is to send a creation request for the frame buffer playback module to the decision-making module. The present application does not limit the specific implementation process.

[0206] S105, after the decision-making module receives the creation request for the frame concurrent decoding playback module and the frame buffer playback module, it creates the frame buffer playback module in the frame player.

[0207] S106, after the frame buffer playback module is successfully created, the frame buffer playback module creates a cache module.

[0208] S107, after the decision-making module receives the creation request for the frame concurrent decoding playback module and the frame buffer playback module, it creates the frame concurrent decoding playback module in the frame player.

[0209] After the gallery application receives a click operation on the cover of Video A, it sends the file path of Video A to the decision-making module.

[0210] In the embodiments of the present application, the file paths of each picture and each video file in the electronic device are stored in the gallery application of the electronic device. The gallery application can display pictures and play videos based on the file paths of each picture and video file.

[0211] In the embodiments of the present application, step S104 and step S108 can be executed as two independent steps, or can be combined into one step.

[0212] After the decision-making module receives the file path of Video A, on the one hand, it sends the file path of Video A to the normal playback module to enable the normal playback module to prepare for the normal playback of Video A; on the other hand, it sends the file path of Video A to the frame concurrent decoding playback module to enable the frame concurrent decoding playback module to prepare for the frame concurrent decoding playback of Video A; on the other hand, it sends the file path of Video A to the frame buffer playback module to enable the frame buffer playback module to prepare for the frame buffer playback of Video A.

[0213] First, describe sending the file path of Video A to the normal playback module and the subsequent preparation work of the normal playback module.

[0214] S109, after the decision-making module receives the file path of Video A, it sends the file path of Video A to the normal playback module.

[0215] S110, after the normal playback module receives the file path of Video A, it obtains Video A according to the file path and parses the encoding format of Video A. In the present application, the encoding format of Video A is taken as H.264 as an example.

[0216] S111, the normal playback module creates an H.264 decoder for normal playback according to the encoding format of Video A. Since this decoder is used for normal playback, it can be denoted as a normal decoder.

[0217] Next, describe sending the file path of Video A to the frame concurrent decoding playback module and the subsequent preparation work of the frame concurrent decoding playback module.

[0218] S112, after the decision-making module receives the file path of Video A, it sends the file path of Video A to the created frame concurrent decoding playback module.

[0219] S113, after the frame concurrent decoding playback module receives the file path of Video A, it obtains Video A according to the file path and parses the encoding format and file format of Video A. In the embodiments of the present application, the encoding format is taken as H.264 and the file format is taken as MP4 as an example.

[0220] S114. The frame concurrent decoding and playing module creates an MP4 parser according to the parsed file format, and at the same time sends the file path of Video A to the created parser, which can be denoted as the GOP parser.

[0221] S115. The frame concurrent decoding and playing module creates the first H.264 decoder according to the parsed encoding format.

[0222] S116. The frame concurrent decoding and playing module creates the second H.264 decoder according to the parsed encoding format.

[0223] For ease of description, the created decoders can be respectively denoted as GOP Decoder 1 and GOP Decoder 2. Of course, in practical applications, more decoders can also be created to improve the decoding ability of the electronic device. If more decoders for frame playing are created, they can also be sequentially denoted as GOP Decoder 3... and so on.

[0224] In the embodiments of the present application, one decoder can be obtained through one creation operation, and multiple creation operations are executed according to the number of decoders to be created; or multiple decoders can be created simultaneously through one creation operation.

[0225] In the embodiments of the present application, the parser is used to parse an MP4 format file into a video stream; the decoder is used to decode the video stream into video pixel information.

[0226] The following describes sending the file path of Video A to the frame buffer playing module and the subsequent preparation work of the frame buffer playing module.

[0227] S117. After receiving the file path of Video A, the decision module sends the file path of Video A to the created frame buffer playing module.

[0228] S118. After receiving the file path of Video A, the frame buffer playing module obtains Video A according to the file path and parses to obtain the encoding format of Video A. In the embodiments of the present application, the encoding format is H.264.

[0229] S119. The frame buffer playing module creates an H.264 encoding module according to the parsed encoding format. This encoding module is used to re-encode P frames and B frames into I frames.

[0230] S120. The frame buffer playing creates an H.264 decoding module according to the parsed encoding format. This decoding module is used to decode the I frames in the cache space into YUV data in the frame buffer playing process.

[0231] After this step, the preparations for normal playback, the preparations for frame concurrent decoding playback, and the preparations for frame buffer playback are all completed. After the electronic device receives an operation for normal playback, it will perform normal playback of the video. After receiving an operation for frame playback, it can determine whether to perform frame concurrent decoding playback or frame buffer playback of the video based on whether the cache space stores the video frames corresponding to the corresponding time.

[0232] The embodiments of this application and Figure 2 the playback process of the shown scenario correspond. Taking the case of first receiving an operation for normal playback as an example. Refer to Figure 11 , which is a timing diagram of a method for normal playback of a video provided by an embodiment of this application. This timing diagram can correspond to the process from the operation shown in (b) in Figure 2 to the corresponding interface shown in (c) in Figure 2 .

[0233] S201, The gallery application receives a click operation on the normal playback control.

[0234] In the embodiments of this application, when the electronic device displays the video playback interface of Video A shown in (b) in Figure 2 , the user clicks on the normal playback control in the video playback area shown in (b) in Figure 2 , which can trigger the electronic device to execute the normal playback process of Video A.

[0235] S202, After the gallery application receives the click operation on the normal playback control, it sends a normal playback request for Video A to the decision module. This request carries the start time of normal playback as the 0th second.

[0236] S203, After the decision module receives the normal playback request for Video A, it sends a playback instruction for Video A to the normal playback module. This playback instruction carries the start playback time as the 0th second.

[0237] S204, After the normal playback module receives the playback instruction for Video A, it parses Video A to obtain video frames in H.264 format.

[0238] S205, The normal playback module sends a decoding request for the video frames to the normal decoder.

[0239] In practical applications, the normal playback module can cache the video frames in H.264 format. This decoding request carries the cache address, and the normal decoder obtains the video frames in H.264 format from the cache address for decoding; of course, the normal playback module can also carry the video frames in H.264 format in the decoding request, and the normal decoder decodes the received video frames in H.264 format. This application does not limit the specific implementation method.

[0240] S206, The ordinary decoder decodes the video frame to obtain the decoded video data: YUV data.

[0241] The decoded video data is divided into I frames, P frames, and B frames. After decoding to obtain the YUV data, on the one hand, the YUV data is rendered and displayed; on the other hand, the YUV data is re-encoded and then cached.

[0242] First, describe the process of rendering and displaying the YUV data.

[0243] S207, The ordinary decoder sends the decoded YUV data to the ordinary player.

[0244] Similarly, the ordinary decoder can also cache the YUV data and then send the address of the cached YUV data to the ordinary player. The embodiments of the present application do not limit the specific transmission method.

[0245] S208, After receiving the decoded YUV data, the ordinary player renders the YUV data to obtain a video picture.

[0246] S209, The ordinary player sends the rendered video picture to the gallery application and carries the timestamp of the video picture.

[0247] S210, After receiving the video picture, the gallery application displays the received video picture in the video playback area and simultaneously updates the position of the progress display frame in the progress display area so that the moment corresponding to the progress indicator line is consistent with the timestamp of the received video picture.

[0248] Next, describe the process of re-encoding and caching the YUV data. After S206:

[0249] S211, The ordinary decoder sends the decoded YUV data to the decision module. The YUV data at this time is an I frame, P frame, or B frame.

[0250] Of course, in practical applications, the ordinary decoder can also send the YUV data in groups. For example, after decoding a group of YUV data, it sends the group of YUV data to the decision module. In this case, the YUV data includes I frames, P frames, and B frames. The embodiments of the present application do not limit the specific implementation method.

[0251] S212, After receiving the YUV data, the decision module sends the YUV data to the frame buffer playback module.

[0252] S213, After receiving the YUV data, the frame buffer playback module sends the YUV data to the cache module.

[0253] S214, after the cache module receives the YUV data, it checks that there is no video frame corresponding to the received YUV data in the cache space.

[0254] In practical applications, the YUV data itself carries a timestamp. In step S118, it has been determined that the currently displayed is the video playback interface of video A. Therefore, the cache module can check in the cache space whether there is a video frame corresponding to the timestamp carried by the YUV data of video A. Of course, the decision module can also carry the identification information of video A when sending the YUV data to the frame buffer playback module.

[0255] S215, when there is no video frame corresponding to the received YUV data in the cache space, the cache module sends the YUV data to the encoding module.

[0256] S216, the encoding module encodes the received YUV data into an I-frame in H.264 format.

[0257] In specific implementation, the YUV data received by the encoding module is the decoded data of I-frames, P-frames, and B-frames; therefore, the encoding module also needs to encode the YUV data of I-frames into I-frame data in H.264 format; re-encode the YUV data of P-frames into I-frame data in H.264 format, and re-encode the YUV data of B-frames into I-frame data in H.264 format.

[0258] In specific implementation, any encoder that can re-encode YUV data into a video frame in H.264 format can be selected, such as VJEncoder2.

[0259] S217, the encoding module sends the encoded I-frame to the cache module.

[0260] S218, after the cache module receives the I-frame sent by the encoding module, it caches the received I-frame.

[0261] In practical applications, the electronic device needs to continuously execute S206 to S218, so as to display each video frame on the video playback interface at the playback moment represented by the timestamp of each video frame in the video file; at the same time, re-encode the decoded video frames into I-frames and then cache them.

[0262] Refer to Figure 2 As shown in (c) in, it is the interface diagram when the video playback interface displayed by the gallery application reaches the 5th second after the user clicks the normal playback control. The normal playback module performs operations such as parsing and rendering. Therefore, the normal playback module can also include a normal parser and a rendering thread used by the normal player.

[0263] In an embodiment of the present application, when video A is played from the 0th second to the 5th second in the normal playback mode, the caching module has cached all the video frames that are I-frames of video A from the 0th second to the 5th second. Some of these cached I-frames were I-frames in the previous encoding, some were P-frames in the previous encoding, and some were B-frames in the previous encoding.

[0264] As described above, the electronic device has also performed the preparatory work for frame concurrent decoding playback and frame caching playback. Therefore, the video A can also be played frame by frame through the gallery application.

[0265] In an embodiment of the present application, by Figure 12 describing the frame concurrent decoding playback process of video A, Figure 12 the timing diagram shown corresponds to Figure 2 the process from (d) to Figure 2 the process of (e) in

[0266] The user can drag Figure 2 the progress display frame in the interface shown in (d) to switch the video A that is being played in the normal mode to frame playback. The user can also first pause Figure 2 the video A that is being played in the normal mode in the interface shown in (c), so that the video A is in a paused state. Then, by dragging the progress display frame, the video A in the paused state can be continuously displayed with the video frames in the video A in the frame playback mode. In an embodiment of the present application, when the video A is played in the normal mode, the user drags the progress display frame to the left to switch to the frame playback mode as an example.

[0267] S301, the gallery application receives the drag operation of the progress display frame.

[0268] In an embodiment of the present application, this drag operation is a leftward drag operation. Refer to Figure 2 the illustration in (d).

[0269] For an electronic device including a touch screen, the touch drive can sample the user's touch operations at a period T. The above period, also known as the polling period or reporting period, is the period for the touch drive to collect the user's touch operations.

[0270] Taking the touch drive sampling 240 times per second as an example. The period T = 1 / 240 = 4.16 ms, that is, the touch drive samples the touch data every 4.16 ms. If in a certain period, the user presses the touch screen with a single finger or two fingers, then the touch drive can determine the touch position of the single finger or two fingers according to voltage changes, etc.

[0271] Exemplarily, when the user wants to change the video frame displayed in the video playback area by dragging the position of the progress display frame, the touch driver receives the user's click operation corresponding to the progress display frame and passes the down event to the application processor. The application processor passes the down event to the gallery application. After receiving the click event, the touch driver continues to receive the user's continuous dragging operation corresponding to the progress display frame and passes the move event to the application processor. The application processor passes the move event to the gallery application.

[0272] S302. The gallery application determines the timestamp t1 of the frame playback of video A according to the dragging position of the dragging operation.

[0273] In the embodiment of the present application, a continuous dragging operation includes: the user's finger touches the progress display frame (down event), the user's finger moves (move event), and the user's finger releases from the progress display frame (up event). Figure 7 The illustrated embodiment describes the frame playback process by taking the dragging operation corresponding to the first two contact positions detected by the electronic device after the user's finger touches the progress display frame as an example.

[0274] S303. When video A is played in the normal playback mode, the gallery application sends a normal playback pause instruction and a frame playback request to the decision module, and the frame playback request carries the timestamp t1 of video A.

[0275] After receiving the normal playback pause instruction and the frame playback request of video A, the decision module needs to pause the normal playback of video A on the one hand and start the frame playback of video A on the other hand.

[0276] S304. After receiving the normal playback pause instruction, the decision module sends an instruction to pause playing video A to the normal playback module.

[0277] S305. After receiving the instruction to pause playing video A, the normal playback module sends a pause decoding instruction to the normal decoder.

[0278] S306. After receiving the temporary decoding instruction, the normal decoder pauses the decoding of video A.

[0279] S307. After receiving the instruction to pause playing video A, the player pauses the rendering of video A.

[0280] When playing a frame of Video A, first check whether the cache space has cached the video frame corresponding to time t1. If the video frame corresponding to time t1 is cached, execute the frame cache playback process; if the video frame corresponding to time t1 is not cached, execute the frame concurrent decoding playback process. Of course, in actual applications, the frame concurrent decoding playback process is only a frame playback process when the video frame corresponding to time t1 is not cached. In actual applications, the frame concurrent decoding playback process can also be other frame playback processes that do not require caching.

[0281] S308, after the decision module receives the frame playback request, first send a query request for the video frame to the frame cache playback module. The query request for the video frame carries the timestamp t1 of Video A.

[0282] S309, after the frame cache playback module receives the query request, send a query request for the video frame corresponding to time t1 of Video A to the cache module.

[0283] S310, the result of the cache module querying the video frame corresponding to time t1 is: there is no video frame at time t1.

[0284] In actual applications, when the cache module queries the video frame corresponding to time t1, it is not necessary to exactly find the video frame at time t1, but to find the video frame whose difference between the timestamp and time t1 is less than or equal to a certain threshold. This is because time t1 may not exactly correspond to a certain playback time (timestamp), but is between two playback times (timestamps). Therefore, half of the difference between the timestamps of two adjacent video frames can be set as this threshold.

[0285] S311, the cache module sends the information that there is no video frame at time t1 of Video A to the frame cache playback module.

[0286] S312, after the frame cache playback module receives the information that there is no video frame at time t1 of Video A, send the query result of no video frame at time t1 to the decision module.

[0287] S313, after the decision module receives the query result of no video frame at time t1 sent by the frame cache playback module, send a frame concurrent decoding playback instruction to the frame concurrent decoding playback module. This instruction carries the timestamp t1 of Video A.

[0288] S314, after the frame concurrent decoding playback module receives the frame playback instruction, send a parsing request to the GOP parser. This parsing request carries the timestamp t1 of Video A.

[0289] S315, after the frame concurrent decoding playback module receives the frame playback instruction, send a rendering request to the rendering thread. This rendering request carries the timestamp t1 of Video A.

[0290] S316. After the GOP parser receives a parsing request, it then parses Video A to obtain the H.264 video frames in the GOP group where the video frame at the current timestamp t1 in Video A is located, and records the timestamp ts of the first video frame and the timestamp te of the last video frame in the GOP group.

[0291] S317. The GOP parser sends a decoding request to the GPO decoding thread.

[0292] In specific implementation, the GOP parser can cache the decoded H.264 video frames, and the decoding request carries the cache address, or directly send the parsed H.264 video frames.

[0293] S318. The decoding thread sends a status acquisition request to GOP Decoder 1.

[0294] S319. The decoding thread receives that the status of GOP Decoder 1 is the idle state.

[0295] In the embodiment of the present application, when the status of GOP Decoder 1 is the idle state, decoding is performed through GOP Decoder 1; if the status of GOP Decoder 1 is not the idle state, it is necessary to check whether other GOP decoders are in the idle state to decode through the decoder in the idle state, so as to decode the video frames at a relatively fast speed.

[0296] This example takes Decoder 1 being in the idle state as an example.

[0297] S320. The decoding thread sends the H.264 video frames or the cache address of the H.264 video frames to the idle GOP Decoder 1.

[0298] S321. GOP Decoder 1 decodes the H.264 video frames to obtain YUV data and caches the YUV data.

[0299] The H.264 video frames obtained by the GOP decoder are all the video frames in the GOP group where the t1 video frame is located, and need to be decoded in sequence according to the decoding order until all the video frames in the group are decoded, and the decoded data is YUV data.

[0300] In practical applications, GOP Decoder 1 can decode the YUV data of a video frame and store the YUV data of this video frame in Cache Space 1. Correspondingly, GOP Decoder 2 has a corresponding Cache Space 2 to store the YUV data of the video frames decoded by GOP Decoder 2. When GOP Decoder 2 is required to decode, GOP Decoder 2 can store the YUV data of each decoded video frame in Cache Space 2 every time it decodes a video frame.

[0301] S322, the rendering thread searches for the GOP decoder corresponding to the video frame with timestamp t1 in the rendering request received. The GOP decoder is: GOP decoder one.

[0302] In practical applications, the rendering thread sends query requests to each GOP decoder in a polling manner. The query request carries the timestamp t1, and determines the GOP decoder corresponding to the video frame with timestamp t1 according to the query result sent by the decoder based on the query request.

[0303] The rendering thread can also search for the video frame at time t1 in the cache space corresponding to each GOP decoder in a polling manner until the video frame at time t1 is found in one of the cache spaces, so as to determine the GOP decoder corresponding to the video frame with timestamp t1.

[0304] Taking the rendering thread sending query requests to the GOP decoder in a polling manner as an example, in the embodiment of this application, after the rendering thread receives the rendering request:

[0305] The rendering thread first sends a query request (carrying t1) to GOP decoder 1; GOP decoder 1 queries the cache space 1 and determines that there is no video frame at time t1, and then sends the information of no video frame at time t1 to the rendering thread. In practical applications, if GOP decoder 1 is not the decoder for decoding the video frame with timestamp t1, then there is no video frame at time t1 in cache space 1, and GOP decoder 1 sends the information of no video frame with timestamp t1 to the rendering thread; if GOP decoder 1 is the decoder for decoding the video frame with timestamp t1, but has not decoded the video frame of t1 according to the decoding order, then there is also no video frame at time t1 in cache space 1, and GOP decoder 1 sends the information of no video frame with timestamp t1 to the rendering thread. This is because when GOP decoder decodes the video frames in the group of pictures (GOP), it decodes them according to the decoding order of the video frames. When GOP decoder 1 receives the query request, the decoder may not have decoded the video frame at time t1 yet. Therefore, there is no video frame at time t1 in cache space 1 either, and it is also necessary to send the information of no video frame with timestamp t1 to the rendering thread. If the rendering thread receives the information of no video frame with timestamp t1 sent by GOP decoder 1, it then sends a query request (carrying t1) to GOP decoder 2; GOP decoder 2 queries the cache space 2 and determines that there is no video frame at time t1, and then sends the information of no video frame at time t1 to the rendering thread. Similarly, if GOP decoder 2 is not the decoder for decoding the video frame with timestamp t1, then there is no video frame at time t1 in cache space 2, and GOP decoder 2 sends the information of no video frame with timestamp t1 to the rendering thread; if GOP decoder 2 is the decoder for decoding the video frame with timestamp t1, but has not decoded the video frame of t1 yet, then there is also no video frame at time t1 in cache space 2, and GOP decoder 2 sends the information of no video frame with timestamp t1 to the rendering thread.

[0306] If the rendering thread receives the information of no video frame with timestamp t1 sent by GOP decoder 2, it then continues to send a query request (carrying t1) to GOP decoder 1...;

[0307] In this polling manner, until the rendering thread receives the information of the video frame with timestamp t1 sent by any decoder; taking the example that the rendering thread receives the information of the video frame with timestamp t1 sent by GOP decoder 1 in the embodiment of the present application, the rendering thread can determine that the decoder for decoding the video frame with timestamp t1 is GOP decoder 1.

[0308] S323, the rendering thread obtains the YUV data at time t1 from the cache (cache space 1) of the determined GOP decoder 1.

[0309] S324, the rendering thread renders the obtained YUV data to obtain the rendered video frame image at time t1.

[0310] S325, the rendering thread sends the video frame corresponding to time t1 to the gallery application.

[0311] S326, the gallery application displays the video frame of video A at time t1 in the video playback area, and at the same time updates the position of the progress display frame, so that the time corresponding to the progress indicator line is consistent with the timestamp of the received video frame.

[0312] During the process of the user dragging the progress display frame, the contact position of the user's finger detected by the electronic device may change. Correspondingly, the positional relationship between the progress indicator line and the progress display frame also changes, and the time corresponding to the progress indicator line also changes. Therefore, during the process of the user dragging the progress display frame, the frame concurrent decoding and playback process shown in Figure 12 may be loop-executed until the corresponding video frame is found in the cache space.

[0313] In the specific implementation process, during the process of the user dragging the progress display frame, the contact position of the user's finger detected by the electronic device may change. Correspondingly, the positional relationship between the progress indicator line and the progress display frame also changes, and the time corresponding to the progress indicator line also changes. Suppose the times corresponding to two consecutive contact positions detected are t1 and t2 respectively, where the GOP group where the video frame at time t2 is located is the same as the GOP group where the video frame at time t1 is located. During the process of playing the frame to display the video frame at time t2, there is no need to repeat decoding.

[0314] As an example, after receiving the parsing request, the GOP parser determines that the video frame corresponding to the timestamp t2 in the parsing request is in the latest determined GOP group.

[0315] As mentioned above, in S316, the timestamp ts of the first video frame and the timestamp te of the last video frame of the GOP group where the video frame at time t1 is located have been recorded. It can be judged whether t2 is greater than or equal to ts and less than or equal to te; if t2 is greater than or equal to ts and less than or equal to te, it means that the video frame corresponding to t2 is in the latest GOP group; if t2 is less than ts or greater than te, it means that the video frame corresponding to t2 is not in the latest GOP group.

[0316] In this case, the GOP parser no longer parses video A, and the rendering thread can find the GOP decoder corresponding to the timestamp t2 according to the timestamp t2: GOP decoder one. Then continue to execute the subsequent process.

[0317] As mentioned above, there is another case: the case where the cache space caches the video frame corresponding to a certain moment, and the video frame corresponding to that moment is displayed through the frame cache playback process. The following is through Figure 13Describe the process of frame buffer playback, which corresponds to the frame playback process shown in (b) to Figure 3 (c) in Figure 3 The user drags the progress bar to the right to display frames, so as to show some video frames before the 5th second. An embodiment of the present application provides a process for realizing frame buffer playback at time t3, where time t3 is located between the 2nd second and the 5th second shown in Figure 3 as an example.

[0318] S401, The gallery application receives a drag operation to display frames.

[0319] This drag operation can be a drag operation to the right based on the interface shown in (b) in Figure 3 Before the drag operation, the user has played the video of video A from the 0th second to the 5th second in the normal playback mode. Correspondingly, the key frames between the 0th second and the 5th second of video A have been cached in the cache space.

[0320] S402, The gallery application determines the timestamp t3 of the frame playback of video A according to the drag position.

[0321] The process of detecting the timestamp t3 can refer to the above description.

[0322] S403, The gallery application sends a frame playback request to the decision module, and this request carries the timestamp t3 of video A.

[0323] S404, After receiving the frame playback request, the decision module first sends a query request for video frames to the frame buffer playback module, and this query request carries the timestamp t3 of video A.

[0324] S405, After receiving the query request, the frame buffer playback module sends a query request for the video frame at time t3 of video A to the cache module.

[0325] S406, The query result of the cache module for the video frame at time t3 of video A is: there is a video frame of video A at time t3.

[0326] S407, The cache module sends the query result to the frame buffer playback module.

[0327] S408, The frame buffer playback module sends the query result to the decision module.

[0328] S409, After receiving the query result that there is a video frame of video A at time t3, the decision module sends a frame buffer playback instruction to the frame buffer playback module, and this instruction carries the timestamp t3 of video A.

[0329] S410, After receiving the frame buffer playback request, the frame buffer playback module sends a request for obtaining the video frame at time t3 of video A to the cache module.

[0330] S411, after the cache module receives the acquisition request, it sends the video frame at time t3 to the decoding module.

[0331] In this application, the cache module can first obtain the video frame at time t3 from the cache space. Since the cache module caches the key frames corresponding to each time, the obtained video frame at time t3 is a key frame at time t3, and there are no other video frames.

[0332] S412, after the decoding module receives the video frame at time t3, it decodes the H.264 format video frame at time t3 to obtain the YUV data at time t3. This step only needs to decode the video frame at time t3.

[0333] S413, the decoding module sends the YUV data at time t3 to the cache module.

[0334] S414, after the cache module receives the YUV data at time t3, it sends the YUV data at time t3 to the frame buffer playback module.

[0335] S415, after the frame buffer playback module receives the YUV data at time t3, it renders the YUV data at time t3 to obtain the video frame picture at time t3.

[0336] S416, the frame buffer playback module sends the video frame picture at time t3 to the gallery application.

[0337] S417, after the gallery application receives the video frame picture at time t3, it displays the video frame picture at time t3 of video A in the video playback area, and at the same time updates the position of the progress display frame, so that the time corresponding to the progress indicator line is consistent with the time stamp of the received video picture.

[0338] Through Figure 13 As can be understood from the shown timing diagram, by means of frame buffer playback, only one video frame at time t3 needs to be decoded during decoding, which reduces the duration of obtaining the video frame at time t3, thereby improving the smoothness during frame playback.

[0339] Of course, as the user drags the progress display frame, the electronic device may execute the Figure 13 shown frame buffer playback in a loop until the corresponding video frame cannot be found in the cache space.

[0340] In addition, after the user finishes dragging the progress display frame, that is, after the user's finger lifts from the area of the progress display frame, the playback process of video A will resume to the state before switching to frame playback: playing video A normally or pausing the playback of video A.

[0341] In specific implementation, the first case: When playing video A in the normal mode, if the user drags the progress display frame, after the user's finger lifts from the progress display frame area, the playback process of video A switches from frame playback to the normal playback mode, and continues to play in the normal mode starting from the moment indicated by the progress indicator line when the user's finger lifts from the progress display frame.

[0342] The second case: When playing video A in the normal mode, if the user first pauses the normal playback process of video A, and then drags the progress display frame in the paused state of video A, after the user's finger lifts from the progress display frame area, the playback process of video A switches from frame playback to the paused state. After the user clicks the normal playback control in the video playback area of the interface, it continues to play in the normal mode starting from the moment indicated by the progress indicator line (the moment indicated by the progress indicator line when the user's finger lifts from the progress display frame area).

[0343] In the embodiment of the present application, taking the first case as an example, at time t4 when the user's finger lifts from the progress display frame area to release the progress display frame, video A switches from frame playback to normal playback.

[0344] Refer to Figure 14 , for the timing diagram of switching from frame playback to normal playback when the moment pointed to by the progress indicator line at the release operation on the progress display frame provided in the embodiment of the present application is time t4 (for example, Figure 3 the 2nd second shown in Figure 3 ). This diagram corresponds to

[0345] S501, The gallery application receives the release operation on the progress display frame.

[0346] In the embodiment of the present application, the release operation in S501 (after dragging ends, the user's finger lifts from the screen) and the previous Figure 13 dragging operation are continuous gesture operations, that is, the user's finger drags the progress display frame so that the moment indicated by the progress indicator line reaches the 2nd second and then lifts the hand, and this hand-lifting operation is the release operation on the progress display frame.

[0347] S502, In response to this operation, the gallery application sends the frame playback end information of video A and the timestamp t3 at the end to the decision module.

[0348] In specific implementation, a flag bit can be set in the gallery application or the decision module to mark whether the video A is in the normal playback state or the paused playback state before frame playback; if it is in the paused state, the subsequent process of resuming normal playback will not be executed; if it is in the playback state, since the decision module pauses the normal playback when switching to frame playback (the normal playback paused simultaneously when switching to frame playback does not modify the flag bit), it is necessary to send a playback instruction to the normal player again, that is, execute the subsequent operations.

[0349] S503, after the decision module receives the frame playback end information and the timestamp t4, it sends a video A start playback instruction to the normal playback module, and this instruction carries the start playback time t4.

[0350] S504, after the normal playback module receives the video A start playback instruction and the timestamp t4, it starts to parse the file of video A based on the time t4, and obtains video frames in H.264 format.

[0351] After S604, the process of normal playback is executed and Figure 11 is similar to the steps executed after S204 in Figure 11 except that in Figure 14 decoding starts from the video frame at the 0th second;

[0352] After S504 in

[0353] playback starts from the video frame at the time t4. Figure 11 Of course, in practical applications, it may be necessary to start decoding from the key frame in the GOP group where the time t4 is located. Therefore, it may start decoding from the previous key frame of the video frame at the time t4.

[0354] S505, the normal playback module sends a decoding request for the video frame to the normal decoder. The description of this decoding request can refer to

[0355] the description in S205 in

[0356] First, the process of rendering and displaying the YUV data will be described.

[0357] S507, the normal decoder sends the decoded YUV data to the normal player.

[0358] Similarly, the normal decoder can also cache the YUV data and then send the address of the cached YUV data to the normal player. The specific transmission method in the embodiments of the present application is not limited.

[0359] S508. After the general player receives the decoded YUV data, it renders the YUV data to obtain a video picture.

[0360] S509. The general player sends the rendered video picture to the gallery application and carries the timestamp of the video picture.

[0361] S510. After the gallery application receives the video picture, it displays the received video picture in the video playback area and simultaneously updates the position of the progress display frame in the progress display area so that the moment corresponding to the progress indicator line is consistent with the timestamp of the received video picture.

[0362] In the specific implementation process, according to Figure 2 in (b) and Figure 2 in (c) and Figure 11 as shown, the video frames from the 0th to the 5th second have been cached; if the drag operation is to the left (refer to Figure 2 in (d) and Figure 2 in (e)), then the frame concurrent decoding and playback process is executed according to Figure 12 ; if the drag operation is to the right (refer to Figure 3 in (b) and Figure 3 in (c)), then the frame caching and playback process is executed according to Figure 13 as shown; if the user's finger is released from the progress display frame based on the interface shown in Figure 3 in (c) (the progress indicator line points to the 2nd second), then in the subsequent general playback process, the video frames from the 2nd to the 5th second are no longer re-encoded and cached; the video frames after the 5th second continue to be re-encoded and cached.

[0363] The following describes in detail the process of whether re-encoding is required in the general playback process after the user's finger is released from the progress display frame based on the interface shown in Figure 3 in (c). After S506:

[0364] S511. The general decoder sends the decoded YUV data to the decision module. The YUV data at this time is an I-frame, a P-frame, or a B-frame.

[0365] S512. After the decision module receives the YUV data, it sends the YUV data to the frame caching and playback module.

[0366] S513. After the frame caching and playback module receives the YUV data, it sends the YUV data to the caching module.

[0367] S514. After the caching module receives the YUV data, it checks whether there is a video frame corresponding to the received YUV data in the cache space.

[0368] If the video frame corresponding to the received YUV data already exists in the cache space, the cache module discards the received YUV data.

[0369] S515. If, in the case where the video frame corresponding to the received YUV data does not exist in the cache space, send the YUV data to the encoding module.

[0370] S516. The encoding module encodes the received YUV data into an I-frame in H.264 format.

[0371] S517. The encoding module sends the encoded I-frame to the cache module.

[0372] S518. After the cache module receives the I-frame sent by the encoding module, it caches the received I-frame.

[0373] In practical applications, as normal playback proceeds, the electronic device needs to continuously execute S506 to S518, so as to display each video frame on the video playback interface at the playback moment represented by the time stamp of each video frame in the video file; at the same time, the decoded video frames are re-encoded and cached.

[0374] The normal playback process and frame playback process of Video A are described above. As mentioned before, when opening the video playback interface of Video A, a parser, a decoder, etc. need to be created; similarly, when exiting the video playback interface of Video A, the created parser, decoder, etc. also need to be destroyed.

[0375] Refer to Figure 15 , which is the timing diagram for destroying the parser and decoder when exiting the video playback interface of Video A provided by the embodiments of the present application.

[0376] S601. In the case of displaying the video playback interface of Video A, the gallery application receives a return operation.

[0377] In the embodiments of the present application, the return operation can be an operation of swiping left from the rightmost side of the screen on the interface, and this operation can trigger the gallery application to return to Figure 2 the grid view interface shown in (a) in

[0378] S602. After receiving the return operation, the gallery application displays a grid view. The interface where the grid view is located is the interface before entering the video playback interface of Video A shown in (b) in Figure 2 , and the grid view includes the cover of Video A.

[0379] S603. After receiving the return operation, the gallery application sends a destruction request to the decision module. The destruction request is used to destroy each module created when entering the video playback interface of Video A.

[0380] It should be noted that there is no strict sequence between S602 and S603. They can be two parallel steps, or one step can be executed first and then the other step.

[0381] After the decision module receives the destruction request, on the one hand, it destroys the relevant modules in the frame player for frame concurrent decoding and playback (frame concurrent decoding and playback module, GOP parser, and GOP decoder). On the other hand, it destroys the relevant modules in the frame player for frame buffer playback (frame buffer playback module, buffer module, encoding module, and decoding module). Additionally, it destroys the relevant modules in the ordinary player (e.g., ordinary decoder). The destruction operations in these three aspects can be executed simultaneously, or the destruction operation in one aspect can be executed first, and then the destruction operations in other aspects can be executed.

[0382] The destruction of the relevant modules for frame buffer playback (e.g., frame buffer playback module, buffer module, encoding module, and decoding module) is described through S604 to S609.

[0383] In S604, after the decision module receives the destruction request, it sends the destruction requests for the buffer module, encoding module, and decoding module to the frame buffer playback module.

[0384] In S605, after the frame buffer playback module receives the destruction requests for the buffer module, encoding module, and decoding module, it destroys the buffer module.

[0385] In S606, after the frame buffer playback module receives the destruction requests for the buffer module, encoding module, and decoding module, it destroys the encoding module.

[0386] In S607, after the frame buffer playback module receives the destruction requests for the buffer module, encoding module, and decoding module, it destroys the decoding module.

[0387] In S608, after the frame buffer playback module destroys the buffer module, encoding module, and decoding module, it sends the information that the destruction of the buffer module, encoding module, and decoding module is successful to the decision module.

[0388] In S609, after the decision module receives the information that the destruction of the buffer module, encoding module, and decoding module is successful, it destroys the frame buffer playback module.

[0389] The process of destroying the relevant modules for frame concurrent decoding and playback in the frame player (e.g., frame concurrent decoding and playback module, GOP parser, and GOP decoder) is described through S610 to S615.

[0390] In S610, after the decision module receives the destruction request, it sends the destruction requests for the parser and decoder to the frame concurrent decoding and playback module.

[0391] S611. After the frame concurrent decoding and playing module receives the destruction requests of the parser and the decoder, it destroys the GOP parser.

[0392] S612. After the frame concurrent decoding and playing module receives the destruction requests of the parser and the decoder, it destroys the first GOP decoder.

[0393] S613. After the frame concurrent decoding and playing module receives the destruction requests of the parser and the decoder, it destroys the second GOP decoder.

[0394] Among them, the steps of destroying the GOP parser and the GOP decoders do not have a strict sequence.

[0395] S614. After the frame concurrent decoding and playing module destroys the GOP parser, the first GOP decoder and the second GOP decoder, it sends the information that the destruction of the parser and the decoder is successful to the decision module.

[0396] S615. After the decision module receives the information that the destruction of the parser and the decoder is successful sent by the frame concurrent decoder playing module, it destroys the frame concurrent decoding and playing module.

[0397] The process of destroying the relevant modules (such as the ordinary decoder) in the ordinary player is described through S616 to S618.

[0398] S616. After the decision module receives the destruction request, it sends the destruction request of the decoder to the ordinary playing module.

[0399] S617. After the ordinary playing module receives the destruction request of the decoder, it destroys the ordinary decoder.

[0400] S618. After the ordinary playing module destroys the ordinary decoder, it sends the information that the destruction of the decoder is successful to the decision module.

[0401] The modules related to frame buffer playing, frame concurrent decoding playing and ordinary playing are described above respectively. After the modules related to the above three playing methods are destroyed, the information that the destruction is successful is sent to the gallery application.

[0402] S619. After the decision module destroys the frame concurrent decoding and playing module, destroys the frame buffer playing module and receives the information that the destruction of the decoder sent by the ordinary playing module is successful, it sends the information that the destruction is successful to the gallery application.

[0403] In the video playing method provided by the embodiment of the present application, it includes an ordinary playing process, a frame buffer playing process and a frame concurrent decoding playing process. Specifically, it may include: the electronic device displays the first interface of the first video, the first interface includes a first area and a first control, and a second control is included in the first area;

[0404] Normal playback process: After the electronic device receives the first operation on the second control, it displays the first video frame obtained by decoding the first encoded data of the first video in the first area in the playback order, moves the first control from the first position to the second position along the first track, caches the second encoded data of the first video frame, and the decoding efficiency of the second encoded data is higher than that of the first encoded data;

[0405] Frame cache playback process: The electronic device receives the first drag operation on the first control; during the process of dragging the first control to the third position through the first drag operation, the electronic device displays the video frame obtained by decoding the second encoded data in the first area, and the third position is on the first track.

[0406] Frame concurrent decoding playback process: In the case of dragging the first control to the fourth position through the first drag operation, the electronic device obtains the third encoded data of the first video. When the first control is in the fourth position, the playback progress of the first video is the first moment, and the encoded data corresponding to the first moment is the fourth encoded data in the third encoded data. The frame type in the third encoded data is the same as the frame type in the first encoded data;

[0407] The electronic device decodes the third encoded data through the first decoder;

[0408] After the electronic device decodes the fourth encoded data through the first decoder, the electronic device displays the video frame obtained by decoding the fourth encoded data in the first area.

[0409] In the case of dragging the first control from the fourth position to the fifth position through the first drag operation, the electronic device obtains the fifth encoded data of the first video. When the first control is in the fifth position, the playback progress of the first video is the second moment, and the encoded data corresponding to the second moment is the sixth encoded data in the fifth encoded data. The frame type in the fifth encoded data is the same as the frame type in the first encoded data, and the first video frame in the fifth encoded data and the last video frame in the third encoded data are adjacent video frames;

[0410] The electronic device decodes the fifth encoded data through the second decoder;

[0411] After the electronic device decodes the fifth encoded data through the second decoder, the electronic device displays the video frame obtained by decoding the sixth encoded data in the first area.

[0412] During the above video playback, the first video may be Video A in the above embodiments, the first interface may be the video playback interface in the above embodiments, the first area may be the video playback area in the above embodiments, and the first control may be the progress display frame; the second control may be the control for triggering normal playback in the video playback area. The position of the first control is the position of the reference point on the progress display frame. The reference point may be any point on the progress display frame. Of course, it may also be the point corresponding to the contact point of the user's finger during the dragging process.

[0413] The first video frame in the above example may be Figure 2 The video frames from the 0th second to the 5th second shown. When the first control is in the first position, refer to Figure 2 The position of the progress display frame shown in (b) in; when the first control is in the second position, refer to Figure 2 The position of the progress display frame shown in (c) in. The first encoded data is the data of the I frames, P frames, and B frames from the 0th second to the 5th second. The second encoded data is the corresponding encoded data after re-encoding the data of the I frames, P frames, and B frames from the 0th second to the 5th second into I frames.

[0414] The first dragging operation may be Figure 3 The dragging operation corresponding to (b) to Figure 3 The (c) in, so the third position may be Figure 3 The position of the progress display frame shown in (c) in. When displaying the video frames decoded from the second encoded data, the video frames decoded from the encoded data between the 2nd second and the 5th second in the second encoded data may be displayed.

[0415] In addition, the positions passed through during the dragging process include the sixth position. For example, when the first control is at the sixth position, the playback progress is the fourth moment, and the fourth moment is between the 2nd second and the 5th second. Therefore, the video frames decoded from the encoded data corresponding to the fourth moment after re-encoding may be displayed.

[0416] Of course, in the case of dragging in the dragging manner shown in Figure 2 , it may be dragged from the 5th second to the 14th second. The position of the first control is Figure 2 The position between that shown in (d) in and Figure 2 The (e) in. From (d) in Figure 2 To Figure 2During the process of (e) in the above, the position of the first control passes through the fourth position and the fifth position. When the position of the first control is at the fourth position, the playback progress of the first video is the first moment, which is a moment between the 5th second and the 14th second; when the position of the first control is at the fifth position, the playback progress of the first video is the second moment, which is a moment between the 5th second and the 14th second. The video frame group at the first moment and the video frame group at the second moment are two adjacent video frame groups, that is, the last video frame in the video frame group at the first moment and the first video frame in the video frame group at the second moment are adjacent video frames. Two adjacent video frame groups may be decoded using different decoders. For example, if the previous video frame group uses the first decoder, the next video frame group uses the second decoder. Of course, an idle decoder may also be used for decoding.

[0417] In addition, in the process of displaying the video frames of the first video triggered by the first operation of the second control, the video frames to be displayed can be obtained by decoding the first encoded data of the first video; in the process of displaying the video frames of the first video triggered by the dragging operation (remembered as frame playback), the video frames to be displayed can be obtained by decoding the second encoded data; since the efficiency of the electronic device in decoding the second encoded data is higher than the efficiency of decoding the first encoded data, the decoding efficiency in the frame playback scenario can be improved, thereby improving the smoothness of the video picture in the frame playback scenario.

[0418] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0419] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed on an electronic device, the steps in the above-mentioned method embodiments can be implemented.

[0420] The embodiment of the present application also provides a computer program product. When the computer program product is run on an electronic device or a wireless router, the electronic device can implement the steps in the above-mentioned various method embodiments.

[0421] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above method embodiments of this application, a computer program can be used to instruct relevant hardware to complete. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the first device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal, and software distribution medium. For example, USB flash drive, mobile hard disk, magnetic disk or optical disc, etc. In some jurisdictions, according to legislation and patent practice, the computer-readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0422] The embodiments of this application also provide a chip. The chip includes a processor, and the processor is coupled with a memory. The processor calls the computer program stored in the memory to implement the steps of any method embodiment of this application. The chip can be a single chip or a chip module composed of multiple chips.

[0423] In the above embodiments, the descriptions of each embodiment have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0424] Those of ordinary skill in the art can realize that the units and method steps of each example described in combination with the embodiments disclosed in this article can be implemented by electronic hardware, or by a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods for each specific application to implement the described functions, but this implementation should not be considered to exceed the scope of this application.

[0425] The above embodiments are only used to illustrate the technical solutions of this application, rather than to limit it; although this application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of each embodiment of this application, and should all be included in the protection scope of this application.

Claims

1. A video playback method, characterized in that, Including: An electronic device displays a first interface of a first video. The first interface includes a first area and a first control, and a second control is included in the first area; After the electronic device receives a first operation on the second control, it displays in the first area a first video frame obtained by decoding first encoded data of the first video; During the process of displaying the first video frame in the first area, the electronic device controls the first control to move from a first position along a first trajectory to a second position; When the first control moves to the second position, the electronic device receives a first drag operation on the first control; During the process of moving the first control from the second position to a third position by the first drag operation, the electronic device displays in the first area a video frame obtained by decoding second encoded data of the first video. The third position is on the first trajectory and the third position is between the first position and the second position. The efficiency of the electronic device in decoding the second encoded data is higher than the efficiency of decoding the first encoded data.

2. The method according to claim 1, characterized in that, During the process of displaying the first video frame in the first area, the method further includes: The electronic device caches the second encoded data of the first video frame.

3. The method according to claim 2, characterized in that The method further includes: In the case of dragging the first control from the third position to a fourth position, the electronic device obtains third encoded data of the first video. The fourth position is not on the first trajectory. When the first control is at the fourth position, the playback progress of the first video is a first moment, and the encoded data corresponding to the first moment is fourth encoded data in the third encoded data. The frame type in the third encoded data is the same as the frame type in the first encoded data; The electronic device decodes the third encoded data through a first decoder; After the electronic device completes the decoding of the fourth encoded data through the first decoder, the electronic device displays in the first area a video frame obtained by decoding the fourth encoded data.

4. The method according to claim 3, wherein The method further includes: In the case of dragging the first control from the fourth position to a fifth position, the electronic device obtains fifth encoded data of the first video. The fifth position is not on the first trajectory. When the first control is at the fifth position, the playback progress of the first video is a second moment, and the encoded data corresponding to the second moment is sixth encoded data in the fifth encoded data. The frame type in the fifth encoded data is the same as the frame type in the first encoded data, and the first video frame in the fifth encoded data and the last video frame in the third encoded data are adjacent video frames; The electronic device decodes the fifth encoded data through a second decoder; After the electronic device completes the decoding of the fifth encoded data through the second decoder, the electronic device displays in the first area a video frame obtained by decoding the sixth encoded data.

5. The method according to any one of claims 2 to 4, characterized in that During the process of displaying in the first area a first video frame obtained by decoding first encoded data of the first video, it includes: For each piece of first encoded data, the electronic device decodes the first encoded data through a third decoder to obtain YUV data; The electronic device displays, in the first area, a video frame corresponding to the YUV data obtained by decoding the first encoded data.

6. The method according to claim 5, wherein During the process that the electronic device caches the second encoded data of the first video frame, it includes: For each piece of first encoded data, the electronic device encodes the YUV data obtained from the first encoded data to obtain second encoded data corresponding to the first encoded data; The electronic device caches, in a first storage space, the second encoded data corresponding to the first encoded data and the timestamp of the first encoded data.

7. The method according to claim 6, wherein Before the electronic device encodes the YUV data obtained from the first encoded data at the third moment, where the first encoded data includes encoded data with a timestamp of the third moment, the method further includes: The electronic device determines that there is no second encoded data of the first video at the third moment in the first storage space.

8. The method according to any one of claims 2 to 7, characterized in that, During the process of moving the first control from the second position to the third position through the first drag operation, the electronic device displays, in the first area, a video frame obtained by decoding the second encoded data, including: In the case of moving the first control from the second position to the sixth position through the first drag operation, the electronic device displays, in the first area, a video frame obtained by decoding the seventh encoded data in the second encoded data. When the first control is at the sixth position, the playback progress of the first video is the fourth moment, the seventh encoded data is the encoded data corresponding to the fourth moment, and the sixth position is on the first track.

9. The method according to claim 8, wherein In the case of moving the first control from the second position to the sixth position through the first drag operation, the electronic device displaying, in the first area, a video frame obtained by decoding the seventh encoded data in the second encoded data includes: In the case of moving the first control from the second position to the sixth position through the first drag operation, the electronic device determines that the playback progress of the first video is the fourth moment; The electronic device queries for the seventh encoded data corresponding to the fourth moment stored in the first storage space; The electronic device queries and finds the seventh encoded data in the first storage space; The electronic device obtains the seventh encoded data from the first storage space; The electronic device decodes the seventh encoded data; The electronic device displays, in the first area, the video frame obtained by decoding the seventh encoded data.

10. The method according to claim 3 or 4, characterized in that In the case of dragging the first control to the fourth position through the first drag operation, the electronic device obtaining the third encoded data of the first video, includes: In the case of dragging the first control to the fourth position through the first drag operation, the electronic device determines that the playback progress of the first video is the first moment; The electronic device does not query and find the encoded data corresponding to the first moment in the first storage space; The electronic device obtains the third encoded data of the first video at the first moment.

11. The method according to claim 10, characterized in that, The encoded data corresponding to the first moment is the encoded data whose difference between the timestamp and the first moment is less than or equal to a first value, and the first value is half of the difference between the timestamps of two adjacent video frames.

12. The method according to any one of claims 1 to 11, characterized in that, The first interface further includes a second area, and the first control is located within the second area and can move within the second area; The first area and the second area do not overlap, or the second area is within the first area.

13. The method according to claim 1, characterized in that, The frame types in the first encoded data include key frames and non-key frames, and the frame types in the second encoded data are all key frames.

14. An electronic device, characterized in that, Comprising one or more processors and one or more memories; wherein, the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer programs. When the one or more processors execute the computer programs, the electronic device is caused to execute the method according to any one of claims 1-13.

15. A chip system, which is applied to an electronic device, and the chip system includes one or more processors, characterized in that, The processor is used to call computer instructions to cause the electronic device to execute the method according to any one of claims 1-13.

16. A computer-readable storage medium, comprising a computer program, characterized in that, When the computer program runs on the electronic device, the electronic device is caused to execute the method according to any one of claims 1-13.

Images