Video playing method, electronic equipment, chip system and storage medium

Abstract

The invention provides a video playing method, electronic equipment, a chip system and a storage medium, and relates to the technical field of electronic equipment, in a playing state of common playing, a pressing operation in a sliding gesture triggers switching from common playing to frame playing, and a frame player temporarily does not take over a rendering task; the dragging operation triggers the frame player to take over the rendering task, so that the picture jump during the pressing operation can be avoided; in the embodiment of the invention, the release operation triggers switching from frame playing to ordinary playing, and the ordinary player takes over the rendering task and controls the first rendered video frame to be later than the last video frame of the frame playing, so that picture bounce during the release operation of the user can be avoided; in the pause state of the common playing mode, when the release operation triggers the switching from the frame playing to the common playing, the common player does not take over the rendering task, but the common player takes over the rendering task again when the operation of switching the pause state to the playing state is detected, so that the rendering task is prevented from being handed over back and forth.

Description

A video playback method, electronic device, chip system, and storage medium Technical Field

[0001] This application relates to the field of electronic equipment technology, and in particular to a video playback method, electronic equipment, chip system, and storage medium. Background Technology

[0002] Gallery apps on electronic devices can provide image preview interfaces, allowing users to view various images stored on the device. Gallery apps also provide video playback interfaces, allowing users to play various videos stored on the device.

[0003] The video playback interface can also display the video playback progress, for example, through a progress bar, progress ring, or progress display frame. User drag gestures (press, drag, and release) on the progress bar, progress ring, or progress display frame can trigger the electronic device to switch between normal playback mode and frame playback mode. However, currently, the video screen often jumps during the switching process. Summary of the Invention

[0004] This application provides a video playback method, electronic device, chip system, and storage medium that can reduce screen jumps when switching between normal playback and frame playback.

[0005] To achieve the above objectives, the first aspect of this application provides a video playback method, employing the following technical solution:

[0006] Applied to electronic devices including general-purpose media players and frame players, the method includes:

[0007] During the normal playback mode of the first video, the electronic device detects the first press operation;

[0008] In response to the first press operation, the electronic device switches from normal playback mode to frame playback mode. The normal player stops executing the decoding and rendering tasks of the first video, and the frame player starts executing the decoding task of the first video.

[0009] The electronic device detected the first drag operation;

[0010] In response to the first drag operation, the frame player takes over the rendering task of the first video;

[0011] After taking over the rendering task of the first video, the frame player begins to execute the rendering task of the first video.

[0012] In this application, during normal playback, a press operation in the swipe gesture triggers a switch from normal playback to frame playback, and the frame player temporarily does not take over the rendering task. A drag operation then triggers the frame player to take over the rendering task. Since each time the rendering task is taken over, it triggers re-decoding and re-rendering. Therefore, when the frame player takes over the rendering task during a press operation, it will cause the re-decoded and re-rendered video frame to be displayed when the user presses the operation. However, the re-decoded and re-rendered video frame may not be completely consistent with the last video frame of the previous normal playback (in order to respond as quickly as possible, a relatively close video frame is usually selected from the decoded video frames for rendering and display). This causes the user to see a change in the screen when the user presses the operation, resulting in a poor user experience. If the time when the frame player takes over the rendering task is set to the initial drag operation, even if there is a change in the screen, it will generally be considered to be related to the user's drag operation. From the user's perspective, the screen jump when pressing the operation can be avoided. The drag operation is part of the frame playback process, and the screen in the frame playback process is not smooth. The smoothness is related to the speed of the user's drag. Therefore, the screen jump when the user presses the operation is within the acceptable range for the user.

[0013] As one implementation of the first aspect, after the electronic device detects the first drag operation, the method further includes:

[0014] The electronic device detects a first release operation, which is the release operation corresponding to the first drag operation.

[0015] In response to the first release operation, the electronic device switches the frame playback mode to the normal playback mode, and the normal player begins to execute the decoding task of the first video and takes over the rendering task of the first video.

[0016] After taking over the rendering task of the first video, the ordinary player begins to execute the rendering task. The timestamp of the first video frame rendered by the ordinary player is later than the timestamp of the last video frame played by the electronic device before the first release operation.

[0017] In this application, the release operation triggers a switch from frame playback to normal playback. The normal player takes over the rendering task and controls the first video frame to be rendered later than the last video frame of the frame playback, thus avoiding screen bounce when the user releases the screen. Normally, after the normal player takes over the rendering task, it also needs to re-decode and render based on the last moment of the frame playback. However, in order to respond as quickly as possible, when rendering is required, it will also select a video frame from the decoded video frames for rendering and display. However, if the first video frame rendered and displayed by the normal player is earlier than the last video frame of the frame playback, it will cause the screen to bounce before continuing playback. This application sets the timestamp of the first rendering task after normal playback to be later than the timestamp of the last video frame displayed by the frame playback, thus avoiding the bounce situation.

[0018] As one implementation of the first aspect, after the electronic device detects the first release operation, the method further includes:

[0019] The electronic device detected a second press operation;

[0020] In response to the second press operation, the normal player stops executing the decoding and rendering tasks of the first video, and the frame player starts executing the decoding task of the first video;

[0021] The electronic device detects a second release operation, which is the release operation corresponding to the second press operation;

[0022] In response to the second release operation, the electronic device switches from frame playback mode to normal playback mode, and the normal player begins to execute the decoding and rendering tasks of the first video. When the normal player executes the rendering task, the timestamp of the first video frame is later than the timestamp of the last video frame played by the electronic device before switching to normal playback mode.

[0023] In this application, although the playback switches from normal to frame-based playback when the button is pressed, frame-based playback does not take over the rendering task. Therefore, if the button is released without dragging, although the playback switches from frame-based to normal, there is no handover of rendering tasks. This avoids the screen jumps that may occur when the button is pressed and also avoids the system resource consumption caused by the handover of rendering tasks when the button is released, resulting in a faster response time when switching to normal playback. Of course, it is also necessary to set the timestamp of the first video frame rendered by the normal player to be later than the timestamp of the last video frame played by the electronic device before switching to normal playback mode to avoid screen bounces.

[0024] As one implementation of the first aspect, the method further includes:

[0025] In the normal playback mode of the first video, the electronic device receives the first click operation;

[0026] In response to the first click operation, the electronic device switches the playback state to a pause state, and the ordinary player stops executing the decoding and rendering tasks of the first video;

[0027] While the first video is paused in normal playback mode, the electronic device detects a third press operation;

[0028] In response to the third press operation, the electronic device switches from normal playback mode to frame playback mode, and the frame player begins to perform the decoding task of the first video;

[0029] The electronic device detected a third dragging operation;

[0030] In response to the third drag operation, the frame player takes over the rendering task of the first video;

[0031] After taking over the rendering task of the first video, the frame player begins to execute the rendering task of the first video.

[0032] In this application, even when paused in normal playback mode, the switch from normal playback to frame playback is triggered by pressing the swipe gesture, and the frame player temporarily does not take over the rendering task; the drag operation then triggers the frame player to take over the rendering task, thereby avoiding screen jump phenomena.

[0033] As one implementation of the first aspect, after the electronic device detects the third drag operation, it also includes:

[0034] The electronic device detects a third release operation, which is the release operation corresponding to the third drag operation;

[0035] In response to the third release operation, the electronic device switches the frame playback mode to the normal playback mode, in which the first video is paused.

[0036] In this application, when the release operation triggers the switch from frame playback to normal playback, since normal playback mode is in a paused state, no rendering work is actually required when switching from frame playback to normal playback. Therefore, the normal player does not take over the rendering task. Instead, the normal player takes over the rendering task when it detects the operation that switches the paused state to the playback state, thus avoiding the consumption of system resources caused by repeatedly handing over rendering tasks.

[0037] As one implementation of the first aspect, after the electronic device detects the third release operation, it also includes:

[0038] The electronic device detected the fourth press operation;

[0039] In response to the fourth press operation, the electronic device switches from normal playback mode to frame playback mode, and the frame player begins to perform the decoding task of the first video;

[0040] The electronic device detects the fourth drag operation, and the frame player performs decoding and rendering tasks.

[0041] As one implementation of the first aspect, after the electronic device detects the fourth drag operation, it further includes:

[0042] The electronic device detects a fourth release operation, which is the release operation corresponding to the fourth drag operation;

[0043] In response to the fourth release operation, the electronic device switches the frame playback mode to the normal playback mode, in which the first video is paused.

[0044] In this application, since the rendering task is switched to the frame player when dragging, although the playback switches from frame playback to normal playback when the operation is released, normal playback does not take over the rendering task. The frame player still takes over the rendering task. Therefore, when the operation is pressed again, regardless of whether the drag is made or not, the frame player takes over the rendering task throughout the entire process of releasing the operation, which avoids frequent handover of rendering tasks and reduces system resource consumption.

[0045] As one implementation of the first aspect, after the electronic device detects the fourth release operation, it also includes:

[0046] The electronic device detected a second click.

[0047] In response to the second click operation, the electronic device switches from frame playback mode to normal playback mode. The normal player starts to execute the decoding task of the first video and takes over the rendering task. After taking over the rendering task, the normal player starts to execute the rendering task. The timestamp of the first video frame rendered by the normal player is later than the timestamp of the last video frame played by the electronic device before switching to the normal playback mode.

[0048] In this application, when switching from paused to play mode in normal playback mode, the normal player is triggered to take over the rendering task. The timestamp of the first video frame rendered by the normal player is later than the timestamp of the last video frame played by the electronic device before switching to the normal playback mode, which can avoid the phenomenon of screen bounce.

[0049] In a second aspect, an electronic device is provided, including a processor, which is configured to invoke a computer program stored in a memory to implement the method of any one of the first aspects of this application.

[0050] Thirdly, a chip system is provided, including a processor coupled to a memory, wherein the processor executes a computer program stored in the memory to cause an electronic device to implement the method of any one of the first aspects of this application.

[0051] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when computer instructions are executed on an electronic device, causes the electronic device to implement the method of any one of the first aspects of this application.

[0052] Fifthly, embodiments of this application provide a computer program product that, when run on a device, causes the electronic device to execute the method of any one of the first aspects of this application.

[0053] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0054] Figure 1 is a schematic diagram of a hardware structure of an electronic device provided in an embodiment of this application;

[0055] Figure 2 is a schematic diagram of the interface of a normal playback mode and a frame playback mode provided in an embodiment of this application;

[0056] Figure 3 is a schematic diagram showing the relationship between the playback order and decoding order of video frames in a video frame group according to an embodiment of this application;

[0057] Figure 4 is a schematic diagram of the switching process between normal playback and frame playback in a normal playback mode according to an embodiment of this application.

[0058] Figure 5 is a timing diagram of the switching process shown in Scheme 1 provided in the embodiments of this application;

[0059] Figure 6 is a timing diagram of the switching process shown in Scheme 2 provided in the embodiments of this application;

[0060] Figure 7 is a schematic diagram of the switching process between normal playback and frame playback in the normal playback mode provided by an embodiment of this application;

[0061] Figure 8 is a timing diagram of the switching process shown in Scheme 3 provided in the embodiments of this application;

[0062] Figure 9 is a timing diagram of the switching process shown in Scheme 4 provided in the embodiments of this application. Detailed Implementation

[0063] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limiting purposes, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details.

[0064] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0065] It should also be understood that in the embodiments of this application, "one or more" refers to one, two, or more; "and / or" describes the relationship between the associated objects, indicating that three relationships can exist; for example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following associated objects have an "or" relationship.

[0066] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," "fourth," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0067] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0068] This application provides a video playback method that can be applied to electronic devices. These electronic devices can be tablets, mobile phones, wearable devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), etc. This application does not limit the specific type of electronic device.

[0069] Figure 1 shows a schematic diagram of the structure of an electronic device. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0070] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0071] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0072] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0073] Internal memory 121 can be used to store computer executable program code, including instructions. Processor 110 executes various functional applications and data processing of electronic device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function (such as image playback). Touch sensor 180K, also called a "touch panel," can be disposed on display screen 194. Touch sensor 180K and display screen 194 together form a touch screen, also called a "touch screen." Touch sensor 180K is used to detect touch operations applied to or near it. Touch sensor can transmit the detected touch operation to application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be disposed on the surface of electronic device 100, in a different location than display screen 194.

[0074] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0075] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a minimized display, a microLED, a micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0076] This application does not specifically limit the structure of the execution entity of a video playback method. As long as communication can be performed according to the video playback method provided by this application by running code containing the code of this application's video playback method, it is acceptable. For example, the execution entity of the video playback method provided by this application can be a functional module in an electronic device capable of calling and executing programs, or a communication device applied in an electronic device, such as a chip.

[0077] Users can use the gallery app on their electronic devices to view various photos stored on the device and play various videos stored on the device.

[0078] Referring to Figure 2, it is a schematic diagram of the interface for playing videos stored on an electronic device through a gallery application according to an embodiment of this application.

[0079] Referring to Figure 2(a), a grid of images and videos stored on an electronic device is displayed in the gallery application of the electronic device. The grid can display images and videos, wherein the grid containing a video displays the cover and duration of the video, and the grid containing an image displays a thumbnail of the image.

[0080] The grid image can be the interface displayed by the Gallery app after the user clicks the Gallery app icon on the system desktop to open the Gallery app, or it can be the interface displayed by the Gallery app after the user clicks the Gallery app icon on the system desktop to open the Gallery app and triggers more steps.

[0081] When displaying images and / or videos stored on an electronic device, the grid chart can be presented in various ways. For example, the way shown in Figure 2(a) is to display images and videos in a photo group in reverse chronological order. In practical applications, the gallery application can also set multiple object groups. For example, the control 20 shown in Figure 2(a) can be used to view multiple groups in the album: image group, video group, screenshot and screen recording group, etc. Users can also display images through the image group in the gallery application, videos through the video group in the gallery application, and screenshots and screen recordings through the screenshot and screen recording group in the gallery application. Each group can also display the corresponding images and / or videos in the grid chart format shown above.

[0082] This application does not limit the number of user operations required from the system desktop to the display of the grid chart in the gallery application, nor does it limit the way the grid chart displays images and videos.

[0083] Referring to Figure 2(b), after a user clicks on the cover of video A in the grid diagram shown in Figure 2(a), the electronic device displays the video playback interface for video A in response to this operation. This video playback interface includes a video playback area 21 and a progress display area 22.

[0084] The video playback area 21 is used to display the video frame during video playback. After entering the video playback interface from the grid interface, the control 211 is displayed in the video playback area. When the user clicks the control 211, the electronic device responds to the click operation on the control 211 and starts playing the video frame in the video playback area 21 from the 0th second. When the video is paused, the control 211 is also displayed in the video playback area 21. When the user clicks the control 211, the electronic device responds to the click operation on the control 211 and starts playing the video frame in the video playback area 21 from the current paused moment.

[0085] The progress display area 22 is used to display the video playback progress. The progress display area 22 includes a progress display frame 221 and a progress indicator line 222. The position of the progress indicator line 222 can remain unchanged. The user can drag the progress display frame 221 located below the progress indicator line 222 so that the progress display frame 221 moves left or right without moving the progress indicator line 222. The position of the progress indicator line 222 above the progress display frame indicates the current playback progress of the video.

[0086] As an example, the total playback duration of the current video is 15 seconds, and the total length of the current video's progress display frames is represented by L pixels. Based on these two parameters, we can obtain the playback duration corresponding to a unit length (or the length corresponding to a unit of time). For example, divide the playback duration 15 by the total length L. Therefore, if the leftmost edge of the progress display frame (the position corresponding to the first pixel) corresponds to second 0 of the video, then different positions of the progress indicator line on the progress display frame will correspond to different playback times. For example, if the distance between the position of the progress indicator line on the progress display frame and the leftmost edge of the progress display frame is X, then the playback time corresponding to the progress indicator line is: X*15 / L.

[0087] For ease of subsequent description, the embodiments in this application are described in the manner of "the time corresponding to the progress indicator line".

[0088] Due to space limitations in the progress display area 22, the progress display area 22 can display a portion of the video frames in the progress display frame 221 or a portion of a portion of a video frame. As shown in Figure 2(b), the progress display frame 221 displayed in the progress display area 22 includes: thumbnails of three complete video frames and a portion of a thumbnail of one video frame.

[0089] The number of thumbnails displayed in the progress bar is usually related to the video length. Generally, the longer the video, the more thumbnails are used to display the video's playback progress (thumbs up in the progress bar frames), and the shorter the video, the fewer thumbnails are used to display the video's playback progress (thumbs up in the progress bar frames).

[0090] Typically, the video playback area 21 displays the video frame corresponding to the time indicated by the progress indicator line 222 in the progress display area 22.

[0091] In a specific implementation, there is a situation where the time corresponding to the progress indicator line 222 is the time between the playback times of two adjacent video frames, that is, there is no video frame in the video file corresponding to the time of the progress indicator line 222.

[0092] As an example, in a video frame sorted by playback time (denoted as timestamp) from smallest to largest, the playback time of the i-th video frame is ti, and the playback time of the (i+1)-th video frame is t(i+1); the time t corresponding to the progress indicator line 222 is greater than ti and less than t(i+1). In this case, the video frame corresponding to the timestamp with the shortest time interval to the time t corresponding to the progress indicator line 221 can be taken as the video frame corresponding to the time of the progress indicator line 222.

[0093] Of course, in practical applications, if there are no video frames in the video file whose timestamps and progress indicator lines correspond to the same time, the video frame preceding time t (the video frame corresponding to ti) of progress indicator line 222 can be used as the video frame corresponding to time t of progress indicator line 221; or the video frame following time t (the video frame corresponding to t(i+1)) of progress indicator line 222 can be used as the video frame corresponding to time t of progress indicator line 222. In practical applications, the progress display area 22 can display the playback progress in many ways. For example, it can also display the video playback progress using a progress bar or a progress ring.

[0094] This application provides two playback modes for video playback: normal playback and frame-by-frame playback. The embodiments of this application use Figure 2 as an example to illustrate the difference between normal playback and frame-by-frame playback.

[0095] Referring to Figure 2(c), this is an interface diagram of the electronic device responding to the user's click on control 211 (which can also be referred to as a playback control) in the interface shown in Figure 2(b), during the playback of video A. This interface diagram indicates that the video A has been played to the 5th second, and the video frame corresponding to the 5th second is displayed in the video playback area 21. At the same time, the progress display frame 221 in the progress display area 22 moves to the position of the progress indicator line 222, so that the time corresponding to the 5th second is displayed. The playback process shown from Figure 2(b) to Figure 2(c) is the normal playback process of the video.

[0096] A video file consists of multiple video frames, each with a timestamp. The timestamp records the playback time of each video frame during the video file's playback process. For example, arranged by timestamp from smallest to largest, the video frames in the file are: Video Frame 0 (timestamp t0), Video Frame 1 (timestamp t1), Video Frame 2 (timestamp t2), Video Frame 3 (timestamp t3), Video Frame 4 (timestamp t4), Video Frame 5 (timestamp t5), Video Frame 6 (timestamp t6), Video Frame 7 (timestamp t7)... Normal playback involves displaying each video frame sequentially in the playback area according to its corresponding timestamp, from smallest to largest.

[0097] As an example of a normal playback process, video frame 0 is displayed in video playback area 21 at time t0 after playback begins; video frame 1 is displayed in video playback area 21 at time t1 after playback begins; video frame 2 is displayed in video playback area 21 at time t2 after playback begins; video frame 3 is displayed in video playback area 21 at time t3 after playback begins; video frame 4 is displayed in video playback area 21 at time t4 after playback begins; video frame 5 is displayed in video playback area 21 at time t5 after playback begins; video frame 6 is displayed in video playback area 21 at time t6 after playback begins; video frame 7 is displayed in video playback area 21 at time t7 after playback begins…

[0098] From Figure 2(b) to Figure 2(c), the normal playback process of video A is shown: starting from the video frame at second 0 of video A, each video frame is displayed sequentially according to the playback time indicated by the timestamp; Figure 2(c) is a schematic diagram of the interface displaying the video frame up to second 5.

[0099] Referring to Figure 2(d), the user drags the progress display frame of the interface shown in Figure 2(c). In response to the dragging operation shown in Figure 2(d), the electronic device displays the screen corresponding to the dragging operation in the video playback area.

[0100] As an example, referring to Figure 2(e), when the user drags the progress bar to display frames such that the time corresponding to the progress indicator line is 14 seconds, the video playback area displays the video frame corresponding to the 14th second. The video frame corresponding to the 14th second can be the video frame in video A with the timestamp of 14 seconds; it can also be the video frame with the timestamp closest to 14 seconds; it can also be the video frame before 14 seconds whose time interval between the timestamp and 14 seconds is closest; or it can be the video frame after 14 seconds whose time interval between the timestamp and 14 seconds is closest.

[0101] It should be noted that, in subsequent embodiments, the specific meaning of the description of "video frame corresponding to a certain moment" can be referred to the detailed explanation of "video frame corresponding to the 14th second," and will not be repeated hereafter. The playback process shown from (d) in Figure 2 to (e) in Figure 2 is the video frame playback process.

[0102] As the user drags the progress display frame 221, a relative position is formed between the progress indicator line 222 and the progress display frame 221. The position of the progress indicator line 222 on the progress display frame 221 is related to the time corresponding to the progress indicator line 222.

[0103] In the specific implementation, assuming that the time indicated by the progress indicator line 222 is t1 before the user moves in the area where the progress display frame 221 is located on the screen. When the user moves left or right in the area where the progress display frame is located, the electronic device can detect the user's real-time touch position at certain time intervals. The electronic device determines the user's movement direction and distance based on the continuous real-time touch positions. The electronic device determines the time change Δt based on the user's movement distance; and determines the calculation method for real-time time t2 based on the user's movement direction. For example, if the movement direction is left, the starting time t1 is subtracted from the time change Δt to obtain the real-time time t2; if the movement direction is right, the starting time t1 is added to the time change Δt to obtain the real-time time t2. After obtaining the real-time time t2, the electronic device finds the video frame corresponding to the real-time time t2 and displays the video frame corresponding to t2 in the video playback area.

[0104] On the other hand, the electronic device obtains the movement direction of the progress display frame based on the movement direction determined by the real-time touch point position, and determines the movement distance of the progress display frame based on the movement distance of the real-time touch point position; the electronic device moves the progress display frame in the progress display area according to the movement direction and movement distance of the progress display frame, so that the time corresponding to the progress indicator line is t2.

[0105] As another example, the electronic device also determines the moving direction (left if t2 is greater than t1, right if t2 is less than t1) and moving distance of the progress display frame based on the start time t1 and the real time t2; the electronic device moves the progress display frame in the progress display area according to the moving direction and moving distance of the progress display frame, so that the time corresponding to the progress indicator line is t2.

[0106] It should be noted that the above process is only an example. In actual applications, there are multiple ways for the user to determine the time corresponding to the progress indicator line 222, the position of the progress display frame 221, and the video frame displayed in the video playback area 21 while dragging the progress display frame 121.

[0107] Based on the above frame playback process, it can be understood that the process of the user dragging the progress to display the frame is an uncertain process. It may move to the left, it may move to the right, it may be dragged quickly, or it may be dragged slowly. Accordingly, the video frame images displayed in the video playback area 21 have randomness.

[0108] As an example, before the user moves within the area of ​​the progress display frame on the screen, the starting time indicated by the progress indicator is t1. During the user's dragging of the progress display frame, the electronic device detects the user's real-time touch position at regular time intervals, thus obtaining multiple consecutive time periods corresponding to the real-time times: t2, t3, t7, t4, and t3. In the video playback area, video frames are played in the following order: starting with the video frame corresponding to time t1, the video frames corresponding to time t2, t3, t7, t4, and t3 are played sequentially.

[0109] It can be understood that the process of a user clicking control 21 to trigger the electronic device to play video in the video playback area is normal playback; the process of a user dragging the progress display frame to trigger the electronic device to display the video frame corresponding to the progress indicator line in the video playback area is frame playback.

[0110] The example shown in Figure 2 is used to illustrate the difference between normal playback and frame playback, and does not mean that the interface during normal playback and frame playback must be the same as shown in Figure 2.

[0111] Of course, in practical applications, there may be both play and pause states in normal playback mode.

[0112] As an example, in the normal playback mode of video A, if the user drags the progress display frame and then releases it from the area where the progress display frame is located, the electronic device continues to play video A in normal playback mode.

[0113] As another example, in the paused state of normal playback mode of video A, if the user drags the progress display frame and then releases it from the area where the progress display frame is located, the electronic device is currently in normal playback mode, but video A is in the paused state of normal playback mode.

[0114] In the above embodiments, whether it is normal playback or frame playback, the video frames need to be decoded before they are displayed in the video playback area. Usually, the video frames are decoded in groups. A group of video frames constitutes a video frame group, and each group of video frames includes a keyframe (I-frame), at least one forward reference frame (P-frame), and at least two bidirectional reference frames (B-frames).

[0115] Referring to Figure 3, it is a schematic diagram of the playback order and decoding order of a set of video frames provided in an embodiment of this application. A set of video frames can also be referred to as a group of pictures (GOP).

[0116] Each group of video frames, in playback order, includes: I-frame, B-frame, B-frame, P-frame, B-frame, B-frame, and P-frame. Among these, the I-frame, as a keyframe, can be decoded to obtain a complete image, but decoding is slower. The P-frame records the changes relative to the previous frame (not a B-frame); information from the previous frame is required to decode the P-frame. Similarly, information from previous P-frames (which may also be I-frames) and subsequent P-frames is needed to decode the B-frame. P-frames and B-frames improve video smoothness while reducing the file size.

[0117] Whether it's an I-frame, P-frame, or B-frame, they are all video frames. When decoding a video frame at a specific moment, you don't directly decode the video frame at that moment. Instead, you decode the group of video frames that the video frame at that moment belongs to. When decoding video frames within a group of video frames, you decode them one by one in the decoding order.

[0118] As an example, if we need to decode the video frame at time t1, and the video frame at time t1 is the 6th video frame in the playback order of a group of video frames: frame B; then the decoding order is:

[0119] First decoding: the video frame played in the first playback order: I-frame;

[0120] The second decoding: the 4th video frame in playback order: P-frame;

[0121] The third decoding step: the second video frame played in the playback order: frame B;

[0122] The fourth decoding: the third video frame in playback order: frame B;

[0123] The fifth decoding: the 7th video frame in playback order: P-frame;

[0124] The sixth decoding: the 5th video frame in playback order: frame B;

[0125] The seventh decoding: the 6th video frame in playback order: frame B;

[0126] Therefore, when decoding the video frame at time t1, it is necessary to perform the seventh decoding operation to decode the video frame at time t1.

[0127] Currently, in normal playback scenarios, the video frames in each video frame group are parsed, decoded, rendered, and displayed sequentially according to the playback order. Of course, when decoding the video frames in each video frame group, each video frame is decoded sequentially according to the decoding order shown in Figure 6. Typically, in continuous playback scenarios, when displaying video frames from the previous video frame group, all video frames in the next video frame group have already been decoded. However, when switching from frame playback mode to normal playback mode, it is necessary to determine the video frame group at the time of the latest frame playback, and then decode the video frames sequentially according to the aforementioned decoding order.

[0128] In a frame playback scenario, the process of a user dragging the progress bar to display a frame is uncertain. The user may drag the progress bar to the left, or to the right, or drag it quickly or slowly. Consequently, the video frame images displayed in the video playback area 11 are random. Therefore, during the process of the user dragging the progress bar to display a frame, the electronic device needs to determine the moment the progress indicator line points based on the real-time detected touch point position, and then decode the video frames sequentially according to the above decoding order.

[0129] Both normal playback and frame-by-frame playback have their own dedicated players. For example, a normal player is used to parse, decode, and render video frames during normal playback, while a frame-by-frame player is used to parse, decode, and render video frames during frame-by-frame playback. The parsing and decoding processes for normal playback and frame-by-frame playback are completely independent. However, the normal player and frame-by-frame player need to hand over the rendering work of video frames. After each handover, they need to re-parse, decode, and render based on the timestamp of the last video frame in the previous switching mode. Therefore, whether switching from normal playback to frame-by-frame playback or vice versa, it is necessary to first decode the video frame group containing the video frame corresponding to the transmitted timestamp. In practical applications, in order to respond to the switching process as quickly as possible and display the video frame of the latest switching scene in the preview area as soon as possible, it may not be possible to guarantee that the timestamps of the video frames before and after the switch are completely consistent.

[0130] As an example, in order to respond quickly to the switching process, the keyframe (the first decoded video frame) in the group of video frames may be used as the first video frame to be displayed after the switch.

[0131] As another example, since the decoding thread and the rendering thread are different threads, the decoding thread caches each video frame in the cache space after decoding it. When the rendering thread needs to render a video frame, it selects the video frame in the cache space that is closest to the timestamp as the first video frame to be displayed after the switch.

[0132] Of course, in practical applications, other methods may be used to determine the first video frame to be displayed after the switch in order to respond to the switching process as quickly as possible. Regardless of the method used, the screen may change abruptly before and after the switch.

[0133] Refer to Figure 4 for a schematic diagram of the process of switching between normal playback and frame playback during the playback of video A in normal playback mode.

[0134] While playing video A in normal playback mode, the user can trigger the electronic device to switch from normal playback mode to frame playback mode using a swipe gesture. Swipe gestures include pressing, dragging, and releasing. When the electronic device detects a finger pressing, it switches from normal playback to frame playback; when it detects a finger releasing, it switches back to normal playback.

[0135] For ease of distinction, in this embodiment of the application, Ti represents macroscopic time, and ti represents the playback moment within video A during the playback process of video A. For example, ti can be used to represent the timestamp of a video frame.

[0136] In Option 1, when the electronic device detects a press operation, it switches from normal playback to frame playback, and frame playback immediately takes over the rendering work; when the electronic device detects a release operation, it switches from frame playback to normal playback, and normal playback immediately takes over the rendering work.

[0137] In Option 2, when the electronic device detects a press operation, it switches from normal playback to frame playback. Frame playback temporarily does not take over the rendering work. When the electronic device detects a swipe operation, frame playback takes over the rendering work again. When the electronic device detects a release operation and switches from frame playback to normal playback, normal playback immediately takes over the rendering work. At the same time, in order to make the screen smooth during the switching, the first video frame rendered by normal playback can be the video frame after the last video frame of frame playback.

[0138] Scheme 1 and Scheme 2 are described in detail below.

[0139] Refer to Scheme 1 in Figure 4:

[0140] Before time T0, the electronic device plays video A in normal playback mode. During this process, the normal player performs rendering. Refer to S101 to S106 in Figure 5 for the normal playback process. This normal playback process takes the click operation on the playback control shown in Figure 2(b) as an example, playing video A in normal playback mode starting from second 0 of video A. During this process, the normal player performs rendering.

[0141] At time T0, the electronic device detects a user's press operation on the progress display frame in the progress display area, specifically as shown in Figure 2(d). Refer to S201 to S208 in Figure 5. The electronic device switches from normal playback to frame playback, pausing the normal player's parsing, decoding, and rendering of video A; and sends t1 to the frame player, where t1 is the moment of the last displayed video frame before switching from normal playback to frame playback, for example, the 5th second as shown in Figure 2(d). The frame player takes over the rendering work, performing decoding and rendering of the video frame at time t1. As mentioned earlier, when switching to frame playback, it is difficult to precisely control the currently playing video frame at time t1 for faster response. Specifically, the video frame group containing video frame t1 can be parsed first to obtain the H.264 format video frames in that group. Then, the video frames in that group are decoded in the decoding order to obtain YUV data. Typically, the decoder stores the YUV data of each video frame in a cache space after decoding it. After the rendering thread takes over the rendering work, it retrieves the video frame with the smallest difference from t1 from the cache space (which stores the YUV data of the currently successfully decoded video frames) for rendering. The time corresponding to the rendered video frame can be recorded as time t2. Time t2 may be earlier or later than time t1. Time t2 and time t1 are in the same video frame group. From the user's perspective, when the user presses down, the screen jumps from video frame t1 to video frame t2, which may cause screen jumps when the finger is pressed down but not slid.

[0142] Between time T1 and time T2, the user drags their finger, and the dragging operation can be seen in the process illustrated in Figures 2(d) to 2(e). The electronic device determines the playback time based on the real-time position of the finger and transmits the playback time to the frame player. The frame player then performs video frame decoding and rendering based on the playback time and displays the video frame corresponding to the playback time in the preview area. This process can be seen in S301 to S306 of Figure 5. During frame playback, the timestamp t4 of the displayed video frame may not be completely consistent with the time t3 transmitted by the drag position. However, since the frame playback process itself is not perfectly smooth depending on the speed of the user's dragging operation, the difference in user experience is not significant in practical applications. Time t3 and timestamp t4 are only examples; in practical applications, multiple video frames will be displayed as the user drags.

[0143] At time T2, the user stops dragging their finger, and video frame t5 is displayed.

[0144] At time T3, the electronic device detects that the user's finger has been released from the progress display area, as shown in S401 to S406 of Figure 5. The electronic device switches from frame playback to normal playback. When sending a playback command to the normal player, it carries the latest frame playback time t5. The normal player parses the video frame group containing the t5 video frame to obtain the H.264 format video frame; it decodes the video frame to obtain YUV data; as mentioned earlier, the rendering thread selects the video frame with the smallest time difference from t5 from the decoded video frames to start rendering the video image, and plays it sequentially from that video frame. The time corresponding to the first rendered video frame is time t6. The time of t6 may be earlier or later than t5. When t6 is earlier than t5, it may cause the image to jump back and then play in sequence when the finger is released.

[0145] As mentioned earlier, when a user presses down with their finger, the video frame may jump from frame t1 to frame t2, causing the viewer's perspective to change before the user's finger has even moved. When the user releases their gesture, the video frame jumps from frame t5 to frame t6, and then plays back sequentially from frame t6, causing the viewer's perspective to change after the user releases their finger, resulting in the video not playing in sequence.

[0146] In practical applications, other scenarios may arise. For example, in normal playback mode, the user presses their finger but doesn't drag it; instead, they hold it for a while before releasing it. This scenario will not be described using a timing diagram. The process of switching from normal playback to frame playback and vice versa can be seen in Figure 5.

[0147] Between time T3 and time T4, the electronic device plays video A in normal playback mode, during which the normal player performs rendering work.

[0148] At time T4, the electronic device detects a user's press operation on the progress display frame in the progress display area. The electronic device switches from normal playback to frame playback, pausing the normal player's parsing, decoding, and rendering of video A. It then sends t7 to the frame player, where t7 is the last video frame displayed before switching from normal playback to frame playback. The frame player immediately takes over the rendering work, parsing the video frame group containing the t7 video frame to obtain H.264 format video frames. It decodes the video frames in this video frame group to obtain YUV data. As mentioned earlier, the rendering thread renders the video frame with the smallest time difference from t7 among the decoded video frames, obtaining the video frame of that video frame. The time corresponding to this video frame is time t8. Time t8 may be earlier or later than time t7, causing a screen jump when the finger is pressed but not slid.

[0149] Between time T4 and time T5, the electronic device displays the video frame at time t8.

[0150] At time T5, the electronic device detects that the user's finger has been released from the progress display area. The electronic device switches from frame playback to normal playback, and the normal player takes over the rendering work. When sending the playback command to the normal player, the device carries the latest frame playback time t8. The normal player parses the video frame group containing the t8 video frame to obtain the H.264 format video frame; decodes the video frame to obtain YUV data; selects the video frame with the smallest time difference from t8 from the decoded video frames to start rendering the video image, and plays it sequentially from that video frame. The time corresponding to this video frame is time t9. Time t9 may be earlier or later than time t8. When time t9 is earlier than time t8, it causes the image to jump back and then play in sequence when the finger is released.

[0151] This application embodiment can update the timing of the frame player taking over the rendering work from when the finger is pressed to when the finger starts to drag. This can avoid the screen jump from video frame t1 to video frame t2 in Scheme 1. Even if the frame player takes over the rendering work when the finger is dragged and the screen changes, from the user's perspective, it will be considered as a normal screen change caused by dragging, which is more user-friendly.

[0152] If the timing of the frame player taking over the rendering work is updated from when the finger is pressed to when the finger starts to drag, it can also avoid the handover of rendering work between normal playback and frame playback in the scenario where the user presses but does not drag and then releases the finger. This avoids the screen jump from t7 video frame to t8 video frame and the screen jump back from t8 video frame to t9 video frame, which is more user-friendly.

[0153] In addition, when switching from frame playback to normal playback, the normal player can be configured to select a video frame whose time is later than the frame playback time from the decoded video frames as the first video frame to be rendered, thereby avoiding the phenomenon of screen jump and improving user-friendliness.

[0154] The improvements described above are explained in detail below using Scheme 2.

[0155] Refer to Scheme 2 in Figure 4:

[0156] Before time T0, the electronic device plays video A in normal playback mode. During this process, the normal player performs rendering. Refer to S101 to S106 in Figure 6 for the normal playback process. During this process, the normal player performs rendering.

[0157] At time T0, the electronic device detects a user's press operation on the progress display frame in the progress display area. Refer to S201 to S206 in Figure 5. The electronic device switches from normal playback to frame playback. The electronic device pauses the normal player's parsing, decoding, and rendering of video A; and sends a playback start command to the frame player, carrying a timestamp t1, where t1 is the moment of the last video frame displayed before switching from normal playback to frame playback. Upon receiving the frame playback start command, the frame player temporarily suspends its rendering work, while the normal player continues to take over the rendering work. Therefore, as shown in S206 of Figure 6, the frame player parses the video frame group containing video frame t1 to obtain H.264 format video frames; it decodes the video frames in this video frame group to obtain YUV data, but does not trigger the rendering and display of new video frames. Thus, the screen jump from video frame t1 to video frame t2 in Scheme 1 is avoided.

[0158] At time T1 (or shortly thereafter), the user begins to drag their finger. The electronic device detects the first touch point after the initial touch point of the drag operation (the touch point corresponding to the press operation), and determines the timestamp t3 based on the first touch point. It can then send a playback command carrying the timestamp t3 to the frame player via the decision module. After receiving the frame playback command carrying time t3, the frame player, if it has not yet taken over the rendering work, will begin to take over the rendering work; if it has already taken over the rendering work, it does not need to repeat the takeover. The frame player first determines whether the video frame at time t3 is in the video frame group decoded in S206 of Figure 6. If it is, then... The video frame at time t3 is directly retrieved from the cache space for rendering. If the video frame at time t3 has not yet been successfully decoded when retrieved from the cache space, the video frame with the smallest time difference from t3 can be retrieved from the cache space for rendering. If the video frame at time t3 is not in the video frame group decoded in S206 of Figure 6, the video frame group containing the video frame at time t3 is parsed to obtain an H.264 format video frame. The video frames in this video frame group are decoded to obtain YUV data. The video frame with the smallest time difference from t3 is retrieved from the cache space and rendered to obtain the video image. In this embodiment, if the video image obtained in this step is not at time t3, it is recorded as the video image at time t3', and the video image at time t3' is a video frame in the video frame group containing the video frame at time t3. Accordingly, the drag operation displays either the video image at time t3 or the video frame at time t3'. Referring to Figure 4, at time T1 (or within a very short time afterward), the video frame switches from video frame t1 to video frame t3 or t3'. Although the image changes, this is precisely the process of the user's finger starting to drag. During frame playback, the image itself is not a very continuous process. Therefore, from the user's perspective, the video frame at time t3 or t3' displayed in this process is the video image corresponding to the position of the user's latest finger touch point, which is more user-friendly.

[0159] The playback process between time T1 and time T2 is frame-by-frame, which will not be described in detail here.

[0160] At time T2, the user stops dragging their finger, and video frame t5 is displayed.

[0161] At time T3, the electronic device detects that the user's finger has been released from the progress display area, as shown in S401 to S406 of Figure 6. The electronic device switches from frame playback to normal playback. When sending the playback start command to the normal player, it carries the latest frame playback time t5. The normal player parses the video frame group containing the t5 video frame to obtain the H.264 format video frame; it decodes the video frame to obtain YUV data. In the second scheme, the rendering thread starts rendering the video frame from the t5 video frame or subsequent video frames (video frames with timestamps later than t5). This video frame can be recorded as t10 video frame. The normal playback process starts from the t10 video frame and plays sequentially. This is equivalent to starting the normal playback process from the last video frame displayed in frame playback according to the playback order, avoiding the screen jump from the t5 video frame to the t6 video frame (which may be earlier than t6), resulting in a smooth switching process and a better user experience.

[0162] Option 2 can be understood as follows: when switching from normal playback to frame playback, the timing of frame playback taking over the rendering work is changed from pressing the finger to dragging the finger. This avoids the screen jump when the finger is pressed but not dragged. The screen is not continuous when the finger is dragging, thus hiding the screen jump phenomenon when the frame playback takes over the rendering work, which is more user-friendly.

[0163] In addition, when switching from frame playback to normal playback, the settings are set to start rendering from the latest video frame that was displayed during frame playback, avoiding the screen jump phenomenon that may occur when switching from t5 to t6, thus improving user-friendliness.

[0164] In normal playback mode, the user presses their finger but does not drag it; instead, they hold it for a short time before releasing it. This scenario is not described using a timing diagram; the process of switching from normal playback to frame playback and vice versa can be seen in Figure 6.

[0165] Between time T3 and time T4, the electronic device plays video A in normal playback mode, during which the normal player performs rendering work.

[0166] At time T4, the electronic device detects a user press in the progress display area and switches from normal playback to frame-by-frame playback. The electronic device pauses the normal player's parsing, decoding, and rendering of video A. However, rendering is still managed by the normal player; therefore, frame-by-frame playback no longer parses, decodes, or renders new video frames. Assuming that normal playback was playing the video frame at time T7 when the user pressed the button, the video playback area effectively maintains the display of the video frame at time T7, preventing any screen jumps and improving user experience.

[0167] Between time T4 and time T5, the electronic device continues to display the video frame at time t7.

[0168] At time T5, the electronic device detects that the user's finger has been released from the progress display area. The electronic device switches from frame playback to normal playback. When sending the playback start command to the normal player, at time t7, which carries the latest frame playback, the normal player itself is still taking over the rendering work. Therefore, it continues the previously paused parsing, decoding, and rendering work and starts playing the video frame of the next playback time (time t11) of the video frame at time t7. Since the video frame t10 itself is the next video frame in the playback sequence of the video frame at time t7, the switching process plays the screen in sequence, which is relatively smooth and provides a better user experience.

[0169] In practical applications, even when video A is paused in normal playback mode, it is possible to switch between normal playback and frame playback.

[0170] Referring to Figure 7, it is a schematic diagram of the process of switching between normal playback and frame playback when video A is paused in normal playback mode, according to an embodiment of this application.

[0171] During the playback of video A in normal playback mode, the user can pause playback of video A by clicking on the video preview area. At this point, the device is in normal playback mode, but video A is paused. Then, the user can use a swipe gesture to switch the electronic device from normal playback mode to frame playback mode to play video A. Swipe gestures include pressing, dragging, and releasing. When the electronic device detects a finger pressing, it switches from normal playback to frame playback; when it detects a finger releasing, it switches back to normal playback.

[0172] Since the video is paused in normal playback mode in this scenario, normal playback does not need to perform rendering when switching between frame playback and normal playback. Therefore, it is possible to set the rendering work to be taken over by frame playback, and then normal playback will not take over the rendering work when the finger is released. Instead, the normal player will take over the rendering work when the user receives a click operation on the playback control in the video playback area, which triggers the electronic device to switch the video from paused to playing in normal playback mode.

[0173] In Option 3, when the electronic device detects a press operation, it switches from normal playback to frame playback, and frame playback immediately takes over the rendering process. When the electronic device detects a release operation, it switches from frame playback to normal playback, and normal playback immediately takes over the rendering process. The difference from Option 1 is that after the gesture is released, the video is paused in normal playback mode.

[0174] In Option 4, when the electronic device detects a press operation, it switches from normal playback to frame playback. Frame playback temporarily suspends rendering. When the electronic device detects a swipe operation, frame playback resumes rendering. When the electronic device detects a release operation and switches from frame playback to normal playback, normal playback temporarily suspends rendering, and frame playback continues to handle rendering. When the user's click on the playback control in the video playback area triggers a switch from paused to play mode in normal playback mode, the normal player takes over rendering and begins playing the video in normal playback mode. To ensure smooth transitions, the first video frame rendered in normal playback can be the video frame following the last video frame rendered in frame playback.

[0175] Scheme 3 and Scheme 4 are described in detail below.

[0176] Refer to Scheme 3 in Figure 7:

[0177] Before time T6, while the electronic device was playing video A in normal playback mode, the user triggered a pause on video A by clicking. Therefore, it is currently in normal playback mode and in a paused state. In the paused state, the normal player continues to handle the rendering work. Refer to Figure 8 for the normal playback process shown in S101 to S106 and the process of triggering the pause in normal playback shown in S501 to S504. The normal player pauses parsing, decoding, and rendering.

[0178] At time T6, the electronic device detects a user's press operation on the progress display frame in the progress display area. Referring to S201 to S208 in Figure 8, the electronic device switches from normal playback to frame playback and sends t12 to the frame player. t12 is the time of the last video frame displayed before switching from normal playback to frame playback. The frame player takes over the rendering work, performing decoding and rendering of the video frame at time t12. As mentioned earlier, when switching to frame playback, the frame player parses the video frame group containing the t12 video frame to obtain H.264 format video frames; it decodes the video frames in this video frame group to obtain YUV data; as mentioned earlier, the rendering thread renders the video frame with the smallest time difference from t12 among the decoded video frames, obtaining the video frame's video image. The time corresponding to this video frame is time t13. Time t13 may be earlier or later than time t12. Time t13 and time t12 are in the same video frame group. From the user's perspective, when the user presses their finger, the image jumps from the t12 video frame to the t13 video frame, which may cause the image to jump even when the finger is pressed but not slid.

[0179] Between time T7 and T8, the user drags their finger. The electronic device determines the playback time based on the finger's real-time position and transmits this time to the frame player. The frame player then decodes and renders the video frame according to the playback time and displays the corresponding video frame in the preview area. This process can be seen in S301 to S306 of Figure 8. During frame playback, the timestamp t15 of the displayed video frame may not be exactly the same as the time t14 transmitted based on the drag position. However, since frame playback itself is not perfectly smooth depending on the speed of the user's dragging operation, the difference in user experience is not significant in practical applications. Time t14 and timestamp t15 are just examples; in actual applications, multiple video frames will be displayed as the user drags.

[0180] At time T8, the user stops dragging their finger, and video frame t16 is displayed.

[0181] At time T9, the electronic device detects that the user's finger has been released from the progress display area. Referring to S401 to S404 in Figure 8, the electronic device switches from frame playback to normal playback. When sending the playback switch command to the normal player, at time t16 carrying the latest frame playback, the normal player determines to take over the rendering work based on the received playback switch command. However, since it is a playback switch command (only switching to normal playback, still in a paused state), the normal player takes over the rendering work but does not perform decoding, parsing, and rendering.

[0182] As mentioned earlier, when a user presses down, the video frame may jump from t12 to t13, causing the user's perspective to change before the user's finger has even moved.

[0183] In practical applications, other scenarios may arise. For example, in the paused state of normal playback mode, the user presses their finger but does not drag it; instead, they hold it for a period of time before releasing it. This scenario will not be described using a timing diagram; the process of switching from normal playback to frame playback and vice versa can be seen in Figure 8.

[0184] Between time T9 and time T10, the electronic device is in normal playback mode, but video A is paused. During this process, the normal player continues to take over the rendering work but does not perform parsing, decoding, or rendering. The latest displayed video frame is still the video frame from time T16.

[0185] At time T10, the electronic device detects a user's press operation on the progress display frame in the progress display area. The electronic device switches from normal playback to frame playback and sends t16 to the frame player. T16 is the time of the last video frame displayed before switching from normal playback to frame playback. The frame player immediately takes over the rendering work. The frame player parses the video frame group where the t16 video frame is located and obtains the H.264 format video frame. It decodes the video frames in the video frame group to obtain YUV data. As mentioned earlier, the rendering thread renders the video frame with the smallest time difference with t16 among the decoded video frames, and obtains the video frame of the video frame. The time corresponding to this video frame is time t17. Time t17 may be earlier or later than time t16, which causes the screen to jump when the finger is pressed but not slid.

[0186] Between time T10 and time T11, the finger slides. This process can be referred to the description between T7 and T8, and will not be repeated here.

[0187] At time T11, the electronic device detects the user's finger being released from the progress display area. The electronic device switches from frame playback to normal playback. When sending the playback switch instruction to the normal player, it carries the latest frame playback time t18. Based on the received playback switch instruction, the normal player determines to take over the rendering work. However, since it is a playback switch instruction (it only switches to normal playback and is still in a paused state), the normal player takes over the rendering work but does not perform decoding, parsing, and rendering.

[0188] Between time T11 and time T12, the electronic device is in normal playback mode, but video A is paused. During this process, the normal player continues to take over the rendering work but does not perform parsing, decoding, or rendering. The latest displayed video frame is still the video frame from time T18.

[0189] At time T12, the electronic device detects a user's click on the playback control in the video playback area. The electronic device switches from paused to playing mode in normal playback mode, as shown in S901 to S906 of Figure 8. The normal player receives a playback start command, which carries the timestamp t18 of the latest video frame. The normal player parses the video frame group containing the t18 video frame to obtain H.264 format video frames; it decodes the video frames to obtain YUV data; as mentioned earlier, the rendering thread selects the video frame with the smallest time difference from t18 from the decoded video frames to start rendering the video image, and plays it sequentially from that video frame. The time corresponding to the first rendered video frame is t19. The time of t19 may be earlier or later than t18. When t19 is earlier than t18, it may cause a phenomenon where the image jumps back and then resumes sequential playback when the finger is released.

[0190] It should be noted that Scheme 3 is only an example of the screen transition in this scenario. In actual applications, the video frame displayed after the switch may be a different video frame.

[0191] This application embodiment can update the timing of the frame player taking over the rendering work from when the finger is pressed to when the finger starts to drag. This can avoid the screen jump from video frame t12 to video frame t13 in Scheme 3. Even if the frame player takes over the rendering work when the finger is dragged and the screen changes, from the user's perspective, it will be considered as a normal screen change caused by dragging, which is more user-friendly.

[0192] Additionally, it is possible to switch from frame playback to normal playback when the finger is released. However, normal playback does not immediately take over the rendering work. Instead, it takes over the rendering work when the user's operation triggers the electronic device to start playing the video in normal playback mode. This avoids the rendering handover at T9 and T10. If the rendering handover at T9 and T10 is avoided, it is equivalent to the rendering work being always taken over by the frame player, which can ensure the continuity of the rendered content.

[0193] Of course, when normal playback switches from paused to play and takes over the rendering work, the normal player can be set to select the video frame whose time is later than the frame playback time from the decoded video frames as the first video frame to be rendered, thereby avoiding the phenomenon of screen jump and improving user-friendliness.

[0194] The above improvements are described in detail below using Scheme 4.

[0195] Refer to Scheme 4 in Figure 7:

[0196] Before time T6, while the electronic device was playing video A in normal playback mode, the user triggered a pause on video A by clicking. Therefore, it is currently in normal playback mode and in a paused state. In the paused state, the normal player continues to handle the rendering work. Refer to Figure 9 for the normal playback process shown in S101 to S106 and the process of triggering the pause in normal playback shown in S501 to S504. The normal player pauses parsing, decoding, and rendering.

[0197] At time T6, the electronic device detects a user's press operation on the progress display frame in the progress display area. Referring to S201 to S206 in Figure 9, the electronic device switches from normal playback to frame playback and sends t12 to the frame player. t12 is the moment of the last video frame displayed before switching from normal playback to frame playback. The frame player temporarily does not take over the rendering work; rendering continues in normal playback. Therefore, the frame player does not perform rendering and display operations. This avoids the screen jump from video frame t12 to video frame t13 as in Scheme 3.

[0198] At time T7 (or shortly thereafter), the user begins to drag their finger. The electronic device detects the first touch point after the initial touch point of the drag operation (the touch point corresponding to the press operation). Based on the first touch point, it determines the timestamp t13 and can send a playback command carrying the timestamp t13 to the frame player through the decision module. After receiving the frame playback command carrying time t13, the frame player, if it has not yet taken over the rendering work, begins to take over the rendering work; if it has already taken over the rendering work, it does not need to take over the rendering work again. The frame player first determines whether the video frame at time t13 is in the video frame group decoded in S206 in Figure 6. If it is, it directly... The video frame at time t13 is retrieved from the cache space and rendered to obtain the video frame at time t13. However, if the video frame at time t13 has not yet been successfully decoded when retrieved from the cache space, the video frame with the smallest time difference from t13 can be retrieved from the cache space for rendering. If the video frame at time t13 is not in the video frame group decoded in S206 of Figure 6, the video frame group containing the video frame at time t13 is parsed to obtain an H.264 format video frame. The video frames in this video frame group are decoded to obtain YUV data. The video frame with the smallest time difference from t13 is retrieved from the cache space and rendered to obtain the video image. In this embodiment, if the video image obtained in this step is not at time t13, it is recorded as the video image at time t13', and the video image at time t13' is a video frame in the video frame group containing the video frame at time t13. Accordingly, the drag operation displays either the video image at time t13 or the video frame at time t13'. Referring to Figure 7, at time T7 (or within a very short time afterward), the video frame switches from video frame t12 to video frame t13 or t13'. Although the image changes, this is precisely the process of the user's finger starting to drag. During frame playback, the image itself is not a very continuous process. Therefore, from the user's perspective, the video frame at time t13 or t13' displayed during this process is considered to be the video frame corresponding to the latest touch point of the user's finger, which is more user-friendly.

[0199] The playback process between time T7 and time T8 is frame-by-frame, which will not be described in detail here.

[0200] At time T8, the user stops dragging their finger, and video frame t16 is displayed.

[0201] At time T9, the electronic device detects that the user's finger has been released from the progress display area, as shown in S401 to S404 of Figure 9. The electronic device switches from frame playback to normal playback. When sending the playback start command to the normal player, it carries the latest frame playback time t16. The normal player parses the video frame group containing the t16 video frame to obtain the H.264 format video frame; it decodes the video frame to obtain YUV data; however, normal playback does not take over the rendering thread, so it does not perform subsequent rendering and display. Therefore, at time T9, the t16 video frame is still displayed.

[0202] In the paused state of normal playback mode, the user presses their finger but does not drag it; instead, they hold it for a short time before releasing it. This scenario is not described using a timing diagram; the process of switching from normal playback to frame playback and vice versa can be seen in Figure 9.

[0203] Between time T9 and time T10, the electronic device is in normal playback mode, but video A is paused. The rendering work is taken over by the frame player, so the video image does not change.

[0204] At time T10, the electronic device detects the user's press operation in the progress display area and switches from normal playback to frame playback. At this time, a frame playback start command is sent. The frame player does not need to take over the rendering work again, so the frame player does not need to re-parse, decode, and render new video frames.

[0205] Between T10 and T11, the finger slides; this process can be referred to the description between T7 and T8, and will not be repeated here.

[0206] At time T11, the electronic device detects that the user's finger has been released from the progress display area. The electronic device switches from frame playback to normal playback. When sending the playback start command to the normal player, it carries the latest frame playback time t16. The normal player parses the video frame group containing the t16 video frame to obtain the H.264 format video frame; it decodes the video frame to obtain YUV data; however, normal playback does not take over the rendering thread, so it does not perform subsequent rendering and display. Therefore, at time T11, the t16 video frame is still displayed.

[0207] Between time T11 and time T12, the electronic device is in normal playback mode, but video A is paused. During this process, the frame player continues to take over the rendering work but does not perform parsing, decoding, or rendering. The latest displayed video frame is still the video frame from time T16.

[0208] At time T12, the electronic device detects the user's click operation on the playback control in the video playback area. The electronic device switches from paused to playing mode in normal playback mode, as shown in S901 to S906 in Figure 9. The normal player receives a playback start command, which carries the timestamp t16 of the latest video frame. The normal player parses the video frame group containing the t16 video frame to obtain the H.264 format video frame; it decodes the video frame to obtain YUV data; as mentioned earlier, the rendering thread selects the video frame with the smallest time difference from t16 from the decoded video frames to start rendering the video image, and plays it sequentially from that video frame. The time corresponding to the first rendered video frame is time t20. Time t20 is later than time t18, avoiding the screen jump from video frame t18 to video frame t19 (which may be earlier than t18), resulting in a smooth transition and a better user experience.

[0209] In this application embodiment, the ordinary player, frame player, and decision module are modules in the video playback engine. This application embodiment uses a gallery application to implement video playback using a video playback engine as an example to describe the switching process between ordinary playback and frame playback. In practical applications, it can also be applied to other applications, such as video editing applications and video playback applications.

[0210] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0211] This application also provides a computer-readable storage medium storing a computer program that, when run on an electronic device, can implement the steps in the above-described method embodiments.

[0212] This application also provides a computer program product that, when run on an electronic device or a wireless router, enables the electronic device to perform the steps described in the various method embodiments above.

[0213] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying the computer program code to the first device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0214] This application also provides a chip, which includes a processor coupled to a memory. The processor calls a computer program stored in the memory to implement the steps of any method embodiment of this application. The chip can be a single chip or a chip module composed of multiple chips.

[0215] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0216] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0217] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A video playback method, characterized in that, The method, applied to an electronic device including a regular media player and a frame player, includes: in a playback state of a first video in a regular playback mode, the electronic device detects a first press operation; in response to the first press operation, the electronic device switches from the regular playback mode to a frame playback mode, the regular media player stops executing the decoding and rendering tasks of the first video, and the frame player begins executing the decoding task of the first video; the electronic device detects a first drag operation; in response to the first drag operation, the frame player takes over the rendering task of the first video; after taking over the rendering task of the first video, the frame player begins executing the rendering task of the first video.

2. The method as described in claim 1, characterized in that, After the electronic device detects the first drag operation, the method further includes: the electronic device detecting a first release operation, the first release operation being the release operation corresponding to the first drag operation; in response to the first release operation, the electronic device switches the frame playback mode to a normal playback mode, and the normal player starts executing the decoding task of the first video and takes over the rendering task of the first video; after taking over the rendering task of the first video, the normal player starts executing the rendering task, wherein the timestamp of the first video frame rendered by the normal player during the rendering task is later than the timestamp of the last video frame played by the electronic device before the first release operation.

3. The method as described in claim 2, characterized in that, After the electronic device detects the first release operation, the method further includes: the electronic device detecting a second press operation; in response to the second press operation, the normal player stops executing the decoding and rendering tasks of the first video, and the frame player starts executing the decoding task of the first video; the electronic device detects a second release operation, the second release operation being the release operation corresponding to the second press operation; in response to the second release operation, the electronic device switches from frame playback mode to normal playback mode, and the normal player starts executing the decoding and rendering tasks of the first video, wherein the timestamp of the first video frame rendered by the normal player during the rendering task is later than the timestamp of the last video frame played by the electronic device before switching to normal playback mode.

4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: in the normal playback mode of the first video, the electronic device receives a first click operation; in response to the first click operation, the electronic device switches the playback state to a paused state, and the normal player stops executing the decoding and rendering tasks of the first video; in the paused state of the normal playback mode of the first video, the electronic device detects a third press operation; in response to the third press operation, the electronic device switches the normal playback mode to a frame playback mode, and the frame player starts executing the decoding task of the first video; the electronic device detects a third drag operation; in response to the third drag operation, the frame player takes over the rendering task of the first video; after taking over the rendering task of the first video, the frame player starts executing the rendering task of the first video.

5. The method as described in claim 4, characterized in that, After the electronic device detects the third drag operation, the method further includes: the electronic device detects the third release operation, which is the release operation corresponding to the third drag operation; in response to the third release operation, the electronic device switches the frame playback mode to the normal playback mode, in which the first video is paused.

6. The method as described in claim 5, characterized in that, After the electronic device detects the third release operation, the following further applies: the electronic device detects the fourth press operation; in response to the fourth press operation, the electronic device switches from normal playback mode to frame playback mode, and the frame player begins to execute the decoding task of the first video; the electronic device detects the fourth drag operation, and the frame player executes the decoding task and the rendering task.

7. The method as described in claim 6, characterized in that, After the electronic device detects the fourth drag operation, it further includes: the electronic device detects the fourth release operation, the fourth release operation being the release operation corresponding to the fourth drag operation; in response to the fourth release operation, the electronic device switches the frame playback mode to the normal playback mode, in which the first video is paused.

8. The method as described in claim 7, characterized in that, After the electronic device detects the fourth release operation, it also includes: the electronic device detects the second click operation; in response to the second click operation, the electronic device switches from frame playback mode to normal playback mode, the normal player starts to execute the decoding task of the first video, the normal player takes over the rendering task, and after the normal player takes over the rendering task, it starts to execute the rendering task. The timestamp of the first video frame rendered by the normal player when executing the rendering task is later than the timestamp of the last video frame played by the electronic device before switching to the normal playback mode.

9. An electronic device, characterized in that, It includes one or more processors and one or more memories; wherein the one or more memories are coupled to the one or more processors, and the one or more memories are used to store a computer program, which, when executed by the one or more processors, causes the electronic device to perform the method as described in any one of claims 1-8.

10. A chip system applied to an electronic device, the chip system comprising one or more processors, characterized in that, The processor is used to invoke computer instructions to cause the electronic device to perform the method as described in any one of claims 1-8.

11. A computer-readable storage medium comprising a computer program, characterized in that, When the computer program is run on an electronic device, it causes the electronic device to perform the method as described in any one of claims 1-8.

Images