A video frame writing and reading method, electronic equipment, storage medium and chip

Abstract

The application provides a video frame writing and reading method, an electronic device, a storage medium and a chip, and relates to the technical field of data processing. The method re-encodes video frames of a video as I frames and writes the I frames into a cache space. When the frames are played, the I frames are read from the cache space, decoded, rendered and displayed. In order to quickly read the I frames when the frames are played, the I frames of a video can be cached in a file B. The file path of the file B, the mapping relationship between the time stamp of the I frames and the storage position of the file B are recorded in the cache information entity of the video. When the frames are played, the I frames at the frame playing time are read according to the information recorded in the cache information entity. In order to repeatedly write and read the I frames in the file B, the file C is generated according to the mapping relationship, and the file path of the file C is recorded in the cache information entity. When the video playing application is exited, the file A is generated according to the cache information entity. When the frames of the video are played again, the cache information entity is recovered according to the file A.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and particularly relates to a video frame writing and reading method, an electronic device, a storage medium and a chip. BACKGROUND

[0002] A gallery application in an electronic device can provide a picture preview interface, and a user can view each picture stored in the electronic device through the picture preview interface of the gallery application. The gallery application also provides a video playing interface, and the user can play each video stored in the electronic device through the video playing interface in the gallery application.

[0003] The video playing interface can also show the playing progress of the video, for example, the playing progress of the video can be shown in the form of a progress bar, a progress ring or a progress display frame. A drag operation of the user on the progress bar, the progress ring or the progress display frame can trigger the electronic device to perform frame playing on the video. However, in the frame playing scenario of the video, the display of the video picture often appears to be delayed. SUMMARY

[0004] The present application provides a video frame writing and reading method, an electronic device, a storage medium and a chip, which can reduce the display delay of the video picture in the frame playing scenario.

[0005] To achieve the above object, the first aspect of the present application adopts the following technical solution:

[0006] The electronic device displays a first video playing interface through a video playing application, and the first video playing interface includes a first video frame of a first video and a first control;

[0007] After the electronic device receives a first operation on the first control, the electronic device displays a first segment of video frames obtained based on first encoding data in the first video playing interface;

[0008] In the process of displaying the first segment of video frames obtained based on the first encoding data in the first video playing interface, the electronic device writes second encoding data of the first segment of video frames into a first cache file on a disk, records a mapping relationship between a timestamp of the second encoding data and a storage position of the second encoding data in the first cache file in a memory, and the decoding efficiency of the second encoding data is higher than that of the first encoding data;

[0009] In the running process of the video playing application, the electronic device generates a second cache file according to the mapping relationship, and stores the second cache file in the disk,

[0010] In the process of exiting the running of the video playing application, the electronic device deletes the mapping relationship in the memory;

[0011] After the video playing application is started again, the electronic device obtains the mapping relationship according to the second cache file;

[0012] In a case where the operation of displaying the video frame of the first time point of the first video is received, the electronic device obtains a first storage position of the third encoding data of the first video in the first cache file from the mapping relationship, reads the third encoding data from the first cache file according to the first storage position, and displays the video frame decoded from the third encoding data in the first video playing interface, the third encoding data being the encoding data corresponding to the first time point in the second encoding data.

[0013] In the present application, the running of the video playing application being exited means that the video playing application is killed. In the normal playing process of the video (triggered by the first operation of the first control), the second encoding data (with higher decoding efficiency) of the first video frame being played is cached, and the mapping relationship between each second encoding data and the corresponding time stamp is recorded. In the subsequent frame playing, the cached video frame can be quickly read through the mapping relationship in the memory, so as to solve the phenomenon of frame freezing of the video picture.

[0014] In addition, during the running of the video playing application, the electronic device generates a second cache file according to the mapping relationship, stores the second cache file in the disk, so that the mapping relationship in the memory is saved in the disk in the form of a file. After the video playing application is started again, the electronic device can obtain the mapping relationship according to the second cache file. Therefore, in the subsequent playing scene, the video frame is decoded and displayed according to the second encoding data with higher decoding efficiency, the method of repeatedly and quickly reading the video frame is realized, and the fluency of the frame playing scene is improved.

[0015] As an example, some encoding data in the first encoding data needs to refer to other encoding data when decoding, and the decoding efficiency is low. Each encoding data in the second encoding data does not need to refer to other encoding data when decoding, and the decoding efficiency is high.

[0016] As another example, some encoding data in the first encoding data needs to refer to other encoding data when decoding, for example, the decoding of P frame and I frame needs to refer to the encoding data of I frame. The second encoding data can also be each YUV data decoded from the first encoding data.

[0017] As a way of the first aspect, the second encoding data includes encoding data of a second time point, and before the video playing application is exited, the method further includes:

[0018] In the case of receiving the operation of displaying the second time video frame of the first video, the fourth encoding data of the first video is obtained from the mapping relationship in the second storage position of the first cache file;

[0019] The fourth encoding data is read from the first cache file according to the second storage position;

[0020] The video frame decoded by the fourth encoding data is displayed in the first video playing interface, and the fourth encoding data is the encoding data corresponding to the second time in the second encoding data.

[0021] As a way of the first aspect, before the video playing application exits, the method further comprises:

[0022] The second segment video frame based on the fifth encoding data is displayed in the first video playing interface, and the sixth encoding data of the second segment video frame is written in the first cache file;

[0023] The mapping relationship between the timestamp of the sixth encoding data and the storage position of the sixth encoding data in the first cache file is recorded in the memory, and the decoding efficiency of the sixth encoding data is higher than that of the first encoding data.

[0024] In this application, before the video playing application exits, the mapping relationship is stored in the memory, and the encoding data of the first video after re-encoding is stored in the first cache file on the disk, so that the fourth encoding data corresponding to the newly played second segment video frame can be continuously cached in the first cache file, and the mapping relationship corresponding to the fourth encoding data can be cached in the memory, realizing the repeatable writing of the cached data.

[0025] As a way of the first aspect, after the video playing application is started again, the method further comprises:

[0026] The third segment video frame based on the seventh encoding data is displayed in the first video playing interface, the eighth encoding data of the third segment video frame is written in the first cache file, and the mapping relationship between the timestamp of the eighth encoding data and the storage position of the eighth encoding data in the first cache file is recorded in the memory, and the decoding efficiency of the eighth encoding data is higher than that of the seventh encoding data.

[0027] In this application, after the video playing application is started again, the mapping relationship can be restored in the memory, and the encoding data of the first video after re-encoding is stored in the first cache file on the disk, so that the sixth encoding data corresponding to the newly played third segment video frame can be continuously cached in the first cache file, and the mapping relationship corresponding to the sixth encoding data can be cached in the memory, realizing the repeatable writing of the cached data.

[0028] As a manner of the first aspect, the frame types in the first encoded data include key frames and non-key frames; and the frame types in the second encoded data are all key frames.

[0029] In the present application, the key frame is an I frame, and the non-key frame is a P frame and a B frame. Since the I frame, the P frame and the B frame in the first encoded data are all re-encoded as I frames, the I frame corresponding to each timestamp can be stored; therefore, each I frame can be decoded separately during decoding, without the need of decoding video frames in groups, so that the decoding speed can be improved, and the fluency of the video picture in the frame playing scene can be improved.

[0030] As a manner of the first aspect, the mapping relationship is recorded in a data structure of a cache information entity of the first video, and the data structure of the cache information entity records the first file path of the first cache file.

[0031] After the electronic device records the mapping relationship between the timestamp of the second encoded data and the storage position of the second encoded data in the first cache file in the memory, the method comprises:

[0032] The electronic device writes the second file path of the second cache file in the data structure of the cache information entity of the first video.

[0033] The present application can also cache the file paths of the two cache files in the memory, so as to facilitate quick reading.

[0034] As a manner of the first aspect, during the process in which the video playing application exits the running, the method further comprises:

[0035] The electronic device generates a third cache file according to the first file path of the first cache file and the second file path of the second cache file recorded in the data structure of the cache information entity of the first video.

[0036] The third cache file is stored in the disk.

[0037] The cache information entity of the first video is deleted.

[0038] In the present application, during the process in which the video playing application exits the running, the file path of the first cache file and the file path of the second cache file are needed to be generated into the third cache file and then stored in the disk, so that the third cache file can be saved for a long time, and the data in the memory is avoided from being lost due to cleaning after the video playing application exits.

[0039] As a manner of the first aspect, after the video playing application is started again, before the mapping relationship is obtained according to the second cache file, the method further comprises:

[0040] The electronic device creates a data structure of the cache information entity of the first video in memory;

[0041] The electronic device obtains a first file path of the first cache file and a second file path of the second cache file according to the third cache file;

[0042] The electronic device writes the first file path and the second file path in the data structure of the cache information entity of the first video;

[0043] The electronic device obtains the second cache file according to the second file path of the second cache file;

[0044] After the electronic device obtains the mapping relationship according to the second cache file, the method further comprises:

[0045] The electronic device writes the mapping relationship in the data structure of the cache information entity.

[0046] As a way of the first aspect, before the electronic device reads the third encoded data from the first cache file according to the first storage location, the method comprises:

[0047] The electronic device obtains a first file path of the first cache file from the cache information entity of the first video;

[0048] The electronic device obtains the first cache file according to the first file path.

[0049] As a way of the first aspect, the cache information entity of the first video is stored in a linked list, the linked list stores a correspondence between a unique identifier of the first video and a cache information entity of the first video, the linked list also stores a correspondence between a unique identifier of a second video and a cache information entity of the second video, and the cache information entity of the first video further comprises a frame number of a video frame corresponding to encoded data cached in the first cache file and a data size of the first cache file.

[0050] The application not only can save the cache information of the first video, but also can save the cache information of other videos, so that when multiple videos are played frame by frame, the picture fluency of the frame playing scene can be improved.

[0051] As a way of the first aspect, in the process of displaying the first segment of video frames obtained based on the first encoded data in the first video playing interface, the method further comprises:

[0052] For each first encoded data, the electronic device decodes the first encoded data to obtain YUV data;

[0053] The electronic device displays a video frame corresponding to the YUV data decoded from the first encoded data in the first video playing interface.

[0054] As a manner of the first aspect, the electronic device comprises, in a process of writing the second encoded data of the first segment of video frames into the first cache file:

[0055] For each first encoded data, the electronic device encodes YUV data decoded from the first encoded data to obtain second encoded data corresponding to the first encoded data.

[0056] The electronic device writes the second encoded data corresponding to the first encoded data into the first cache file.

[0057] In the present application, YUV data needs to be decoded in the process of the last ordinary playing, so the decoded YUV data is re-encoded, and each re-encoded encoded data and a timestamp are cached. No additional decoding operation is needed, and the efficiency is improved.

[0058] In a second aspect, an electronic device is provided, comprising a processor configured to invoke a computer program stored in a memory to implement the method of any one of the first aspect.

[0059] In a third aspect, a chip is provided, comprising a processor coupled with a memory, and the processor executes a computer program stored in the memory to implement the method of any one of the first aspect.

[0060] In a fourth aspect, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. When the computer program is run on an electronic device, the electronic device implements the method of any one of the first aspect.

[0061] In a fifth aspect, an embodiment of the present application provides a computer program product, when the computer program product is run on a device, the device executes the method of any one of the first aspect.

[0062] It can be understood that the beneficial effects of the above-mentioned second aspect to fifth aspect can be referred to the related description in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0063] Figure 1 A hardware structure schematic diagram of an electronic device provided by an embodiment of the present application;

[0064] Figure 2 An interface schematic diagram of playing a video in an ordinary playing manner provided by an embodiment of the present application;

[0065] Figure 3 An interface diagram for playing a video by using a frame playing method according to an embodiment of the present application;

[0066] Figure 4 An interface diagram for playing a video by using a frame playing method according to an embodiment of the present application;

[0067] Figure 5 An interface diagram for playing a video by using a frame playing method according to an embodiment of the present application;

[0068] Figure 6 An interface diagram for playing a video by using a frame playing method according to an embodiment of the present application;

[0069] Figure 7 An interface diagram for playing a video by using a frame playing method according to an embodiment of the present application;

[0070] Figure 8 An interface diagram for playing a video by using a frame playing method according to an embodiment of the present application;

[0071] Figure 9 An interface diagram for playing a video by using a frame playing method according to an embodiment of the present application;

[0072] Figure 10 An interface diagram for playing a video by using a frame playing method according to an embodiment of the present application;

[0073] Figure 11 An interface diagram for playing a video by using a frame playing method according to an embodiment of the present application;

[0074] Figure 12 An interface diagram for playing a video by using a frame playing method according to an embodiment of the present application;

[0075] Figure 13 An interface diagram for playing a video by using a frame playing method according to an embodiment of the present application;

[0076] Figure 14 An interface diagram for playing a video by using a frame playing method according to an embodiment of the present application;

[0077] Figure 15 An interface diagram for playing a video by using a frame playing method according to an embodiment of the present application;

[0078] Figure 16 An interface diagram for playing a video by using a frame playing method according to an embodiment of the present application; Figure 15An execution flow diagram of S116 in FIG. 1. DETAILED DESCRIPTION

[0079] In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular sequences of acts, techniques, etc., in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application can be practiced in other embodiments that depart from these specific details.

[0080] It should be understood that, when used in the specification and the appended claims, the terms "comprises", "comprising", "includes", "including" and the like, mean "including but not limited to", and are intended to be inclusive of the stated features, integers, steps, operations, elements, and / or components but not to the exclusion of from others.

[0081] It should also be understood that, in the embodiments of the present application, "one or more" means one, two, or more than two; "and / or" describes the association relationship of the associated objects, which means that there can be three relationships; for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after it.

[0082] In addition, in the description of the specification and the appended claims, the terms "first", "second", "third", "fourth" and the like are only used to distinguish description, and cannot be understood as indicating or implying relative importance.

[0083] In the present specification, the reference "one embodiment" or "some embodiments" and the like means that a particular feature, structure or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Therefore, the statements "in one embodiment", "in some embodiments", "in other some embodiments", "in yet some embodiments" and the like appearing in various places in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized. The terms "include", "contain", "have" and their variants mean "including but not limited to", unless otherwise specifically emphasized.

[0084] The method for writing and reading a video frame provided in the embodiments of the present application can be applied in an electronic device, which can be a tablet computer, a mobile phone, a wearable device, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), or the like. The embodiments of the present application do not limit the specific type of the electronic device.

[0085] Figure 1 A structural schematic diagram of an electronic device is shown. The electronic device 100 can include a processor 110, an external memory interface 120, an internal memory 1211, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a loudspeaker 170A, a receiver 170B, a microphone 170C, a headset interface 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, and the like. The sensor module 180 can include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, and the like.

[0086] It can be understood that the structure shown in the embodiments of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 can include more or fewer components than shown, or combine certain components, or split certain components, or different component arrangements. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0087] The processor 110 can include one or more processing units, for example: the processor 110 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units can be independent devices, or can be integrated in one or more processors. For example, the processor 110 is configured to perform the video writing and reading method in the embodiments of the present application.

[0088] The processor 110 can also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory can hold instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instructions or data again, it can be directly called from the memory. This avoids repeated access and reduces the waiting time of the processor 110, thereby improving the efficiency of the system.

[0089] The internal memory 1211 can be used to store computer executable program codes, and the executable program codes include instructions. The processor 110 performs various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 1211. The internal memory 1211 can include a program storage area and a data storage area. Among them, the program storage area can store an operating system and at least one application program required by a function (such as a sound playing function, an image playing function, etc.). The data storage area can store data created during the use of the electronic device 100 (such as a competitive value, a wake-up voiceprint, etc.).

[0090] In addition, the internal memory 1211 can include a high-speed random access memory, and can also include a non-volatile memory, for example, at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), etc. The video file, the cache file, etc. in the embodiments of the present application can be stored in the internal memory.

[0091] Touch sensor 180K, also referred to as "touch panel". Touch sensor 180K can be disposed on display screen 194, and touch screen, also referred to as "touch panel", is formed by touch sensor 180K and display screen 194. Touch sensor 180K is configured to detect a touch operation applied thereto or in the vicinity thereof. Touch sensor 180K can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In some embodiments, touch sensor 180K can also be disposed on the surface of electronic device 100, which is different from the position of display screen 194. For example, the click operation of the general play button, the drag operation of the progress display frame, and the like provided in the embodiments of the present application can be recognized by the touch sensor.

[0092] Electronic device 100 implements display functions through a GPU, display screen 194, and an application processor. The GPU is a microprocessor for image processing, which is connected to display screen 194 and the application processor. The GPU is configured to perform mathematical and geometric calculations for graphics rendering. Processor 110 can include one or more GPUs that execute program instructions to generate or change display information. For example, the process of rendering YUV data in the embodiments of the present application can be implemented by the GPU.

[0093] Display screen 194 is configured to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light-emitting diode (QLED), and the like. In some embodiments, electronic device 100 can include one or N display screens 194, where N is a positive integer greater than 1. For example, the embodiments of the present application can include a display screen 194 on the front surface of electronic device 100 and a display screen 194 on the back surface of electronic device 100. Figure 2 or Figure 3 The interfaces shown in the embodiments of the present application are displayed by the display.

[0094] A video codec is used to compress or decompress digital video. The electronic device 100 can support one or more video codecs. In this way, the electronic device 100 can play or record videos in multiple encoding formats, for example: the video codec can implement the decoding of a video frame in H.264 format, or re-encode the video frame in H.264 format into all key frames, according to the video frame writing and reading method provided in the embodiments of the present application.

[0095] The embodiments of the present application do not particularly limit the specific structure of the execution subject of the video frame writing and reading method, as long as the code of the video frame writing and reading method provided in the embodiments of the present application can be run to communicate according to the video frame writing and reading method provided in the embodiments of the present application. For example, the execution subject of the video frame writing and reading method provided in the embodiments of the present application can be a functional module in the electronic device that can call and execute programs, or a communication device applied in the electronic device, for example, a chip.

[0096] The user can view each picture stored in the electronic device through the gallery application in the electronic device, and play each video stored in the electronic device.

[0097] Referring to Figure 2 The interface diagram for playing the video stored in the electronic device through the gallery application is provided in the embodiments of the present application.

[0098] Referring to Figure 2 (a) in FIG. 1 is a system desktop of the electronic device, and the system desktop includes an icon of the gallery application. The user can click the icon of the gallery application shown in (a) in FIG. 1 to open the gallery application. Figure 2

[0099] Referring to Figure 2 (b) in FIG. 1 is a grid view of pictures and videos stored in the electronic device displayed by the gallery application of the electronic device after the user opens the gallery application; the grid view can display pictures and videos, wherein the cover of the video and the duration of the video are displayed in the grid where the video is located, and the thumbnail of the picture is displayed in the grid where the picture is located.

[0100] The grid view can be the interface view displayed by the gallery application after the user opens the gallery application by clicking the icon of the gallery application on the system desktop, or the interface view displayed by the gallery application after more steps of operation are triggered by the gallery application after the user opens the gallery application by clicking the icon of the gallery application on the system desktop.

[0101] When the grid view displays the pictures and / or videos stored in the electronic device, it can be presented in multiple ways, for example, Figure 2 ​The method shown in (b) is to display pictures and videos in reverse chronological order. In practical applications, the gallery application can also set up multiple object groups: photo group, video group, screenshot and screen recording group, etc. Users can display photos taken through the photo group in the gallery application, videos through the video group in the gallery application, and screenshots and screen recordings through the screenshot and screen recording group in the gallery application.

[0102] When displaying images and / or videos stored on an electronic device, this grid chart can be presented in various ways, for example, Figure 2 The method shown in (b) displays photos and videos in a photo group in reverse chronological order; in practical applications, the gallery application can also set up multiple object groups, for example, through... Figure 2 The album control shown in (b) allows users to view multiple groups in the album: image group, video group, screenshot and screen recording group, etc. Users can also display pictures through the image group in the gallery application, videos through the video group in the gallery application, and screenshots and screen recordings through the screenshot and screen recording group in the gallery application. Each group can also display the corresponding pictures and / or videos in the grid format shown above.

[0103] This application does not limit the number of user operations required from the system desktop to the display of the grid chart in the gallery application, nor does it limit the way the grid chart displays images and videos.

[0104] Reference Figure 2 (c) in the text represents the user's click. Figure 2 Following the cover of video A in the grid diagram shown in (b), the electronic device responds to the operation by displaying the video playback interface of video A (this video playback interface can also be referred to as the first video playback interface). This video playback interface includes a video playback area 11 and a progress display area 12. The video frame currently displayed in the video playback area can be referred to as the first video frame.

[0105] The video playback area 11 is used to display the video frame during video playback. After entering the video playback interface from the grid interface, control 111 (which can be referred to as the first control) is displayed in the video playback area. When the user clicks control 111, the electronic device responds to the click operation on control 111 (which can be the first operation) and starts playing the video frame in the video playback area 11 from the 0th second. When the video is paused, control 111 is also displayed in the video playback area 11. When the user clicks control 111, the electronic device responds to the click operation on control 111 and starts playing the video frame in the video playback area 11 from the current paused moment.

[0106] The progress display area 12 is used to display the video playing progress. The progress display area 12 includes a progress display frame 121 and a progress indication line 122. The position of the progress indication line 122 can remain unchanged. The user can drag the progress display frame 121 below the progress indication line 122, so that the progress display frame 121 moves left and right without the progress indication line 122 moving. The position of the progress indication line 122 above the progress display frame 121 indicates the playing progress of the current video.

[0107] As an example, the total playing time of the current video is 15 seconds, and the total length of the progress display frame of the current video is L pixels. According to the two parameters, the playing time corresponding to the unit length (or the length corresponding to the unit time) can be obtained. Therefore, if the leftmost edge position of the progress display frame (the position corresponding to the first pixel) corresponds to the 0th second of the video, different positions of the progress indication line on the progress display frame correspond to different playing time points. For the convenience of subsequent description, the "time point corresponding to the progress indication line" or "time point pointed to by the progress indication line" is used for description.

[0108] Due to the space limitation of the progress display area 12, only part of the video frames or part of the area of the video frames in the progress display frame 121 can be displayed in the progress display area 12. As shown in (a) of FIG. 12, the progress display frame 121 displayed in the progress display area 12 includes a part of the area of the video frame. Figure 2 As shown in (c) of FIG. 12, the progress display frame 121 displayed in the progress display area 12 includes three complete video frame thumbnails and a part of the area of a video frame thumbnail.

[0109] Generally, the picture played by the video playing area 11 is the picture of the video frame at the time point pointed to by the progress indication line 122 in the progress display area 12.

[0110] In a specific implementation, there is a case that the time point pointed to by the progress indication line 122 is a time point between the playing time points of two adjacent video frames, that is, there is no video frame corresponding to the time point pointed to by the progress indication line 122 in the video file.

[0111] As an example, in the video frames arranged in ascending order of playing time points (denoted as time stamps), the playing time point of the ith video frame is ti, and the playing time point of the (i+1)th video frame is t(i+1). The time point pointed to by the progress indication line 122 is t, which is greater than ti and less than t(i+1). In this case, the video frame corresponding to the time stamp with the shortest time interval from the time point pointed to by the progress indication line 122 can be used as the video frame at the time point pointed to by the progress indication line 122.

[0112] Of course, in actual application, if there is no video frame with the same time as the time pointed by the timestamp and the progress line in the video file, the video frame corresponding to the time t before the time pointed by the progress line 122 (the video frame corresponding to t(i) can also be regarded as the video frame corresponding to the time pointed by the progress line 122; or the video frame corresponding to the time t after the time pointed by the progress line 122 (the video frame corresponding to t(i+1) can also be regarded as the video frame corresponding to the time pointed by the progress line 122. In actual application, there are many ways to display the playing progress in the progress display area 12, for example, the playing progress of the video can also be displayed in the form of a progress bar, or in the form of a progress ring. The embodiment of the present application only takes the progress display area shown in (c) in Figure 2 as an example to describe the difference between the ordinary playing and the frame playing.

[0113] Referring to (d) in Figure 2 , after the user clicks the control 111 in the interface shown in (c) in Figure 2 , the electronic device plays one interface diagram in the process of playing the video A in response to the operation, the interface diagram indicates that the video A has been played to the 5th second, and the video picture corresponding to the 5th second is displayed in the video playing area 11; at the same time, the progress display frame 121 in the progress display area 12 moves the position, so that the time corresponding to the progress line 122 is the 5th second. Wherein, the playing process from (c) in Figure 2 to (d) in Figure 2 is the ordinary playing process of the video.

[0114] The ordinary playing process can play all the video frames of the video file, or play part of the video frames. Of course, in order to distinguish, it is possible that the first segment of the video frames of the video is played through the ordinary playing mode the first time; then the second segment of the video frames of the same video is played through the ordinary playing mode during this time of opening the video playing interface, or the second segment of the video frames of the second video is played through the ordinary playing mode during the next time of opening the video playing interface.

[0115] A video file includes a plurality of video frames, each video frame has a timestamp, and the timestamp records the playing time of the video frame in the playing process of the video file. As an example, according to the timestamp from small to large, the video frames in the video file are arranged as: video frame 0 (the timestamp is t0), video frame 1 (the timestamp is t1), video frame 2 (the timestamp is t2), video frame 3 (the timestamp is t3), video frame 4 (the timestamp is t4), video frame 5 (the timestamp is t5), video frame 6 (the timestamp is t6), video frame 7 (the timestamp is t7)……. The ordinary playing is: according to the timestamp from small to large, the video frames are displayed in the video playing area according to the playing time corresponding to the timestamp in turn.

[0116] As an example of the normal playing process, video frame 0 is displayed in the video playing area 11 at the t0 moment when the playing is started, video frame 1 is displayed in the video playing area 11 at the t1 moment after the playing is started, video frame 2 is displayed in the video playing area 11 at the t2 moment after the playing is started, video frame 3 is displayed in the video playing area 11 at the t3 moment after the playing is started, video frame 4 is displayed in the video playing area 11 at the t4 moment after the playing is started, video frame 5 is displayed in the video playing area 11 at the t5 moment after the playing is started, video frame 6 is displayed in the video playing area 11 at the t6 moment after the playing is started, video frame 7 is displayed in the video playing area 11 at the t7 moment after the playing is started, and so on.

[0117] In combination with (c) to (d) in Figure 2 , the normal playing process of video A is shown: starting from the video frame at the 0th second of video A, each video frame is displayed in turn according to the playing moment represented by the timestamp; Figure 2 (d) in Figure 3 is an interface schematic diagram showing the video frame at the 5th second. In actual application, video A can be played completely in the normal playing mode, i.e., in the case where the playing duration of video A is 15 seconds, from the 0th second to the 15th second of video A, until the normal playing process of video A is ended.

[0118] Referring to Figure 3 , the interface schematic diagram of the frame playing of video A provided by the embodiment of the present application is shown.

[0119] Referring to Figure 2 (a), the interface diagram after video A is ended in the normal playing mode on the basis of the interface shown in Figure 3 . That is, video A has been played completely in the normal playing mode.

[0120] Referring to Figure 3 (b), the operation of the user dragging the progress of the interface shown in Figure 3 (a) to the right to show the frame.

[0121] In response to the dragging operation shown in Figure 3 (b), the electronic device will display the picture corresponding to the dragging operation in the video playing area.

[0122] As an example, referring to Figure 3In (c), when the user drags the progress bar to display a frame so that the progress indicator points to the 2nd second, the video playback area displays the video frame corresponding to the 2nd second. The video frame corresponding to the 2nd second can be the video frame in video A with the timestamp of the 2nd second; it can also be the video frame with the timestamp closest to the 2nd second; it can also be the video frame before the 2nd second whose time interval between the timestamp and the 2nd second is closest; or it can be the video frame after the 2nd second whose time interval between the timestamp and the 2nd second is closest.

[0123] Reference Figure 3 As shown in (d), when the user drags the progress display frame to the 2nd second, the user lifts their finger to release the progress display frame, and the electronic device continues to display the interface corresponding to the 2nd second of the video frame.

[0124] The operations shown in the embodiments of this application are merely examples. In practical applications, other operations can be set according to specific circumstances. For example, some single-click operations can also be double-click operations or other air gestures. Some drag operations can also be air gestures. Similarly, some examples of operations in subsequent embodiments can also be other operations.

[0125] It should be noted that, in subsequent embodiments, the specific meaning of the description method of "video frame corresponding to a certain moment" can be referred to the detailed explanation of "video frame corresponding to the 2nd second," and will not be repeated hereafter. Figure 3 (b) in Figure 4 The playback process shown in (c) is the frame playback process of the video.

[0126] As the user drags the progress display frame 121, a relative position is formed between the progress indicator line 122 and the progress display frame 121, and the position of the progress indicator line 122 on the progress display frame 121 is related to the time that the progress indicator line 122 points to.

[0127] In the specific implementation, assuming that the time indicated by the progress indicator line 122 is t1 before the user moves in the area where the progress display frame 121 is located on the screen. When the user moves left or right in the area where the progress display frame is located, the electronic device can detect the user's real-time touch position at certain time intervals. The electronic device determines the user's movement direction and distance based on the continuous real-time touch positions. The electronic device determines the time change Δt based on the user's movement distance; it determines the calculation method for real-time time t2 based on the user's movement direction. For example, if the movement direction is left, the starting time t1 is subtracted from the time change Δt to obtain the real-time time t2; if the movement direction is right, the starting time t1 is added to the time change Δt to obtain the real-time time t2. After obtaining the real-time time t2, the electronic device finds the video frame corresponding to the real-time time t2 and displays the video frame corresponding to t2 in the video playback area.

[0128] In another aspect, the electronic device determines the moving direction of the progress display frame according to the moving direction of the real-time touch position, and determines the moving distance of the progress display frame according to the moving distance of the real-time touch position; the electronic device moves the progress display frame of the progress display area according to the moving direction and the moving distance of the progress display frame, so that the progress indication line points to the time t2. As another example, the electronic device also determines the moving direction (moving left if t2 is greater than t1, and moving right if t2 is less than t1) and the moving distance of the progress display frame according to the starting time t1 and the real-time time t2; the electronic device moves the progress display frame of the progress display area according to the moving direction and the moving distance of the progress display frame, so that the progress indication line points to the time t2.

[0129] It should be noted that the above process is only an example, and in actual application, there can be multiple ways to determine the time pointed to by the progress indication line 122 and the position of the progress display frame 121 during the process of dragging the progress display frame 121 by the user.

[0130] Based on the above frame playing process, it can be understood that the process of dragging the progress display frame by the user is a process of uncertainty, which can move left, right, fast or slow, and accordingly, the video frame displayed in the video playing area has randomness.

[0131] As an example, before the user moves the progress display frame in the area of the screen, the starting time indicated by the progress indication line is t1; during the process of dragging the progress display frame by the user, the electronic device detects the real-time touch position of the user at certain time intervals, thereby obtaining multiple continuous time periods corresponding to the real-time times: t2, t3, t7, t4, t3. In the video playing area, the video frames are played in the following order: the video frame corresponding to the time t1 is taken as the starting video frame, and the video frames corresponding to the times t2, t3, t7, t4, t3 are played in sequence. It can be understood that the process of playing the video in the video playing area triggered by the electronic device when the user clicks the control 111 is ordinary playing; the process of displaying the video frame corresponding to the time pointed to by the progress indication line in the video playing area triggered by the electronic device when the user drags the progress display frame is frame playing.

[0132] In the above embodiments, whether it is ordinary playing or frame playing, the video frame needs to be decoded before being displayed in the video playing area, and in the ordinary playing scenario, the video frame is usually decoded in groups. A group of video frames forms a video frame group, and each group of video frames includes one key frame (I frame), at least one forward reference frame (P frame), and at least two bidirectional reference frames (B frames).

[0133] Reference Figure 4Fig. 1 is a schematic diagram of a set of video frames provided by an embodiment of the present application, showing the playing order and decoding order of the set of video frames. The set of video frames can also be referred to as a group of pictures (GOP) group.

[0134] Each group of video frames includes, in the playing order, an I frame, a B frame, a B frame, a P frame, a B frame, a B frame, and a P frame. The I frame can be decoded to obtain a complete picture and has a slow decoding speed. The P frame records changes relative to a previous frame (non-B frame), and the picture of the P frame can be decoded with the information of the previous frame. The picture of the B frame can be decoded with the information of the previous P frame (or I frame) and the information of the subsequent P frame. The P frame and the B frame can improve the smoothness of the picture and reduce the data volume of the video file.

[0135] Regardless of the I frame, the P frame, or the B frame, each is a video frame. When decoding a video frame at a certain time, the video frame group in which the video frame is located is decoded, and the video frames in the video frame group are decoded in the decoding order.

[0136] For example, if the video frame at t1 needs to be decoded, and the video frame at t1 is the sixth video frame in the playing order in a group of video frames, which is a B frame, the decoding order is as follows:

[0137] First decoding: the first video frame in the playing order, which is an I frame;

[0138] Second decoding: the fourth video frame in the playing order, which is a P frame;

[0139] Third decoding: the second video frame in the playing order, which is a B frame;

[0140] Fourth decoding: the third video frame in the playing order, which is a B frame;

[0141] Fifth decoding: the seventh video frame in the playing order, which is a P frame;

[0142] Sixth decoding: the fifth video frame in the playing order, which is a B frame;

[0143] Seventh decoding: the sixth video frame in the playing order, which is a B frame;

[0144] Therefore, when the video frame at t1 is decoded, the seventh decoding is needed to decode the video frame at t1.

[0145] Currently, in a normal playing scenario, the video frames in each group of video frames are parsed, decoded, rendered, and displayed in the playing order of the group of video frames. When decoding the video frames in each group of video frames, the video frames are decoded in the decoding order. Figure 5The decoding order shown decodes each video frame in the video frame group sequentially. Typically, when displaying video frames from the previous video frame group, all video frames in the next video frame group have already been decoded. Therefore, in normal playback scenarios, each video frame can be displayed according to its timestamp, resulting in smooth video playback.

[0146] During normal video playback, each video frame (I-frame, P-frame, and B-frame) in each group of video frames needs to be decoded. Therefore, during normal video playback, the P-frames and B-frames decoded during normal playback can be re-encoded into I-frames. This is equivalent to re-encoding the I-frames, P-frames, and B-frames decoded during normal playback into a video frame consisting entirely of I-frames. This video frame consisting entirely of I-frames can then be stored in a cache file. The I-frames stored during normal playback can be used in frame playback scenarios.

[0147] Of course, in practical applications, during the idle time of electronic devices, each video frame of video A can be decoded to obtain the YUV data of the decoded video frames (I-frames, P-frames, and B-frames); then the YUV data of the decoded I-frames, P-frames, and B-frames can be re-encoded into I-frames, and the re-encoded I-frames can be stored in the form of cache files. The stored I-frames can be used in frame playback scenarios.

[0148] For two video frames, to distinguish them, the encoded data of the first video frame itself is I-frame, P-frame, and B-frame; for example, it is denoted as the first encoded data; after being re-encoded into encoded data that is all I-frames, it can be denoted as the second encoded data.

[0149] For another video frame (e.g., the second video frame), its own encoded data consists of I-frames, P-frames, and B-frames; for example, it can be denoted as the fifth encoded data. After being re-encoded into encoded data consisting entirely of I-frames, it can be denoted as the sixth encoded data.

[0150] Reference Figure 4 As shown, to... Figure 6 The diagram shows the playback and decoding order of a set of video frames encoded as I-frames. The playback order remains unchanged; however, for the decoding order, since each video frame is an I-frame, it's equivalent to decoding each video frame as the first one.

[0151] In a frame playback scenario, after determining the frame playback time (the time indicated by the progress indicator line), the I-frame corresponding to the frame playback time can be obtained directly from the cache file, and then the I-frame corresponding to the frame playback time can be decoded, rendered, and displayed.

[0152] As described above, in the frame playing scenario, the electronic device needs to determine the frame playing time according to the real-time detected touch position during the user dragging the progress display frame, then needs to find the I frame corresponding to the frame playing time from the cache file, and further needs to decode, render and display the found I frame, so that the video picture display is relatively delayed in the frame playing scenario. In order to reduce the display delay of the video picture in the frame playing scenario, the cache method for quickly reading the video frame from the cache file is provided, so as to reduce the display delay of the video picture in the frame playing scenario.

[0153] In the cache method, in the writing stage of the I frame itself and related information of the video, the file path of the cache file B used to store the I frame of the video is recorded in the cache information entity of the video; after the I frame of the video is cached in the cache file B, the mapping relationship between the timestamp of the I frame and the storage position of the I frame in the cache file B is recorded in the cache information entity of the video. The writing process of the I frame itself and related information of the video can be located in the normal playing process of the video. Of course, the writing process can also be located in the idle time period.

[0154] In the reading stage of the I frame of the video, when the I frame corresponding to the frame playing time is acquired, the cache file B recording the I frame is found according to the file path of the cache file B recorded in the cache information entity; the storage position of the I frame corresponding to the frame playing time in the cache file B is found according to the mapping relationship, so that the I frame corresponding to the frame playing time can be quickly read from the cache file B.

[0155] Of course, the I frame can be further decoded, rendered and displayed.

[0156] It can be understood that the I frame corresponding to the frame playing time can be read according to the file path of the cache file storing the I frame and the mapping relationship, the reading process is simple and efficient, and the fluency of the video picture in the frame playing scenario can also be improved. The reading process of the I frame of the video is usually located in the frame playing scenario of the video.

[0157] In addition, the cache file B (i.e. the re-encoded I frame) can be stored in the disk space, so the cache file B will not exit with the gallery application. However, in order to quickly obtain the mapping relationship, the cache information entity is usually cached in the memory, and the cache information entity will be deleted with the gallery application, so that the previously cached I frame can still be read again when the gallery application is started again. The mapping relationship can also be set:

[0158] When the gallery application ends playing a video in the normal playing mode each time (for example, pause, playing completion), the cache file C is regenerated according to the mapping relationship of all I frames of the video recorded in the cache information entity, and the cache file C can also be stored in the disk, so that all the mapping relationships of the I frames of the video recorded in the newly generated cache file C are recorded in the newly generated cache file C; and the file path of the newly generated cache file C is also recorded in the cache information entity.

[0159] Before the gallery application exits, the cache file A is generated according to the file path of the cache file B and the file path of the newly generated cache file C recorded in the cache information entity, and the cache file A can be stored in the disk; and after the cache file A is generated, the cache information entity can be cleared.

[0160] When the gallery application is started again, the gallery application can obtain the file path of the cache file B and the file path of the cache file according to the cache file A, the file path of the cache file B and the file path of the cache file C recorded in the cache information entity, and the mapping relationship between the time stamp of the I frame in the cache file B and the storage position of the I frame in the cache file B in the cache information entity can also be recovered according to the cache file C. After the cache information entity of the video is recovered, the I frame can be repeatedly read again based on the related information recorded in the cache information entity.

[0161] The detailed process of the writing and reading of the video frame provided in the embodiments of the present application can be referred to the detailed description of the subsequent embodiments.

[0162] In order to more easily understand the writing and reading method of the video frame provided in the embodiments of the present application, the technical architecture diagram of the video playing engine provided in the embodiments of the present application is first described.

[0163] Referring to Figure 7 The technical architecture diagram of the playing engine corresponding to the frame playing method of the video provided in the embodiments of the present application.

[0164] The application layer of the electronic device includes a plurality of applications, for example, a gallery application, a file management application, a video playing application, an audio playing application and the like. The embodiments of the present application only list some applications related to audio and video playing, of course, in actual application, only some of the applications listed in the application layer can be used, and other applications not listed in the application layer can also be used to implement the video playing process.

[0165] The basic platform in the application framework layer has a unified playing engine. The unified playing engine is used to provide a basis for the upper layer application to implement audio and video playing.

[0166] The unified playing engine has an interface layer, which is used to transmit data between the upper layer application and other modules in the unified playing engine.

[0167] The common player is used to implement the common playing process described in the above embodiments. The common player comprises a basic interface module, a common playing module, an audio rendering module and a video rendering module.

[0168] The basic interface module is used to provide a basic interface of the video playing process, which can be used by a video playing application without an interface. The gallery application in the embodiments of the present application can not use the basic interface, but use a video playing interface provided by the gallery application itself.

[0169] The audio rendering module is used to call an audio rendering engine to render an audio stream in a video file in the common playing process.

[0170] The video rendering module is used to call a graphics rendering engine to render a video stream in a video file in the common playing process.

[0171] The common playing module is used to perform playing of an audio / video file and call other modules (for example, a codec module) to implement the common playing process and the like. In actual applications, the audio rendering module and the video rendering module can also be arranged in the common playing module.

[0172] The frame player is used to implement the frame playing process. The frame player provides two playing modes, a frame cache playing mode and a frame concurrent decoding playing mode.

[0173] In the frame cache playing mode, a frame extracting module and a cache cleaning module are involved. The frame extracting module is used to implement extraction of non-key frames, so as to re-encode and cache the extracted non-key frames. The cache cleaning module is used to clean relevant information (for example, a linked list) in the cache at a proper time.

[0174] In the frame concurrent decoding playing mode, a GOP parser and a GOP decoder are involved, which are used to implement parsing of a video file and decoding of video frames.

[0175] The decision module can be a module independent of the common player and the frame player, or a module contained in the frame player, which is used to decide whether to perform frame concurrent decoding playing or frame cache playing. The embodiments of the present application take the decision module in the frame player as an example.

[0176] The media parsing module comprises a metadata module and a thumbnail generating module. The metadata module is used to parse a video duration. The thumbnail generating module is used to generate a plurality of thumbnails as progress display frames according to the video duration in a one-to-many scenario.

[0177] The data extraction module comprises a subtitle extraction module, a video data extraction module and an audio data extraction module. The subtitle extraction module is configured to extract subtitle data of the video file; the video data extraction module is configured to extract video stream data of the video file; and the audio data extraction module is configured to extract audio stream data of the video file.

[0178] The codec module comprises a video encoding module, a video decoding module, an audio decoding module and a software and hardware adaptive module.

[0179] The video encoding module, the video decoding module and the audio decoding module can be invoked by a general player and a frame player, and the codec engine below is simultaneously invoked to implement corresponding functions.

[0180] The software and hardware adaptive module is configured to implement corresponding functions by using a software module in a case where hardware resources are limited.

[0181] The application framework layer further comprises an audio rendering engine, a graphics rendering engine and a codec engine.

[0182] The audio rendering engine is configured to be invoked to provide audio rendering; the graphics rendering engine is configured to be invoked to provide picture rendering; and the codec engine is configured to be invoked to decode and encode video frames.

[0183] The video frame writing and reading process provided by the embodiments of the present application involves general playing and frame playing of a video, and therefore, the embodiments of the present application first perform the following steps. Figure 2 The timing diagram describes the preparation process of the electronic device before general playing and frame playing. Figure 2 The timing diagram can correspond to (a) to Figure 2 in (c).

[0184] The electronic device displays a system desktop, and the system desktop comprises an icon of a gallery application.

[0185] The interface can refer to the interface schematic diagram shown in (a) of Figure 2 In addition, the gallery application is an example of a video playing application provided by the embodiments of the present application for playing a video, and in actual applications, other video playing applications can also play a video by using the general playing mode and the frame playing mode provided by the embodiments of the present application.

[0186] In S101, the gallery application receives a starting instruction.

[0187] In the embodiments of the present application, when the electronic device displays a system desktop, the system desktop comprises an icon of a gallery application. The interface schematic diagram shown in (a) of Figure 2 When a user clicks the icon of the gallery application on the system desktop, the electronic device can be triggered to start the gallery application. The gallery application can receive a starting instruction.

[0188] In addition, the gallery application provides an example of a video playing application for playing a video, and in actual applications, other video playing applications can also play a video in the general playing mode and the frame playing mode provided by the embodiments of the present application.

[0189] On one hand, the gallery application starts to display a main interface of the gallery application, and on the other hand, the gallery application starts to create a cache module for caching I frames of a video that are re-encoded when the video is played in a general mode in the future. S102, after receiving the starting instruction, the gallery application sends a creation request of the cache module to the decision module.

[0190] S103, after receiving the creation request of the cache module, the decision module creates a frame cache playing module in the frame player.

[0191] S104, after the frame cache playing module is successfully created, the frame cache playing module creates the cache module.

[0192] S105, after receiving the starting instruction, the gallery application displays a grid view, and the grid view includes a cover of a video A.

[0193] The grid view displayed by the gallery application can include covers of other videos, and of course, can also include thumbnails of pictures. Figure 2 The grid view displayed by the interface shown in (b) of FIG. 10A can be an example of the grid view in this step.

[0194] S106, the gallery application receives a click operation on the cover of the video A.

[0195] In the embodiments of the present application, when the electronic device displays the interface shown in (b) of FIG. 10A, the user clicks any cover of a video displayed by the grid view in the interface shown in (b) of FIG. 10A, and the electronic device responds to the operation to display a video playing interface of the video by the gallery application. The click operation can refer to the click operation of the user's finger in the interface shown in (b) of FIG. 10A. Figure 2 Figure 2 In the embodiments of the present application, when the electronic device displays the interface shown in (b) of FIG. 10A, the user clicks any cover of a video displayed by the grid view in the interface shown in (b) of FIG. 10A, and the electronic device responds to the operation to display a video playing interface of the video by the gallery application. The click operation can refer to the click operation of the user's finger in the interface shown in (b) of FIG. 10A. Figure 2

[0196] It should be noted that in actual applications, other modules of the electronic device can receive the click operation and transmit the click operation to the gallery application, and in order to simplify the description, the embodiments of the present application omit the intermediate steps and take the gallery application receiving the click operation on the cover of the video A as an example.

[0197] S107, after receiving the click operation on the cover of the video A, the gallery application displays a video playing interface of the video A.

[0198] ​​The video playing interface of the video A can refer to the interface shown in (c) of FIG. 13, and the video playing interface of the video A includes a video playing area and a progress display area for displaying the playing progress. Figure 2

[0199] It should be noted that the electronic device is responsive to the click operation, and on one hand, the process of displaying the video playing interface of the video A is performed; on the other hand, the process corresponding to the following steps is also performed. Therefore, there is no strict sequence between the step S107 and the step S108 (and the subsequent steps).

[0200] S108, after the gallery application receives the click operation on the cover of the video A, the gallery application sends the file path of the video A to the decision module.

[0201] In the embodiment of the present application, the gallery application of the electronic device stores the file paths of each picture and each video file in the electronic device, and the gallery application can display the pictures and play the videos based on the file paths of each picture and each video file.

[0202] After the decision module receives the file path of the video A, on one hand, the decision module sends the file path of the video A to the normal playing module, so that the normal playing module performs the preparation work for the normal playing of the video A; on the other hand, the decision module sends the file path of the video A to the frame buffer playing module, so that the frame buffer playing module performs the preparation work for the frame buffer playing of the video A.

[0203] First, the sending of the file path of the video A to the normal playing module and the subsequent preparation work of the normal playing module are described.

[0204] S109, after the decision module receives the file path of the video A, the decision module sends the file path of the video A to the normal playing module.

[0205] S110, after the normal playing module receives the file path of the video A, the normal playing module acquires the video A according to the file path and parses to obtain the encoding format of the video A. In the present application, the encoding format of the video A is taken as H.264 as an example.

[0206] S111, the normal playing module creates an H.264 decoder for normal playing according to the encoding format of the video A. The decoder is used for normal playing, and therefore, can be recorded as a normal decoder.

[0207] Next, the sending of the file path of the video A to the frame buffer playing module and the subsequent preparation work of the frame buffer playing module are described.

[0208] S112, after the decision module receives the file path of the video A, the decision module sends the file path of the video A to the created frame buffer playing module.

[0209] ​S113, after the frame buffer playing module receives the file path of the video A, the frame buffer playing module acquires the video A according to the file path and parses the coding format of the video A. In the embodiment of the application, the coding format is H.264.

[0210] S114, the frame buffer playing module creates the H.264 coding module according to the parsed coding format. The coding module is used to re-encode the P frame and the B frame into the I frame.

[0211] S115, the frame buffer playing creates the H.264 decoding module according to the parsed coding format. The decoding module is used to decode the I frame in the cache space to obtain the YUV data in the frame buffer playing process.

[0212] In actual application, the steps S102 to S104 can also be executed between the step S106 and the step S108. The application will not be exemplified by the diagram.

[0213] After the step, the preparation work for the normal playing and the preparation work for the frame playing are both completed. After the electronic device receives the operation for the normal playing, the electronic device will execute the normal playing of the video, and after the electronic device receives the operation for the frame playing, the electronic device will execute the frame playing of the video.

[0214] The embodiment of the application and Figure 8 The playing process of the scene shown in the figure corresponds to the case that the operation for the normal playing is received first. Refer to Figure 2 , a timing diagram of a normal playing method of a video provided by the embodiment of the application, which can correspond to the process that the operation (c) shown in the (b) is received first. Figure 2

[0215] S201, the gallery application receives the clicking operation on the normal playing control.

[0216] In the embodiment of the application, when the electronic device displays the video playing interface of the video A shown in the (c) in the (b), the user clicks the normal playing control in the video playing area shown in the (c) in the (b), which can trigger the electronic device to execute the normal playing process of the video A. Figure 2 Figure 10

[0217] S202, after the gallery application receives the clicking operation on the normal playing control, the gallery application sends the normal playing request of the video A to the decision module, and the request carries the start time of the normal playing as the 0th second.

[0218] S203, after the decision module receives the normal playing request of the video A, the decision module sends the playing instruction of the video A to the normal playing module, and the playing instruction carries the time of the start playing as the 0th second.

[0219] ​​​S204, after receiving the playing instruction of the video A, the common playing module parses the video A to obtain a video frame in H.264 format.

[0220] S205, the common playing module sends a decoding request of the video frame to the common decoder.

[0221] In actual application, the common playing module can cache the video frame in H.264 format, and the decoding request carries a cache address, and the common decoder obtains the video frame in H.264 format from the cache address to perform decoding; of course, the common playing module can also carry the video frame in H.264 format in the decoding request, and the common decoder performs decoding on the received video frame in H.264 format. The specific implementation manner is not limited in the application.

[0222] S206, the common decoder decodes the video frame to obtain decoded video data: YUV data.

[0223] The decoded video data is divided into I frame, P frame and B frame.

[0224] S207, the common decoder sends the decoded YUV data to the common playing module.

[0225] After receiving the YUV data, the common playing module displays the YUV data on one hand, and transmits the YUV data to the cache module to be re-encoded and cached on the other hand.

[0226] Firstly, the process of rendering and displaying the YUV data is described. S208, after receiving the decoded YUV data, the common playing module renders the YUV data to obtain a video picture.

[0227] S209, the common playing module sends the video picture obtained by rendering to the gallery application, and carries a timestamp of the video picture.

[0228] S210, after receiving the video picture, the gallery application displays the received video picture in a video playing area, and updates the position of a progress display frame in a progress display area, so that the time corresponding to the progress indication line is consistent with the timestamp of the received video picture.

[0229] Next, the process of re-encoding and caching the YUV data is described. After S207:

[0230] S211, the common playing module sends the YUV data to the decision module. At this time, the YUV data is I frame, P frame or B frame.

[0231] S212, after receiving the YUV data, the decision module sends the YUV data to the frame cache playing module.

[0232] S213, after the frame buffer playing module receives the YUV data, the frame buffer playing module sends the YUV data to the buffer module.

[0233] S214, after the buffer module receives the YUV data, the buffer module checks whether the buffer space has the video frame corresponding to the received YUV data.

[0234] In actual application, the YUV data itself carries a time stamp. In S118, it has been determined that the currently displayed video playing interface is that of video A. Therefore, the buffer module can check whether the buffer space has the video frame corresponding to the time stamp carried by the YUV data of video A. Of course, the decision module can also carry the identification information of video A when sending the YUV data to the frame buffer playing module.

[0235] S215, in the case that the buffer module finds that the buffer space does not have the video frame corresponding to the received YUV data, the buffer module sends the YUV data to the encoding module.

[0236] S216, the encoding module encodes the received YUV data into an I frame of H.264 format.

[0237] In specific implementation, the YUV data received by the encoding module is the decoded data of I frame, P frame and B frame; therefore, the encoding module also needs to re-encode the YUV data of I frame into I frame data of H.264 format, re-encode the YUV data of P frame into I frame data of H.264 format, and re-encode the YUV data of B frame into I frame data of H.264 format.

[0238] S217, the encoding module sends the encoded I frame to the buffer module.

[0239] S218, after the buffer module receives the I frame sent by the encoding module, the buffer module buffers the received I frame.

[0240] If the first segment of video frames (any segment of video frames) of a video is buffered through the above-mentioned normal playing process, the second segment of video frames of the same video can also be buffered through the above-mentioned process in the case that the gallery application is restarted.

[0241] If the encoding data (I frame, P frame and B frame) of the first segment of video frames in the video file is first encoding data, the re-encoded data of the first segment of video frames is second encoding data.

[0242] When the subsequent frame is played, if the frame playing time is the first time, the third encoded data corresponding to the first time in the second encoded data is played. If the frame playing time is the second time, the fourth encoded data corresponding to the second time in the second encoded data is played. The subsequent frame playing process can be a frame playing operation during the running of the current gallery application, or a frame playing process after the gallery application is exited and then started again.

[0243] In step S218, when the buffer module receives the first encoded I frame after being created, it first needs to create a corresponding relationship between the unique identifier of the video in which the received I frame is located and the data structure of the buffer information entity of the video. Then the received first I frame is cached in the buffer file B, and the corresponding information is filled in the buffer information entity.

[0244] The unique identifier of the video is used as an index, and some buffer information of the video in the data structure of the buffer information entity of the video is recorded (for details, refer to the detailed description in the subsequent Figure 7

[0245] In the specific implementation, in the process of transmitting the data of the video frame itself, in addition to carrying the timestamp of the video frame, the unique identifier (for example, the file path) of the video file in which the video frame is located also needs to be carried.

[0246] Of course, in actual application, after the execution of S113 shown in Figure 9 is completed, the frame buffer playing module has recorded the file path of video A, so in the process of executing S213 of the frame buffer playing module, the file path of video A in which the YUV data is located can also be carried, and the specific implementation process is not limited by the embodiments of the application.

[0247] The data structure of the buffer structure diagram in which the unique identifier of the video and the data structure of the buffer information entity of the video are located will be described below. Figure 9

[0248] In the specific implementation process, the user can play multiple videos in the normal playing mode through the gallery application, so multiple video frames re-encoded by the video files can be cached in the electronic device, and correspondingly, multiple buffer information entities of the video files can also be cached.

[0249] As shown in Figure 10 ​​As shown, the unique identifier of the video file and the cache information entity of the video file can be stored in association, for example, the unique identifier of the video file can be the file path of the video file, wherein the file path of the video file is the storage path of the video file in the electronic device, which usually points to a certain physical location in the disk of the electronic device; the cache information entity records the mapping relationship between the time stamp of the re-encoded video frame of the video file and the cache location of the re-encoded video frame in the cache file B and other related information, and the specific content recorded in the cache information entity can refer to the detailed description in Figure 10 .

[0250] When the re-encoded video frames of multiple video files are cached in the electronic device, the cache information entities of the multiple video files can be stored in the form of a linked list.

[0251] For example, the multiple data nodes from the head of the table to the tail of the table include the file path of video A and the cache information entity of video A, the file path of video B and the cache information entity of video B, and the file path of video C and the cache information entity of video C.

[0252] Generally, a certain cache space in the memory can be allocated, which can store the file paths and cache information entities of multiple video files, and each time a new cache information entity of a video file is added, the new cache information entity of the video file can be cached in the data node at the head position of the linked list; when the cache information entity of a video file is accessed, the accessed cache information entity of the video file is also updated to the head position of the linked list; in this way, the cache information entities of the video files cached in the linked list are sorted in the order of access time, and the cache information of the video file at the head position is the most recently accessed, and the cache information of the video file at the tail position is the most distant from the current time.

[0253] Since the cache space allocated for caching the cache information entities of the video files has a fixed size, when the cache space caches a large number of cache information entities of the video files, and a new cache information entity of a video file is added, it will cause the cache space to be insufficient, and the data node at the tail position can be deleted, which is equivalent to deleting the cache information entity of the video file that is not frequently accessed. Of course, in actual application, the cache file A (which can be referred to as the third cache file), the cache file B (which can be referred to as the first cache file), and the cache file C (which can be referred to as the second cache file) stored in the disk can be deleted to release the disk space.

[0254] In addition, this linked list can be cached in the cache module of the frame player, generated after the cache module is created and cleared before the cache module exits.

[0255] As mentioned earlier, each data node stores the file path of the video file and the cache information of the video file. The following example uses video A. Figure 11 Describe the contents of the cached information entity.

[0256] The cache information entity includes:

[0257] map: The mapping relationship between the timestamps of video frames (i.e., the re-encoded keyframes) in video A and the storage location of the video frames in cache file B;

[0258] frameNum: The number of video frames that have been cached in video A;

[0259] length: The size of cache file B containing the cached video frames of video A;

[0260] frameCachePath: The file path B of the cache file B where the cached video frames of video A are located (can be remembered as the first cache path);

[0261] ptsPosition: The file path C (which can be denoted as the second file path) is generated by mapping the timestamps of cached video frames of video A to the storage location of the video frames in cache file B.

[0262] After describing the cache structure and cache information entity, through Figure 8 Description of embodiments provided in this application Figure 10 The detailed process of step S218 is as follows: Step S218 includes:

[0263] S2181, the buffer module receives the I-frame of video A sent by the encoding module.

[0264] S2182, the cache module checks if the linked list exists.

[0265] S2183, if a linked list exists, the cache module determines whether there is a cache information entity for video A where the I-frame is located in the linked list.

[0266] S2184, if there is no cache information entity for video A where I-frame is located, the cache module creates a data structure for the cache information entity corresponding to the file path of video A in the linked list.

[0267] Of course, it is also necessary to establish the correspondence between the file path of video A and the data structure of the cache information entity of video A. At this time, the data in the data structure of the cache information entity is empty.

[0268] S2185, after creating the data structure of the cache information entity, the cache module creates the cache file B, and writes the file path B of the cache file B used for caching the I frame in the data structure frameCachePath of the cache information entity.

[0269] The cache file B can be stored in the memory or in the disk.

[0270] S2186, the cache module writes the I frame in the cache file B.

[0271] S2187, the cache module writes the mapping relationship between the timestamp of the I frame and the position in the cache file B in the data structure map of the cache information entity, updates the number of cached frames in the data structure frameNum of the cache information entity, and updates the size of the cache file B in length.

[0272] The execution order of the above process is not determined by the size of the step number, but by its inherent logic.

[0273] After S2182, it also includes:

[0274] S2188, if there is no linked list, the cache module creates a linked list, and performs S2184 and the subsequent steps.

[0275] After S2183, it also includes:

[0276] S2189, if there is a cache information entity of the video A in the linked list, the cache module finds the file path of the cache file B from the cache information entity. After finding the file path of the cache file B, S2185 and S2186 are executed.

[0277] As can be understood from the description of step S218, after S218 is executed, the cache information entity of the video A has been filled with map, Long frameNum, Long length and FrameCachePath, but PtsPosition has not been filled. Of course, as the I frame is received subsequently, map, Long frameNum, Long length will continue to be filled.

[0278] After the encoding module re-encodes each video frame, the video frame obtained by re-encoding is sent to the cache module, the cache module stores the video frame, and updates the information related to the video frame in the cache information entity structure; when the next video frame is received, the cache module stores the video frame, and updates the information related to the newly received video frame in the cache information entity structure. Figure 12 For example, the following steps can be specifically referred to:

[0279] When the cache module receives the key frame I1 at time t1, the cache module caches the received key frame I1 at time t1 in the position 1000 of the cache file B, fills the map: the timestamp t1 of the key frame I1 and the storage position 1000 of the key frame I1 in the cache file B in the data structure of the cache information entity of the video A, updates the number of currently cached frames to 1, and updates the size of the cache file B corresponding to the cached video frame to 0.13 Mb.

[0280] After the cache module receives the key frame I2 at time t2, the cache module caches the received key frame I2 at time t2 in the position 2000 of the cache file B, fills the map: the timestamp t2 of the key frame I2 and the storage position 2000 of the key frame I2 in the cache file B in the data structure of the cache information entity of the video A, updates the number of currently cached frames to 2, and updates the size of the cache file B corresponding to the cached video frame to 0.26 Mb.

[0281] After the cache module receives the key frame I3 at time t3, the cache module caches the received key frame I3 at time t3 in the position 3000 of the cache file B, fills the map: the timestamp t3 of the key frame I3 and the storage position 3000 of the key frame I3 in the cache file B in the data structure of the cache information entity of the video A, updates the number of currently cached frames to 3, and updates the size of the cache file B corresponding to the cached video frame to 0.40 Mb.

[0282] …

[0283] When the normal play is paused or ended, the cache file C is generated according to the mapping relationship between the timestamps and the storage positions in the map table, the cache file C is stored in the disk, and the file path C of the cache file C is written in the cache information entity structure.

[0284] In actual application, the electronic device needs to continuously perform S206 to S218, so that each video frame is displayed according to the playing time indicated by the timestamp of each video frame in the video file in the video playing interface; and the decoded video frame is re-encoded as an I frame and then cached.

[0285] As described above, when the normal play is paused or ended, the cache file C is generated according to the mapping relationship between the timestamps and the storage positions in the map table, the cache file C is stored in the disk, and the file path C of the cache file C is written in the cache information entity structure. The following describes a timing diagram when the normal play is ended. Figure 8

[0286] ​As described above, after decoding the video frame to obtain the YUV data, the S206 common decoder needs to send the YUV data to the common playing module. The S207 common decoder sends the decoded YUV data to the common playing module. The common playing module renders the YUV data for display and sends the YUV data to the decision module for sending the decoded YUV data to the cache module through the decision module.

[0287] The embodiment of the present application is implemented based on the timing diagram shown in S207 to S218 in Figure 8 .

[0288] After S207, it further includes:

[0289] The S219 common playing module determines that the received YUV data is the last video frame of the video A.

[0290] As shown in Figure 13 , the common playing module can parse the video A in step S204, and thus can record the timestamp of the last video frame of the video A. After receiving the YUV data sent by the common decoder, the common playing module can determine whether the timestamp of the received YUV data is the timestamp of the last video frame of the video A.

[0291] In S211 to S218, the YUV data can carry the video A playing end flag when being transmitted.

[0292] After S218, it further includes:

[0293] The S220 cache module generates the cache file C according to the map in the cache information entity of the video A after caching the I frame re-encoded from the last video frame of the video A, stores the cache file C, and records the file path of the cache file C in the cache information entity.

[0294] In the embodiment of the present application, when the video A is played from the 0th second to the end (for example, played to the 15th second of the video A) in the common playing mode, the cache module has cached all the video frames of the video A from the 0th second to the 15th second as I frames. Some of the cached I frames are I frames in the previous encoding, some are P frames in the previous encoding, and some are B frames in the previous encoding.

[0295] As described above, the electronic device also performs the preparation work of the frame cache playing, and thus the video A can also be played by frame through the gallery application.

[0296] The embodiment of the present application describes the frame cache playing process of the video A through Figure 13 , and the timing diagram corresponding to Figure 3 . Figure 3the process of (c) in Figure 3 the process of (c) in

[0297] The user can drag the progress display frame in the interface shown in (b) to trigger the electronic device to frame play video A, that is, to display a video frame of video A at a certain time in the video playing area. Embodiments of the present application take the user dragging the progress display frame to the right as an example. Figure 3

[0298] S301, the gallery application receives the dragging operation of the progress display frame.

[0299] The dragging operation can be a dragging operation to the right on the basis of the interface shown in (b). Before the dragging operation, the user has played the video frames of video A from the 0th second to the 15th second through normal playing, and accordingly, the cache space has cached the key frames between the 0th second and the 15th second of video A. Figure 13

[0300] S302, the gallery application determines the timestamp t1 of the frame play of video A according to the dragging position of the dragging operation.

[0301] In embodiments of the present application, a continuous dragging operation includes: the user's finger contacting the progress display frame, the user's finger moving, and the user's finger releasing from the progress display frame. The timestamp t1 determined in embodiments of the present application is the frame play moment determined by any contact position in the dragging operation.

[0302] S303, the gallery application sends a frame play request to the decision module, and the frame play request carries the timestamp t1 of video A.

[0303] S304, after the decision module receives the frame play request, the decision module first sends a query request of the video frame to the frame cache play module, and the query request carries the timestamp t1 of video A.

[0304] S305, after the frame cache play module receives the query request, the frame cache play module sends a query request of the video frame of video A at the t1 moment to the cache module.

[0305] S306, the query result of the cache module querying the video frame of video A at the t1 moment is that there is the video frame of video A at the t1 moment.

[0306] When the cache module queries the video frame of video A at the t1 moment, the following methods can be used:

[0307] S3061, find the cache information entity corresponding to the file path of video A from the linked list;

[0308] S3062, find whether there is the timestamp t1 from the map of the cache information entity of video A. ​​

[0309] If there is, the query result is the video frame of time t1 of video A. If there is not, the query result is that there is no video frame of time t1 of video A.

[0310] It can be understood that it is not necessary to actually find the video frame of time t1 in the cache file B, but only to query whether there is a timestamp t1 in the map of the cache information entity. The query process is simple and efficient.

[0311] S307, the cache module sends the query result to the frame cache playing module.

[0312] S308, the frame cache playing module sends the query result to the decision module.

[0313] S309, after the decision module receives the query result of the video frame of time t1 of video A, the decision module sends a frame cache playing instruction to the frame cache playing module, and the instruction carries the timestamp t1 of video A.

[0314] S310, after the frame cache playing module receives the frame cache playing request, the frame cache playing module sends a video frame of time t1 of video A to the cache module.

[0315] S311, after the cache module receives the acquisition request, the cache module acquires the video frame of time t1 of video A from the cache space.

[0316] The specific implementation process of step S311 can include:

[0317] S3111, the cache module finds the cache information entity of video A from the linked list according to the file path of video A.

[0318] S3112, the cache module finds the file path B of the cache file B from the cache information entity, and acquires the cache file B according to the file path B.

[0319] S3113, the cache module finds the storage address of the video frame corresponding to the timestamp t1 in the cache file B from the map of the cache information entity.

[0320] S3114, the cache module acquires the video frame corresponding to t4 from the cache file B according to the storage address of the video frame corresponding to t1 in the cache file B.

[0321] It can be understood that the process of acquiring the video frame of a certain time from the cache space is also simple and efficient.

[0322] S312, after the cache module receives the acquisition request, the cache module sends the video frame of time t1 to the decoding module.

[0323] In this application, the caching module can first obtain the video frame at time t1 from the cache space. Since the caching module caches keyframes corresponding to each time, the obtained video frame at time t1 is a keyframe at time t1, and there are no other video frames.

[0324] S313, after receiving the video frame at time t1, the decoding module decodes the H.264 format video frame at time t1 to obtain the YUV data at time t1. This step only requires decoding the video frame at time t1.

[0325] S314, the decoding module sends the YUV data at time t1 to the buffer module.

[0326] S315, after receiving the YUV data at time t1, the buffer module sends the YUV data at time t1 to the frame buffer playback module.

[0327] S316, after the frame buffer playback module receives the YUV data at time t1, it renders the YUV data at time t1 to obtain the video frame at time t1.

[0328] S317, the frame buffer playback module sends the video frame at time t1 to the gallery application.

[0329] S318, after the gallery application receives the video frame at time t1, it displays the video frame of video A at time t1 in the video playback area, and at the same time updates the position of the progress display frame so that the time corresponding to the progress indicator line is consistent with the timestamp of the received video frame.

[0330] pass Figure 14 As can be understood from the timing diagram, the caching method provided in this application embodiment is simple and efficient for obtaining the video frame at time t1. During decoding, only one video frame at time t1 needs to be decoded, which reduces the time required to obtain the video frame at time t1 and the time required to decode the video frame at time t1, thereby improving the smoothness of frame playback.

[0331] The above describes the normal playback process and frame playback process of video A. As mentioned earlier, when opening the video playback interface of video A, it is necessary to create a decoder (or decoding module), an encoder (or encoding module), etc. Similarly, when exiting the video playback interface of video A, it is also necessary to destroy the created parser, decoder, and cache information in the linked list.

[0332] Reference Figure 2 This is a timing diagram showing the destruction of the decoder and encoder when exiting the video playback interface of video A, as provided in this embodiment of the application.

[0333] S401, while displaying the video playback interface of Video A, the gallery application receives a return operation.

[0334] In the embodiments of the present application, the return operation can be an operation of swiping from the rightmost side of the screen to the left on the interface, which can trigger the gallery application to return to the grid interface shown in (b) of Figure 9

[0335] S402, after receiving the return operation, the gallery application displays the grid interface on one hand.

[0336] S403, after receiving the return operation, the gallery application sends a destruction request of the normal decoder of video A, the encoding module and the decoding module to the decision module, and the destruction request is used to destroy the modules created when entering the video playing interface of video A.

[0337] It should be noted that there is no strict sequence between S402 and S403, which can be two parallel steps, or one step can be executed before the other step.

[0338] After receiving the destruction request, the decision module destroys the related modules (encoding module and decoding module) for frame buffer playing in the frame player on one hand, and destroys the related modules (for example, normal decoder) in the normal player on the other hand. The destruction operations of the two aspects can be executed simultaneously, or the destruction operation of one aspect can be executed before the destruction operation of the other aspect.

[0339] S404, after receiving the destruction request of the normal decoder of video A, the encoding module and the decoding module, the decision module sends a destruction request of the encoding module and the decoding module to the frame buffer playing module.

[0340] S405, after receiving the destruction request of the encoding module and the decoding module, the frame buffer playing module destroys the encoding module.

[0341] S406, after receiving the destruction request of the encoding module and the decoding module, the frame buffer playing module destroys the decoding module.

[0342] The execution sequence of S405 and S406 has no strict sequence.

[0343] S407, after destroying the encoding module and the decoding module, the frame buffer playing module sends information of successful destruction of the encoding module and the decoding module to the decision module.

[0344] S408, after receiving the destruction request of the normal decoder of video A, the encoding module and the decoding module, the decision module sends a destruction request of the normal decoder to the normal playing module.

[0345] S409, after receiving the destruction request of the normal decoder, the normal playing module destroys the normal decoder. ​

[0346] S410, after the common playing module destroys the common decoder, the common playing module sends information of successful destruction of the common decoder to the decision module.

[0347] S411, after the decision module determines that the related modules in the frame player and the common player are successfully destroyed, the decision module sends information of successful destruction to the gallery application.

[0348] In the embodiment of the present application, if the user clicks the cover of the video B in the grid diagram at this time, the preparation work of the common playing and the preparation work of the frame playing are executed from S106. Only the decoder, the encoding module and the decoding module matching the encoding format of the video B are needed to be created according to the encoding format of the video B. The present application will not be described again.

[0349] Of course, the user can also exit the gallery application. For example, the user can return to the system desktop through a shortcut or return to the system desktop through a back operation.

[0350] S412, the gallery application receives an exit instruction.

[0351] The exit instruction can be an exit instruction when the gallery application is returned to the background, or an exit instruction before the gallery application is killed in the background.

[0352] S413, the gallery application sends a destruction request of the cache information to the decision module.

[0353] S414, after the decision module receives the destruction request of the cache information, the decision module sends a destruction request of the cache information to the frame cache playing module.

[0354] S415, after the frame cache playing module receives the destruction request of the cache information, the frame cache playing module sends a deletion instruction of the linked list to the cache module.

[0355] S416, after the cache module receives the deletion instruction of the linked list, the cache module generates a cache file A of the cache structure body according to the linked list and stores the cache file A.

[0356] Referring to FIG. 4B, Figure 15 The cache file A is generated by the information recorded in all the cache information entities in the linked list.

[0357] As an example, when the linked list includes: the cache information entity of the video A and the unique identifier (the file path of the video A), the cache information entity of the video B and the unique identifier (the file path of the video B), and the cache information entity of the video C and the unique identifier (the file path of the video C).

[0358] The generated cache file A includes:

[0359] The file path of the cache file B where the key frame in the video A is located, the file path of the cache file C generated according to the mapping relationship corresponding to the key frame in the video A, and the unique identifier of the above information (the file path of the video A).

[0360] The file path of the cache file B where the key frame in the video B is located, the file path of the cache file C generated according to the mapping relationship corresponding to the key frame in the video B, and the unique identifier of the above information (the file path of the video B).

[0361] The file path of the cache file B where the key frame in the video C is located, the file path of the cache file C generated according to the mapping relationship corresponding to the key frame in the video C, and the unique identifier of the above information (the file path of the video C).

[0362] S417, after storing the cache file A, the cache module sends the file path of the cache file A to the frame cache playing module.

[0363] S418, after receiving the file path of the cache file A, the cache module sends the file path of the cache file A to the decision module.

[0364] S419, after receiving the file path of the cache file A, the decision module sends the file path of the cache file A to the gallery application.

[0365] S420, after storing the cache file A, the cache module deletes the linked list.

[0366] S421, after deleting the linked list, the cache module sends the information that the deletion of the linked list is successful to the frame cache playing module.

[0367] S422, after receiving the information that the deletion of the linked list is successful, the frame cache playing module destroys the cache module.

[0368] S423, after destroying the cache module, the frame cache playing module sends the information that the destruction of the cache module is successful to the decision module.

[0369] S424, after receiving the information that the destruction of the cache module is successful, the decision module destroys the frame cache playing module.

[0370] S425, after destroying the frame cache playing module, the decision module sends the information that the destruction of the frame cache playing module is successful to the gallery application.

[0371] It should be noted that the above process of destroying each module and each information is only an example, and in actual application, the destruction order of each module can be changed, and the present application will not be repeated.

[0372] In addition, as described above, the frame buffer playing module and the cache module can be created after the gallery application is started, or the frame buffer playing module and the cache module can be created when the video playing interface of the video is displayed.

[0373] Similarly, the frame buffer playing module, the cache module and the linked list can be destroyed when the video playing interface of the video is exited, or the frame buffer playing module, the cache module and the linked list can be destroyed when the gallery application is exited.

[0374] The embodiment of the present application does not limit the specific implementation process.

[0375] As described above, the embodiment of the present application can realize repeated reading of the cached I frame, and the timing diagram when the video playing interface of the video A is started again is described below.

[0376] Referring to Figure 15 The timing diagram of the steps performed by the electronic device when the gallery application is started again is provided for the embodiment of the present application.

[0377] Figure 7 In the embodiment of the present application, the data transmission process of S102 to S104 is the same as that in the embodiment of the present application, but the file path of the cache file A is carried in the data transmission process of S102 to S104. Figure 16 After the frame buffer playing module creates the cache module in S104, the cache module also needs to perform S116 to generate the linked list according to the file path of the cache file A.

[0378] The execution process of S116 is shown in

[0379] ​

[0380] S1161, the cache module obtains the cache file A according to the file path A.

[0381] S1162, the cache module obtains the file path of the video A, the file path B of the cache file B and the file path C of the cache file C according to the cache file A.

[0382] S1163, the cache module creates the linked list, and writes the file path of the video A and the file path B of the cache file B and the file path C of the cache file C in the cache information entity in the created linked list.

[0383] S1164, the cache module obtains the cache file C according to the file path C, and writes the mapping relationship between the timestamp and the storage position of the corresponding video frame in the cache file B in the map according to the cache file C.

[0384] S1165, the cache module obtains the cache file B according to the file path B, and obtains the frame number and size of the cached I frame according to the cache file B.

[0385] ​​Of course, if there is another video B in the chain table, the cache information entity of video B is also recovered in the above manner.

[0386] After step S116 is performed, the cache information entity of each video in the chain table is restored to the state before the gallery application was last exited. Therefore, when playing the frames of a video, if the cache information of the video is stored in the chain table, the video frame corresponding to the timestamp can be quickly searched from the cache information entity of the chain table as described above, thereby achieving repeatable re-reading.

[0387] Of course, in actual applications, if during the normal playing of video A, video A is not played completely in the normal playing manner, but is played partially, for example, the video frames of video A from the 0th second to the 5th second are played in the normal playing manner, the I frames of video A from the 0th second to the 5th second are cached during the playing of the video frames from the 0th second to the 5th second in the normal playing manner. After the gallery application is exited and then started again, the video frames of video A at any time period can continue to be played in the normal playing manner, for example, the video frames from the 2nd second to the 15th second can be played, and when the video frames from the 2nd second to the 5th second are played, the I frames from the 2nd second to the 5th second do not need to be cached repeatedly, and when the video frames from the 5th second to the 15th second are played, the I frames from the 5th second to the 15th second can continue to be cached. Of course, the I frames from the 5th second to the 15th second can also be continuously cached in the writing and reading manner provided in the embodiments of the present application. For example, the I frames from the 5th second to the 15th second of video A are continuously stored in the cache file B, and the related information in the cache information entity of video A is continuously filled.

[0388] As another embodiment of the present application, the writing and reading method of the video frames comprises:

[0389] The electronic device displays a first video playing interface through a video playing application, wherein the first video playing interface comprises a first video frame of a first video and a first control;

[0390] After the electronic device receives a first operation on the first control, the electronic device displays a first segment of video frames obtained based on first encoding data in the first video playing interface;

[0391] During the displaying of the first segment of video frames obtained based on the first encoding data in the first video playing interface, the electronic device writes second encoding data of the first segment of video frames in a first cache file on a disk, and records a mapping relationship between a timestamp of the second encoding data and a storage position of the second encoding data in the first cache file in a memory, wherein a decoding efficiency of the second encoding data is higher than a decoding efficiency of the first encoding data;

[0392] In a running process of the video playing application, the electronic device generates a second cache file according to the mapping relationship, and stores the second cache file in a disk,

[0393] In a running process of the video playing application, the electronic device generates a second cache file according to the mapping relationship, and stores the second cache file in a disk,

[0394] In a running process of the video playing application, the electronic device generates a second cache file according to the mapping relationship, and stores the second cache file in a disk,

[0395] In a running process of the video playing application, the electronic device generates a second cache file according to the mapping relationship, and stores the second cache file in a disk,

[0396] In the present application, the video playing application can be the gallery application in the above embodiments, or a video playing plug-in integrated in the gallery application. Of course, it can also be other video playing applications. The first control can be a progress display frame. The first video frame can be any one of the video frames in the first video frame. In actual application, before the video playing application exits, if the re-encoded data corresponding to the frame playing operation at the time has been cached, the frame caching process can also be performed. Of course, during the process of playing another video frame (for example, the second video frame) of the same video in the normal playing mode before the video playing application exits, the corresponding re-encoded video frame can also be cached. For example, the sixth encoded data of the second video frame.

[0397] It should be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution. The execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0398] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program. When the computer program runs on an electronic device, the steps in each of the above method embodiments can be implemented.

[0399] The embodiments of the present application also provide a computer program product. When the computer program product runs on an electronic device or a wireless router, the electronic device can implement the steps in each of the above method embodiments.

[0400] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the computer program for instructing the related hardware to complete all or part of the processes in the above-mentioned embodiments can be stored in a computer readable storage medium, and the computer program can implement the steps of the above-mentioned method embodiments when executed by a processor. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms. The computer readable medium at least includes any entity or device capable of carrying the computer program code to the first device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunications signal and a software distribution medium. For example, a U disk, a mobile hard disk, a magnetic disk or an optical disk, etc. In some jurisdictions, according to legislation and patent practice, the computer readable medium can not be an electrical carrier signal and a telecommunications signal.

[0401] The embodiments of the present application also provide a chip, which includes a processor and a memory. The processor is coupled with the memory. The processor invokes a computer program stored in the memory to implement the steps of any method embodiment of the present application. The chip can be a single chip or a chip module composed of multiple chips.

[0402] In the above embodiments, the description of each embodiment has its own focus. The parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0403] Those skilled in the art can appreciate that the units and method steps of the examples described in combination with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0404] The above embodiments are only used to illustrate the technical solutions of the present application, but not limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalent ones. Such modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method for writing and reading video frames, characterized in that, include: An electronic device displays a first video playback interface via a video playback application. The first video playback interface includes a first video frame of the first video and a first control. After receiving the first operation on the first control, the electronic device displays the first video frame obtained based on the first encoded data on the first video playback interface. During the process of displaying the first video frame obtained based on the first encoded data on the first video playback interface, the second encoded data of the first video frame is written into the first cache file on the disk, and the mapping relationship between the timestamp of the second encoded data and the storage location of the second encoded data in the first cache file is recorded in memory. The electronic device decodes the second encoded data more efficiently than it decodes the first encoded data. During the operation of the video playback application, the electronic device generates a second cache file according to the mapping relationship, and stores the second cache file on the disk. During the process of the video playback application exiting operation, the electronic device deletes the mapping relationship in the memory; After the video playback application is restarted, the electronic device obtains the mapping relationship based on the second cache file; Upon receiving an operation to display a video frame at the first moment of the first video, the electronic device obtains the first storage location of the third encoded data of the first video in the first cache file from the mapping relationship, reads the third encoded data from the first cache file according to the first storage location, and displays the video frame after decoding the third encoded data on the first video playback interface. The third encoded data is the encoded data corresponding to the first moment in the second encoded data.

2. The method as described in claim 1, characterized in that, The second encoded data includes encoded data at a second moment. Before the video playback application exits, the method further includes: Upon receiving an operation to display a video frame at a second moment of the first video, the second storage location of the fourth encoded data of the first video in the first cache file is obtained from the mapping relationship; The fourth encoded data is read from the first cache file according to the second storage location; The video frame after decoding the fourth encoded data is displayed on the first video playback interface. The fourth encoded data is the encoded data corresponding to the second moment in the second encoded data.

3. The method as described in claim 1 or 2, characterized in that, After the video playback application restarts, the method further includes: The first video playback interface displays the second video frame obtained based on the fifth encoded data. The sixth encoded data of the second video frame is written into the first cache file. The mapping relationship between the timestamp of the sixth encoded data and the storage location of the sixth encoded data in the first cache file is recorded in memory. The electronic device decodes the sixth encoded data more efficiently than it decodes the fifth encoded data.

4. The method according to any one of claims 1 to 3, characterized in that, The frame types in the first encoded data include keyframes and non-keyframes; the frame types in the second encoded data are all keyframes.

5. The method according to any one of claims 1 to 4, characterized in that, The mapping relationship is recorded in the data structure of the cache information entity of the first video, and the data structure of the cache information entity records the first file path of the first cache file; After the electronic device records the mapping relationship between the timestamp of the second encoded data and the storage location of the second encoded data in the first cache file in memory, the method includes: The electronic device writes the second file path of the second cache file into the data structure of the cache information entity of the first video.

6. The method as described in claim 5, characterized in that, During the process of the video playback application exiting operation, the method further includes: The electronic device generates a third cache file based on the first file path of the first cache file and the second file path of the second cache file recorded in the data structure of the cache information entity of the first video. The third cache file is stored on the disk; Delete the cached information entity of the first video.

7. The method as described in claim 6, characterized in that, After the video playback application restarts, and before obtaining the mapping relationship based on the second cache file, the method further includes: The electronic device creates a data structure in memory for the cache information entity of the first video; The electronic device obtains the first file path of the first cache file and the second file path of the second cache file based on the third cache file; The electronic device writes the first file path and the second file path into the data structure of the cache information entity of the first video; The electronic device obtains the second cache file according to the second file path of the second cache file; After the electronic device obtains the mapping relationship based on the second cache file, the method further includes: The electronic device writes the mapping relationship into the data structure of the cached information entity.

8. The method as described in claim 5 or 7, characterized in that, Before the electronic device reads the third encoded data from the first cache file according to the first storage location, the method includes: The electronic device obtains the first file path of the first cached file from the cache information entity of the first video; The electronic device obtains the first cached file according to the first file path.

9. The method according to any one of claims 5 to 8, characterized in that, The cache information entity of the first video is stored in a linked list. The linked list stores the unique identifier of the first video and the correspondence between the cache information entity of the first video. The linked list also stores the unique identifier of the second video and the correspondence between the cache information entity of the second video. The cache information entity of the first video also includes: the number of video frames corresponding to the encoded data cached in the first cache file and the data size of the first cache file.

10. The method as described in claim 4, characterized in that, During the process of displaying the first video frame obtained based on the first encoded data on the first video playback interface, the method further includes: For each first encoded data, the electronic device decodes the first encoded data to obtain YUV data; The electronic device displays the video frames corresponding to the YUV data obtained by decoding the first encoded data on the first video playback interface.

11. The method as described in claim 10, characterized in that, The electronic device, during the process of writing the second encoded data of the first video frame into the first cache file, includes: For each first encoded data, the electronic device encodes the YUV data obtained by decoding the first encoded data to obtain the second encoded data corresponding to the first encoded data; The electronic device writes the second encoded data corresponding to the first encoded data into the first cache file.

12. An electronic device, characterized in that, It includes one or more processors and one or more memories; wherein the one or more memories are coupled to the one or more processors, and the one or more memories are used to store a computer program, which, when executed by the one or more processors, causes the electronic device to perform the method as described in any one of claims 1-11.

13. A chip system applied to an electronic device, the chip system comprising one or more processors, characterized in that, The processor is configured to invoke computer instructions to cause the electronic device to perform the method as described in any one of claims 1-11.

14. A computer-readable storage medium comprising a computer program, characterized in that, When the computer program is run on an electronic device, it causes the electronic device to perform the method as described in any one of claims 1-11.

Images