Method and device for playing multimedia file stream

Abstract

The invention discloses a method and a device for playing a multimedia file stream, which are used for improving the playing fluency of the multimedia file stream. The method comprises the following steps: receiving a multimedia file stream to be played, analyzing the multimedia file stream to be played, determining the video frame rate and frame type information of the multimedia file stream to be played, and storing the frame type information, the video frame rate and the multimedia file stream to be played into a transition cache region; decoding the multimedia file stream to be played to obtain decoding information and decoding data; when it is determined that the video frame rate is larger than the decoding frame rate in the decoding information and the frame type information comprises key frame information, the storage amount of the multimedia file stream to be played in the transition cache region is adjusted according to a preset frame loss rule; storing the decoded data into a corresponding cache region according to a preset self-adaptive cache rule; and when it is determined that the cache amount of the decoded data in the cache region reaches a preset threshold value, playing the decoded data in the cache region.

Description

Technical Field

[0001] This invention relates to the field of playback technology, and in particular to a method and apparatus for playing multimedia file streams. Background Technology

[0002] Currently, with the rapid development of the times and the continuous progress of technology, people have put forward the demand for video playback in various electronic devices such as electronic photo frames, children's cameras, door locks and other embedded electronic devices.

[0003] Specifically, in embedded devices, video playback is typically achieved using third-party software libraries such as ffmpeg, gstreamr, and exoplayer, as well as hardware decoding interfaces provided by integrated circuit chip (System on Chip, SoC) manufacturers to decode the video and audio data of multimedia files.

[0004] However, since video playback is relatively smooth when the decoding performance is higher than the actual playback frame rate, and the SoC's decoding performance generally has a margin, under high system operating conditions, the SoC's hardware or software decoding may not reach its maximum performance, resulting in lower playback smoothness of multimedia file streams.

[0005] It is evident that improving the smoothness of multimedia file playback on embedded device players is an urgent problem to be solved. Summary of the Invention

[0006] This invention provides a method and apparatus for playing multimedia file streams, which improves the smoothness of playing multimedia files.

[0007] In a first aspect, embodiments of the present invention provide a method for playing a multimedia file stream, applied to an electronic device, the method comprising:

[0008] Receive the multimedia file stream to be played, and parse the multimedia file stream to be played to determine the video frame rate and frame type information of the multimedia file stream to be played, and store the frame type information, the video frame rate and the multimedia file stream to be played into the transition buffer area.

[0009] Once it is determined that the information exists in the transition buffer area, the multimedia file stream to be played is decoded to obtain decoding information and decoding data indicating the complete content of the multimedia file stream to be played.

[0010] When it is determined that the video frame rate is greater than the decoding frame rate in the decoding information, and the frame type information includes keyframe information, the storage amount of the multimedia file stream to be played in the transition buffer area is adjusted according to the preset frame dropping rules; and the decoding data is stored in the corresponding buffer area according to the preset adaptive caching rules.

[0011] When the amount of decoded data in the buffer area reaches a preset threshold, the decoded data in the buffer area is played.

[0012] In one possible implementation, when it is determined that the video frame rate is greater than the decoding frame rate in the decoding information, and the frame type information includes keyframe information, the storage amount of the multimedia file stream to be played in the transition buffer area is adjusted according to a preset frame dropping rule, including:

[0013] The target number of decodes is determined based on the decoding start time in the decoding information, the current decoding time corresponding to the keyframe information in the frame type information, and the video frame rate;

[0014] Based on the target number of decoded data and the preset frame dropping rules, the adjustment information for the transition buffer region is determined.

[0015] In one possible implementation, determining the storage information of the transition buffer region based on the target decoding quantity and the preset frame dropping rule includes:

[0016] Based on the target decoding quantity information and the decoding start quantity information in the decoding information, the number of frames to be discarded is determined;

[0017] Based on the number of frames to be discarded and the number of decodable frames in the transition buffer region, the adjustment information of the transition buffer region is determined.

[0018] In one possible implementation, after obtaining the decoded data, the method further includes:

[0019] Determine the device type of the electronic device.

[0020] In one possible implementation, the decoded data is stored in the corresponding cache area according to a preset adaptive caching rule, including:

[0021] If the device type of the electronic device is an embedded device, then the cache area is determined to include a first cache area and a second cache area;

[0022] Based on the decoding frame rate, the video frame rate, and the preset adaptive caching rule, determine the first cache quantity to be cached in the first cache region and the second cache quantity to be cached in the second cache region;

[0023] Once it is determined that the first cache area has cached the first cache amount of decoded data, the data other than the first cache amount of decoded data are sequentially stored in the second cache area.

[0024] In one possible implementation, the first cache region is a memory-based cache region, and the second cache region is a cache region based on flash memory storage space.

[0025] Secondly, embodiments of the present invention provide a multimedia file stream playback device, applied to an electronic device, the device comprising:

[0026] The processing unit is used to receive a multimedia file stream to be played, and to parse the multimedia file stream to be played to determine the video frame rate and frame type information of the multimedia file stream to be played, and to store the frame type information, the video frame rate and the multimedia file stream to be played into a transition buffer area.

[0027] The obtaining unit is used to perform decoding processing on the multimedia file stream to be played after determining that the information exists in the transition buffer area, and to obtain decoding information and decoding data indicating the complete content of the multimedia file stream to be played;

[0028] The caching unit is configured to, when it is determined that the video frame rate is greater than the decoding frame rate in the decoding information and the frame type information includes keyframe information, adjust the storage amount of the multimedia file stream to be played in the transition caching area according to a preset frame dropping rule; and store the decoding data into the corresponding caching area according to a preset adaptive caching rule.

[0029] The playback unit is used to play the decoded data in the buffer area when it is determined that the amount of decoded data in the buffer area has reached a preset threshold.

[0030] In one possible implementation, the cache unit is used for:

[0031] The target number of decodes is determined based on the decoding start time in the decoding information, the current decoding time corresponding to the keyframe information in the frame type information, and the video frame rate;

[0032] Based on the target number of decoded data and the preset frame dropping rules, the adjustment information for the transition buffer region is determined.

[0033] In one possible implementation, the cache unit is used for:

[0034] Based on the target decoding quantity information and the decoding start quantity information in the decoding information, the number of frames to be discarded is determined;

[0035] Based on the number of frames to be discarded and the number of decodable frames in the transition buffer region, the adjustment information of the transition buffer region is determined.

[0036] In one possible implementation, after obtaining the decoded data, the apparatus further includes a determining unit for:

[0037] Determine the device type of the electronic device.

[0038] In one possible implementation, the cache unit is used for:

[0039] If the device type of the electronic device is an embedded device, then the cache area is determined to include a first cache area and a second cache area;

[0040] Based on the decoding frame rate, the video frame rate, and the preset adaptive caching rule, determine the first cache quantity to be cached in the first cache region and the second cache quantity to be cached in the second cache region;

[0041] Once it is determined that the first cache area has cached the first cache amount of decoded data, the data other than the first cache amount of decoded data are sequentially stored in the second cache area.

[0042] In one possible implementation, the first cache region is a memory-based cache region, and the second cache region is a cache region based on flash memory storage space.

[0043] Thirdly, embodiments of the present invention provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps included in any of the methods in the first aspect.

[0044] Fourthly, embodiments of the present invention provide a computer-readable storage medium including program code, which, when the program product is run on an electronic device, causes the electronic device to perform the steps included in any of the methods in the first aspect.

[0045] Fifthly, embodiments of the present invention provide a computer program product, the computer program product comprising: computer program code, which, when executed on an electronic device, causes the electronic device to perform the steps included in any of the methods in the first aspect.

[0046] The technical solutions provided by the embodiments of the present invention bring at least the following beneficial effects:

[0047] In this embodiment of the invention, a multimedia file stream to be played can be received and parsed to determine the video frame rate and frame type information of the multimedia file stream to be played. The frame type information, video frame rate, and the multimedia file stream to be played are stored in a transition buffer area. When it is determined that the transition buffer area contains information, the multimedia file stream to be played is decoded to obtain decoding information and decoding data indicating the complete content of the multimedia file stream to be played. When it is determined that the video frame rate is greater than the decoding frame rate in the decoding information and the frame type information includes keyframe information, the storage amount of the multimedia file stream to be played in the transition buffer area is adjusted according to a preset frame dropping rule. The decoded data is stored in the corresponding buffer area according to a preset adaptive caching rule. When it is determined that the cached amount of decoded data in the buffer area reaches a preset threshold, the decoded data in the buffer area is played.

[0048] In other words, in this embodiment of the invention, the use of space resources in the cache area is fully considered, and a scheme for adjusting the caching of multimedia file streams in the transition cache area by frame dropping rules is provided to solve the playback stuttering problem caused by the large instantaneous flow of multimedia file streams. A scheme for caching decoded data based on preset adaptive caching rules is also provided to solve the problem of unsmooth playback of multimedia file streams in electronic devices with weak decoding performance, thereby improving the smoothness of multimedia file playback.

[0049] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practice. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings.

[0050] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0051] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention, but do not constitute an undue limitation of the invention.

[0052] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of the present invention;

[0053] Figure 2 A flowchart illustrating a method for playing a multimedia file stream, provided as an embodiment of the present invention;

[0054] Figure 3 A structural block diagram of a multimedia file stream playback device provided in an embodiment of the present invention;

[0055] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention. Without conflict, the embodiments and features in the embodiments of this invention can be arbitrarily combined with each other. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here. The term "comprising" and any variations thereof in the specification and claims of this invention are intended to cover non-exclusive protection. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0057] In this embodiment of the invention, "and / or" describes the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following associated objects have an "or" relationship.

[0058] Furthermore, unless otherwise stated, the ordinal numbers such as "first" and "second" mentioned in the embodiments of this invention are used to distinguish multiple objects and are not used to limit the size, content, order, timing, priority, or importance of multiple objects. For example, "first cache area" and "second cache area" are only used to distinguish cache areas and do not indicate that the two cache areas are different in size, priority, or importance.

[0059] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These are to be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description. It should be noted that in the embodiments of the present invention, references may be made to existing industry solutions such as software, components, and models. These should be considered exemplary, intended only to illustrate the feasibility of implementing the technical solutions of the present invention, and do not imply that the applicant has already used or necessarily used such solutions.

[0060] In the technical solution of this invention, the collection, dissemination, and use of multimedia files such as video and audio all comply with the requirements of relevant national laws and regulations.

[0061] To facilitate understanding of the technical solutions provided in the embodiments of the present invention, some key terms used in the embodiments of the present invention will be explained here first:

[0062] 1. ffmpeg is an open-source audio and video processing tool that can be used for audio and video acquisition, encoding, decoding, transcoding, and processing.

[0063] 2. GStreamer is a powerful multimedia framework that can be used for audio and video acquisition, encoding, decoding, processing, and transmission.

[0064] 3. Exoplayer is a high-performance open-source media player library provided by Google for the Android and JavaFX platforms.

[0065] 4. H.264: An encoding standard used by most MP4 video files; H.265: An encoding standard, an upgraded version of H.264; MPEG2: An encoding standard used by older videos; MPEG4: An encoding standard used by older videos; WMV: An encoding standard defined by Microsoft, used by WMV video files.

[0066] As mentioned earlier, while smooth video playback is possible when decoding performance exceeds the actual playback frame rate, and SoC decoding performance typically has a margin of safety, under high system operating conditions, SoC hardware or software decoding may not reach its maximum performance, resulting in lower playback smoothness for multimedia file streams. Therefore, improving the smoothness of multimedia file stream playback on embedded device players is a pressing issue that needs to be addressed.

[0067] In view of this, the present invention provides a method for playing a multimedia file stream. This method involves receiving a multimedia file stream to be played, parsing the stream to determine its video frame rate and frame type information, and storing the frame type information, video frame rate, and the multimedia file stream in a transition buffer area. Once the existence of information in the transition buffer area is confirmed, the multimedia file stream to be played is decoded to obtain decoding information and decoded data indicating the complete content of the stream. When it is determined that the video frame rate is greater than the decoding frame rate in the decoding information, and the frame type information includes keyframe information, the storage amount of the multimedia file stream to be played in the transition buffer area is adjusted according to a preset frame dropping rule. The decoded data is then stored in the corresponding buffer area according to a preset adaptive caching rule. When the cached amount of decoded data in the buffer area reaches a preset threshold, the decoded data in the buffer area is played.

[0068] In other words, in this embodiment of the invention, the use of space resources in the cache area is fully considered, and a scheme for adjusting the caching of multimedia file streams in the transition cache area by frame dropping rules is provided to solve the playback stuttering problem caused by the large instantaneous flow of multimedia file streams. A scheme for caching decoded data based on preset adaptive caching rules is also provided to solve the problem of unsmooth playback of multimedia file streams in electronic devices with weak decoding performance, thereby improving the smoothness of multimedia file stream playback.

[0069] After introducing the design concept of the embodiments of the present invention, the following is a brief introduction to the application scenarios applicable to the technical solutions of the embodiments of the present invention. It should be noted that the application scenarios described in the embodiments of the present invention are for the purpose of more clearly illustrating the technical solutions of the embodiments of the present invention, and do not constitute a limitation on the technical solutions provided by the embodiments of the present invention. As those skilled in the art will know, with the emergence of new application scenarios, the technical solutions provided by the embodiments of the present invention are also applicable to similar technical problems.

[0070] Please see Figure 1 The diagram illustrates a scenario to which the invention's embodiments can be applied. This scenario includes an information acquisition device 101 and an electronic device 102. The multimedia file stream playback method of this embodiment can be achieved through... Figure 1 The information acquisition device 101 and the electronic device 102 work together to achieve this, or it can be achieved solely through the electronic device 102. This embodiment of the invention does not limit the implementation in this way. The electronic device 102 may include one or more processors 1021, a memory 1022, and an I / O interface 1023 that interacts with the information acquisition device 101.

[0071] In this embodiment of the invention, the aforementioned information acquisition device 101 can generate a multimedia file stream. After the information acquisition device 101 generates the multimedia file stream, it can send the multimedia file stream to the electronic device 102 via the network 103. Electronic device 102 can receive the multimedia file stream to be played sent by information acquisition device 101 through I / O interface 1023. Then, processor 1021 receives the multimedia file stream to be played and performs parsing processing on it to determine the video frame rate and frame type information of the multimedia file stream to be played. The frame type information, video frame rate, and multimedia file stream to be played are stored in a transition buffer area. When it is determined that there is information in the transition buffer area, the multimedia file stream to be played is decoded to obtain decoding information and decoding data indicating the complete content of the multimedia file stream to be played. When it is determined that the video frame rate is greater than the decoding frame rate in the decoding information and the frame type information includes keyframe information, the storage amount of the multimedia file stream to be played in the transition buffer area is adjusted according to the preset frame dropping rules. The decoded data is stored in the corresponding buffer area according to the preset adaptive caching rules. When it is determined that the cached amount of decoded data in the buffer area reaches the preset threshold, the decoded data in the buffer area is played.

[0072] In specific implementation, multiple electronic devices 102 may interact with multiple information collection devices 101, or one electronic device 102 may interact with multiple information collection devices 101, or one electronic device 102 may interact with one information collection device 101. No limitation is made in this embodiment of the invention. Figure 1 The example shown is an interaction between an electronic device 102 and an information acquisition device 101.

[0073] In this embodiment of the invention, the electronic device 102 may be, for example, a tablet computer (PAD), a personal computer (PC), a smartphone, a smart TV, a smart in-vehicle device, a wearable device, an electronic photo frame, a children's camera, a door lock, or other devices that can receive and play audio and video. This embodiment of the invention does not limit the scope of the invention.

[0074] In this embodiment of the invention, the information collection device 101 may be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms, or it may be a physical server, but it is not limited to these.

[0075] The information acquisition device 101 and the electronic device 102 can communicate with each other through one or more networks 103. The network 103 can be a wired network or a wireless network. For example, the wireless network can be a mobile cellular network or a Wi-Fi (Wi-Fi) network. Of course, it can also be other possible networks. This embodiment of the invention does not limit the types of networks.

[0076] Of course, the method provided in the embodiments of the present invention is not limited to... Figure 1 The application scenarios shown can also be used in other possible scenarios, such as a method for playing multimedia file streams on an electronic device. This embodiment of the invention does not impose any limitations. Figure 1 The functions that each device in the application scenario shown can achieve will be described in subsequent method embodiments, and will not be elaborated on here.

[0077] To further illustrate the multimedia file stream playback method provided in the embodiments of the present invention, a detailed description is provided below in conjunction with the accompanying drawings and specific implementation methods. Although the embodiments of the present invention provide method operation steps as shown in the following embodiments or drawings, more or fewer operation steps may be included in the method based on conventional or non-inventive methods. In steps where there is no logically necessary causal relationship, the execution order of these steps is not limited to the execution order provided in the embodiments of the present invention. In actual processing or device execution, the method may be executed in the order shown in the embodiments or drawings, or in parallel (e.g., in a parallel processor or multi-threaded processing application environment).

[0078] The following combination Figure 2 The flowchart shown illustrates the method for playing multimedia file streams in this embodiment of the invention. The method can be executed by an electronic device, and the specific implementation flow of the method is as follows:

[0079] Step 201: Receive the multimedia file stream to be played, and parse the multimedia file stream to be played to determine the video frame rate and frame type information of the multimedia file stream to be played, and store the frame type information, video frame rate and multimedia file stream to be played into the transition buffer area.

[0080] In this embodiment of the invention, the electronic device can receive a multimedia file stream to be played transmitted over a network, and then determine the video frame rate and frame type information of the multimedia file stream. Optionally, the video frame rate can also be called the playback frame rate, for example, represented by fps1. The frame type information includes keyframe information and non-keyframe information. Wherein, if the frame type information corresponding to the multimedia file stream includes keyframe information, then the decoding of the multimedia file stream frames, except for the keyframes, requires the information of the previous or next frame, or the keyframe, to be decoded.

[0081] In this embodiment of the invention, a multimedia file stream can be understood as an information stream including video streams and / or audio streams. Optionally, the multimedia file stream can be directly captured by an electronic device or obtained from other information acquisition devices; this embodiment of the invention does not limit this. Optionally, a video stream can be a video with a transmission duration of less than 3 minutes (e.g., short videos in the prior art); a video stream can also be a video with a transmission duration of more than 3 minutes (e.g., long videos in the prior art). Of course, the content of the video stream can be videos generated in various scenarios; this embodiment of the invention does not limit the duration and content of the video stream.

[0082] In this embodiment of the invention, after the electronic device obtains the video frame rate and frame type information of the multimedia file stream to be played, it can store the frame type information, video frame rate, and the multimedia file stream to be played into a transition buffer area. The transition buffer area is, for example, represented by buffer1. Optionally, the size of buffer1 corresponds to the frame rate of the multimedia file stream to be played.

[0083] Step 202: After determining that information exists in the transition buffer area, decode the multimedia file stream to be played to obtain decoding information and decoding data indicating the complete content of the multimedia file stream to be played.

[0084] Step 203: When it is determined that the video frame rate is greater than the decoding frame rate in the decoding information and the frame type information includes keyframe information, the storage amount of the multimedia file stream to be played in the transition buffer area is adjusted according to the preset frame dropping rules; and the decoding data is stored in the corresponding buffer area according to the preset adaptive caching rules.

[0085] Step 204: When the amount of decoded data in the buffer area reaches the preset threshold, the decoded data in the buffer area is played.

[0086] In an embodiment of the present invention, considering that there are video files with multiple coding standards in the related art, such as H.264, H.265, WMV, MPEG-2, MPEG-4, etc., the characteristic of these video files is that when decoding video frames, except for key frames, other video frames need to rely on the information of the previous frame, the subsequent frame, or the key frame to be decoded. In view of this situation, in an embodiment of the present invention, a transitional buffer area is added between parsing the bitstream and decoding the video file, that is, a buffer area is provided, so as to ensure that after reading the video frame associated with the key frame, the data before the key frame can be discarded, avoiding the situation of low playback fluency caused by the discontinuous playback of video content.

[0087] In an embodiment of the present invention, the electronic device can determine the target decoding quantity information according to the decoding start time, the frame type information (which is the key frame information corresponding to the current decoding time), and the video frame rate in the decoding information; and determine the adjustment information of the transitional buffer area according to the target decoding quantity information and the preset frame dropping rule. Specifically, the electronic device can determine the number of frames to be discarded information based on the target decoding quantity information and the decoding start quantity information in the decoding information; and determine the adjustment information of the transitional buffer area based on the number of frames to be discarded information and the decodable frame number information in the transitional buffer area.

[0088] For example, let the decoding start time be represented by time1, and the value corresponding to the decoding start quantity information, which is the number of decoded frames at the start of recording, be represented by num1. When a new key frame is put into buffer1, the electronic device can obtain the current system time as the current decoding time, and represent the current decoding time by time2. Then, it can calculate (time2 - time1) * fps1, and thus obtain the number of frames that the decoder of the electronic device should theoretically decode, that is, the target decoding quantity, which is represented by num2. If num1 + (frame_num / 2) < num2, then num2 - num1 is the number of frames to be discarded, drop_num, which is the value corresponding to the aforementioned number of frames to be discarded information. Here, (frame_num / 2) is used to represent the trigger frame dropping threshold, and this trigger frame dropping threshold can be a smaller value or a larger value, and can be adjusted according to the actual implementation. In an embodiment of the present invention, no limitation is made thereto.

[0089] In this embodiment of the invention, after obtaining the value corresponding to the number of frames to be dropped, i.e., drop_num, the electronic device can also obtain the number of decodable frames in buffer1, i.e., the number of decodable frames in the transition buffer area, which is represented by pkt_num. Then, pkt_num - drop_num - 1 can be calculated to obtain the number of frames that still need to be decoded before dropping the frames. When the remaining number of frames has been decoded, frame dropping processing can be triggered, deleting the data of the previous frames from buffer1, thereby achieving the goal of increasing drop_num to increase the number of frames to be decoded in num1. It should be noted that the maximum value of drop_num will not exceed pkt_num - 1.

[0090] In this embodiment of the invention, the electronic device can also determine its device type. If the device type of the electronic device is determined to be an embedded device, then the cache area is determined to include a first cache area and a second cache area. Optionally, the first cache area is a memory-based cache area, and the second cache area is a cache area based on flash memory (Flash EEPROM Memory, flash) storage space.

[0091] In this embodiment of the invention, the electronic device can determine the first cache quantity to be cached in the first cache region and the second cache quantity to be cached in the second cache region based on the decoding frame rate, the video frame rate, and a preset adaptive caching rule. Then, the decoded data of the first cache quantity can be cached in the first cache region in a manner that specifies the memory address, and the decoded data of the first cache quantity can be cached in the second cache region in a manner that specifies the file name.

[0092] In the specific implementation process, after it is determined that the first cache area has cached the first cache amount of decoded data, the data in the decoded data other than the first cache amount of decoded data are stored into the second cache area in sequence.

[0093] For example, if the multimedia file stream is a video file, and the actual playback frame rate of the video file is greater than the decoding frame rate, the electronic device can first obtain the video length, and then determine the difference between the playback frame rate and the decoding frame rate, and multiply the difference by the video length as the number of frames that need to be cached in advance. For example, assuming the video file is 10 seconds long, the playback frame rate is 30fps, and the decoding performance of the video at this resolution is 28fps, then it can be determined that the number of frames that need to be cached in advance is 20 frames.

[0094] Furthermore, it can be determined whether the device type of the electronic device is an embedded device.

[0095] Optionally, if the electronic device is a non-embedded device, the electronic device can create a buffer based on runtime memory, and trigger playback of the video file when the number of frames corresponding to the decoded data reaches 20 frames.

[0096] Optionally, if the electronic device is an embedded device, since RAM is relatively scarce in embedded devices, a larger cache is needed when the video file is 20 seconds long or longer, which the embedded device's RAM may not be sufficient for. Therefore, the electronic device can create a second storage area based on flash memory. Furthermore, after determining the first cache size and the second cache size created from RAM, assuming that the first storage area can buffer a maximum of 10 frames (i.e., the first cache size), data can be written to the second storage area, i.e., flash memory.

[0097] In this embodiment of the invention, when the electronic device determines that the playback thread of the player has finished playing the video file, if the cached decoding data is obtained from memory, it is returned to the decoding thread of the decoder; if the cached decoding data is obtained from flash, it can be deleted.

[0098] In this embodiment of the invention, the electronic device includes a parser, a player, and a decoder. The parser executes a file parsing thread, the player executes a playback thread, and the decoder executes a decoding thread; in other words, the file parsing thread, playback thread, and decoding thread are three independent threads. Optionally, the communication method between the file parsing thread, playback thread, and decoding thread can be a message queue. Thus, due to pre-caching, even if playback follows the original frame rate of the multimedia file stream, the playback thread will not fail to obtain the decoded data, thereby improving the smoothness of video file playback. Furthermore, since this embodiment of the invention also provides a frame dropping rule to adjust the caching of the multimedia file stream in the transition buffer area, it avoids the problem of excessively long pre-caching times, thereby solving the playback stuttering problem caused by large instantaneous traffic of the multimedia file stream. In other words, the solution provided by this embodiment of the invention improves the playback smoothness of some video format multimedia file streams with long instantaneous decoding times, further enhancing the user experience.

[0099] Based on the same inventive concept, embodiments of the present invention provide a playback device for multimedia file streams, which can realize the functions corresponding to the aforementioned multimedia file stream playback method. The multimedia file stream playback device can be a hardware structure, a software module, or a hardware structure plus a software module. The multimedia file stream playback device can be implemented by a chip system, which can be composed of chips or include chips and other discrete components. Please refer to [link to previous text]. Figure 3 As shown, the playback device for the multimedia file stream includes:

[0100] The processing unit 301 is used to receive a multimedia file stream to be played, and to parse the multimedia file stream to be played to determine the video frame rate and frame type information of the multimedia file stream to be played, and to store the frame type information, the video frame rate and the multimedia file stream to be played into a transition buffer area.

[0101] The obtaining unit 302 is used to perform decoding processing on the multimedia file stream to be played after determining that the information of the transition buffer area exists, and to obtain decoding information and decoding data indicating the complete content of the multimedia file stream to be played;

[0102] The caching unit 303 is configured to, when it is determined that the video frame rate is greater than the decoding frame rate in the decoding information and the frame type information includes keyframe information, adjust the storage amount of the multimedia file stream to be played in the transition buffer area according to a preset frame dropping rule; and store the decoding data into the corresponding buffer area according to a preset adaptive caching rule.

[0103] The playback unit 304 is used to play the decoded data in the buffer area when it is determined that the amount of decoded data in the buffer area has reached a preset threshold.

[0104] In one possible implementation, the cache unit 303 is used for:

[0105] The target number of decodes is determined based on the decoding start time in the decoding information, the current decoding time corresponding to the keyframe information in the frame type information, and the video frame rate;

[0106] Based on the target number of decoded data and the preset frame dropping rules, the adjustment information for the transition buffer region is determined.

[0107] In one possible implementation, the cache unit 303 is used for:

[0108] Based on the target decoding quantity information and the decoding start quantity information in the decoding information, the number of frames to be discarded is determined;

[0109] Based on the number of frames to be discarded and the number of decodable frames in the transition buffer region, the adjustment information of the transition buffer region is determined.

[0110] In one possible implementation, after obtaining the decoded data, the apparatus further includes a determining unit for:

[0111] Determine the device type of the electronic device.

[0112] In one possible implementation, the cache unit 303 is used for:

[0113] If the device type of the electronic device is an embedded device, then the cache area is determined to include a first cache area and a second cache area;

[0114] Based on the decoding frame rate, the video frame rate, and the preset adaptive caching rule, determine the first cache quantity to be cached in the first cache region and the second cache quantity to be cached in the second cache region;

[0115] Once it is determined that the first cache area has cached the first cache amount of decoded data, the data other than the first cache amount of decoded data are sequentially stored in the second cache area.

[0116] In one possible implementation, the first cache region is a memory-based cache region, and the second cache region is a cache region based on flash memory storage space.

[0117] All relevant content of each step involved in the aforementioned embodiments of the multimedia file stream playback method can be referenced to the functional description of the corresponding functional module of the multimedia file stream playback device in the embodiments of the present invention, and will not be repeated here.

[0118] The module division in this embodiment of the invention is illustrative and represents only one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in the various embodiments of the invention can be integrated into a single controller, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0119] Based on the same inventive concept, embodiments of the present invention provide an electronic device, please refer to... Figure 4 As shown, the electronic device includes at least one processor 401 and a memory 402 connected to the at least one processor. In this embodiment of the invention, the specific connection medium between the processor 401 and the memory 402 is not limited. Figure 4Taking the connection between processor 401 and memory 402 via bus 400 as an example, bus 400 in... Figure 4 The connections between other components are indicated by thick lines and are for illustrative purposes only, not as limiting information. The 400 bus can be divided into address bus, data bus, control bus, etc., for ease of representation. Figure 4 The symbol is represented by only one thick line, but this does not indicate that there is only one bus or one type of bus. In addition, the multimedia file stream playback device also includes a communication interface 403 for receiving or sending data.

[0120] In this embodiment of the invention, the memory 402 stores instructions that can be executed by at least one processor 401. By executing the instructions stored in the memory 402, at least one processor 401 can perform the steps included in the aforementioned multimedia file stream playback method.

[0121] The processor 401 is the control center of the electronic device. It can connect to various parts of the electronic device through various interfaces and lines. By running or executing instructions stored in the memory 402 and calling data stored in the memory 402, it can monitor the various functions and data processing of the electronic device as a whole.

[0122] Optionally, processor 401 may include one or more processing units. Processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into processor 401. In some embodiments, processor 401 and memory 402 may be implemented on the same chip; in some embodiments, they may be implemented separately on independent chips.

[0123] Processor 401 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit, field-programmable gate array or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this invention can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0124] Memory 402, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 402 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. Memory 402 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. In embodiments of the present invention, memory 402 may also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0125] By designing and programming the processor 401, the code corresponding to the multimedia file stream playback method described in the foregoing embodiments can be embedded into the chip, so that the chip can execute the steps of the aforementioned multimedia file stream playback method when running. How to design and program the processor 401 is a well-known technique to those skilled in the art, and will not be described in detail here.

[0126] Based on the same inventive concept, embodiments of the present invention also provide a computer-readable storage medium including program code, which, when the program product is run on an electronic device, is used to cause the electronic device to perform the steps of the aforementioned multimedia file stream playback method.

[0127] In some possible implementations, various aspects of the multimedia file stream playback method provided by the present invention can also be implemented as a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the steps in the multimedia file stream playback method according to various exemplary embodiments of the present invention described above.

[0128] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0129] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0130] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0131] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0132] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for playing a multimedia file stream, characterized in that, Applied to electronic devices, the method includes: Receive the multimedia file stream to be played, and parse the multimedia file stream to be played to determine the video frame rate and frame type information of the multimedia file stream to be played, and store the frame type information, the video frame rate and the multimedia file stream to be played into the transition buffer area. Once it is determined that the information exists in the transition buffer area, the multimedia file stream to be played is decoded to obtain decoding information and decoding data indicating the complete content of the multimedia file stream to be played. When it is determined that the video frame rate is greater than the decoding frame rate in the decoding information, and the frame type information includes keyframe information, the storage amount of the multimedia file stream to be played in the transition buffer area is adjusted according to the preset frame dropping rules; and the decoding data is stored in the corresponding buffer area according to the preset adaptive caching rules. When the amount of decoded data in the buffer area reaches a preset threshold, the decoded data in the buffer area is played.

2. The method as described in claim 1, characterized in that, When it is determined that the video frame rate is greater than the decoding frame rate in the decoding information, and the frame type information includes keyframe information, the storage amount of the multimedia file stream to be played in the transition buffer area is adjusted according to a preset frame dropping rule, including: The target number of decodes is determined based on the decoding start time in the decoding information, the current decoding time corresponding to the keyframe information in the frame type information, and the video frame rate; Based on the target number of decoded data and the preset frame dropping rules, the adjustment information for the transition buffer region is determined.

3. The method as described in claim 2, characterized in that, Based on the target number of decodes and the preset frame dropping rules, the storage information of the transition buffer region is determined, including: Based on the target decoding quantity information and the decoding start quantity information in the decoding information, the number of frames to be discarded is determined; Based on the number of frames to be discarded and the number of decodable frames in the transition buffer region, the adjustment information of the transition buffer region is determined.

4. The method according to any one of claims 1-3, characterized in that, After obtaining the decoded data, the method further includes: Determine the device type of the electronic device.

5. The method as described in claim 4, characterized in that, The decoded data is stored in the corresponding cache area according to a preset adaptive caching rule, including: If the device type of the electronic device is an embedded device, then the cache area is determined to include a first cache area and a second cache area; Based on the decoding frame rate, the video frame rate, and the preset adaptive caching rule, determine the first cache quantity to be cached in the first cache region and the second cache quantity to be cached in the second cache region; Once it is determined that the first cache area has cached the first cache amount of decoded data, the data other than the first cache amount of decoded data are sequentially stored in the second cache area.

6. The method as described in claim 5, characterized in that, The first cache area is a memory-based cache area, and the second cache area is a cache area based on flash memory storage space.

7. A multimedia file stream playback device, characterized in that, Applied to electronic devices, the device includes: The processing unit is used to receive a multimedia file stream to be played, and to parse the multimedia file stream to be played to determine the video frame rate and frame type information of the multimedia file stream to be played, and to store the frame type information, the video frame rate and the multimedia file stream to be played into a transition buffer area. The obtaining unit is used to perform decoding processing on the multimedia file stream to be played after determining that the information exists in the transition buffer area, and to obtain decoding information and decoding data indicating the complete content of the multimedia file stream to be played; The caching unit is configured to, when it is determined that the video frame rate is greater than the decoding frame rate in the decoding information and the frame type information includes keyframe information, adjust the storage amount of the multimedia file stream to be played in the transition caching area according to a preset frame dropping rule; and store the decoding data into the corresponding caching area according to a preset adaptive caching rule. The playback unit is used to play the decoded data in the buffer area when it is determined that the amount of decoded data in the buffer area has reached a preset threshold.

8. An electronic device, characterized in that, The electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1-6.

10. A computer program product, characterized in that, The computer program product includes: computer program code, which, when run on an electronic device, causes the electronic device to perform the method described in any one of claims 1-6.

Images