Media file playing method and device, electronic equipment and storage medium
By calculating the keyframe transmission delay and decoding time of the media file, and determining the target start-up buffer time, the problem of buffer size affecting the start-up speed in the prior art is solved, and the stable and efficient playback of the media file is achieved.
Patent Information
- Application Number
- CN202411698558.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-25
- Publication Date
- 2025-02-11
AI Technical Summary
In the prior art, the starting speed of media files is limited by the buffer size. If it is too large, the waiting time will be long, and if it is too small, the playback may be interrupted.
By determining the keyframe transmission delay and decoding time of the media file to be played, the target start-up buffer time is calculated, and the media file is played based on this time.
Accurately determine the start buffering time to avoid excessive buffering or insufficient buffering, ensure normal playback of media files, and improve playback adaptability and stability.
Smart Images
Figure CN119420988A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present disclosure relate to the technical field of media playback, and in particular to a method, device, electronic device, and computer-readable storage medium for playing a media file. Background Art
[0002] In some digital TV broadcasting standards, the playback end (such as a TV or streaming device) usually cannot start decoding and displaying the received video frames immediately because the playback end needs a certain buffering time to wait for enough data to accumulate in the buffer until it can be played.
[0003] In the related art, the size of the buffer is usually set to a fixed value, which often affects the playback speed of the media data. For example, if the buffer is set too large, the playback device needs to wait longer to accumulate enough data, and if the buffer is too small, the playback device may not be able to handle network fluctuations or data bursts, resulting in playback interruptions. Summary of the invention
[0004] The embodiments of the present disclosure provide a method, device, electronic device and computer-readable storage medium for playing a media file, which can automatically select a playback source with the highest priority from multiple playback sources to play a target program, thereby improving the flexibility of program playback.
[0005] In a first aspect, an embodiment of the present disclosure provides a method for playing a media file, the method comprising:
[0006] Determine the key frame transmission delay corresponding to the media file to be played;
[0007] Determine a decoding time of a decoder for a frame sequence in the media file;
[0008] Determining a target start buffer time corresponding to the media file according to the key frame transmission delay and the decoding time;
[0009] The media file is played based on the target start buffer time.
[0010] In a second aspect, an embodiment of the present disclosure further provides a device for playing a media file, the device comprising:
[0011] A first determination module, used to determine a key frame transmission delay corresponding to a media file to be played;
[0012] A second determination module, used to determine a decoding time of a decoder for a frame sequence in the media file;
[0013] A third determination module, configured to determine a target start buffer time corresponding to the media file according to the key frame transmission delay and the decoding time;
[0014] The playing module is used to play the media file based on the target start buffering time.
[0015] In a third aspect, an embodiment of the present disclosure further provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps in the above-mentioned method for playing a media file when executed by the processor.
[0016] In a fourth aspect, the embodiments of the present disclosure further provide a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the above-mentioned method for playing a media file are implemented.
[0017] In a fifth aspect, the embodiments of the present disclosure further provide a computer program product or a computer program, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the various optional implementations described in the embodiments of the present disclosure.
[0018] In summary, the disclosed embodiment first determines the key frame transmission delay corresponding to the media file to be played, then determines the decoding time of the frame sequence in the media file by the decoder, and then determines the target start buffer time corresponding to the media file based on the key frame transmission delay and decoding time, and finally plays the media file based on the target start buffer time. Thus, the target start buffer time can be accurately determined through the key frame transmission delay and decoding time corresponding to the media file, which can avoid over-buffering or under-buffering. Over-buffering will waste system resources and user time, while under-buffering may cause playback interruptions. By accurately calculating the start buffer time, the normal playback of media files can be more guaranteed, and the adaptability and stability of playback can be improved.
[0019] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] In order to more clearly illustrate the technical solutions in the present disclosure, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.
[0021] Figure 1 is a flowchart of a method for playing a media file provided in the first embodiment of the present disclosure;
[0022] Figure 2 is a flowchart of a method for playing a media file provided in the second embodiment of the present disclosure;
[0023] Figure 3 is a flowchart of a method for playing a media file provided in the third embodiment of the present disclosure;
[0024] Figure 4 is a schematic diagram of the structure of a GOP provided by an embodiment of the present disclosure;
[0025] Figure 5 is a start-up flow chart provided by an embodiment of the present disclosure;
[0026] Figure 6 is a schematic diagram of the structure of a media file playback device provided by an embodiment of the present disclosure;
[0027] Figure 7 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION
[0028] Some embodiments of the present disclosure will be described in detail here, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. Various changes, modifications and equivalents of the methods, devices and / or systems described herein will become apparent after understanding the present disclosure. For example, the order of operations described herein is merely an example and is not limited to those orders set forth herein, but can be changed as becomes apparent after understanding the present disclosure, except for operations that must be performed in a specific order. In addition, for clarity and brevity, descriptions of features known in the art may be omitted.
[0029] The embodiments described in some embodiments of the present disclosure below do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.
[0030] The following is a brief description of some professional terms that may be involved in the embodiments of the present disclosure.
[0031] ATSC 3.0 is a new digital television broadcasting standard developed by the American Television Systems Committee. It uses the MMTP / ROUTER transmission protocol for audio and video data transmission. The audio and video data format is fMP4 (Fragmented MP4).
[0032] Among them, fMP4 is a container format, mainly used for adaptive streaming and dynamic streaming based on the Hypertext Transfer Protocol HTTP. fMP4 files can contain audio data. In some scenarios, video and audio will be encapsulated in the same fMP4 file and distinguished by different tracks.
[0033] The fMP4 format complies with the ISO / IEC 14496-12 standard and is an implementation of the ATSC 3.0 standard for storing and transmitting media data, especially for online video streaming. Unlike traditional single MP4 files, fmp4 files break down media data into multiple segments, each of which contains data for one or more media tracks.
[0034] The video encoding format of media data is mainly H.265. The H.265 encoded video stream consists of multiple frames, including I frames (Intra frames), P frames (Predicted frames) and B frames (Bi-directional frames). Different types of frames have different functions in the video compression and decompression process.
[0035] It should be noted that the executor of the media file playback method of this embodiment can be a media file playback device, which can be configured in any type of terminal equipment, such as a television, a set-top box, a mobile phone, and other equipment that has the ability to receive and process broadcast signals, and is not limited here.
[0036] As an implementation method, the playback device of the media file may be an ASTC3.0 broadcast receiving terminal.
[0037] In the embodiments of the present disclosure, a method for playing a media file is described by taking a "media file playing device" as an execution subject to execute the media file playing method, which is hereinafter referred to as "device".
[0038] It should be noted that the description order of the following embodiments is not intended to limit the priority order of the embodiments.
[0039] Figure 1 It is a flowchart of a method for playing a media file provided according to the first embodiment of the present disclosure.
[0040] like Figure 1 As shown, the method includes:
[0041] Step 101: Determine the key frame transmission delay corresponding to the media file to be played.
[0042] The media files may be files storing multimedia contents such as digital audio, video, and images.
[0043] As a possible implementation manner, the media file to be played in the embodiment of the present disclosure contains audio and video data in fMP4 format.
[0044] The key frame transmission delay can be the time delay from the video source sending the key frame to the player receiving the key frame. In video encoding, a key frame is an independently encoded frame that does not rely on other frames for decoding. It contains complete image information and is important for video decoding and playback startup.
[0045] It should be noted that key frames are important reference points for video decoding, and determining the key frame transmission delay allows the device to plan buffering parameters in advance to ensure that decoding and playback can start quickly after the key frame arrives. For example, if it is known that the key frame transmission delay is long, the playback system can pre-buffer more data before the key frame arrives to reduce the waiting time when playback starts.
[0046] As a possible implementation method, the transmission delay of key frames can be estimated based on the characteristics of the media file such as the video encoding format and frame rate. For example, for H.265 encoded media files, I frames are usually key frames, which are relatively large in size and have a longer transmission delay. For example, the size of the key frame can be determined by analyzing the received configuration file, and then the transmission delay of the key frame can be estimated in combination with the network transmission speed, which is not limited here.
[0047] Alternatively, the transmission time of the key frame in the network can be estimated by measuring parameters such as the bandwidth, delay, packet loss rate, etc. of the network. For example, a network test tool can be used to measure the network delay from the media file to the device, and then the transmission delay of the key frame can be estimated by combining the size of the key frame and the network bandwidth, which is not limited here.
[0048] In the disclosed embodiment, the key frame may be an IDR frame. In video coding, an IDR (Instantaneous Decoding Refresh) frame represents an instant decoding refresh frame. An IDR frame is a special type of I frame that can be decoded independently without relying on data from other frames, and serves to refresh the decoded image.
[0049] As a possible implementation method, the average transmission speed of the currently received media data may be determined first, and then the key frame transmission delay corresponding to the media file may be determined based on the average transmission speed and the amount of key frame data in the media file.
[0050] The average transmission speed indicates the amount of media data received by the device per second.
[0051] The calculation formula of average transmission speed is: average transmission speed = total amount of data (bytes) ÷ transmission time (seconds). Assuming that the size of the media data currently received by the receiving device is M bytes, and the time required to transmit this data is T seconds, the average transmission speed V1 = M / T (unit: bytes / second).
[0052] For example, within 600 seconds, the size of the media data received by the device is 500 megabytes (524288000 bytes), so the data size transmitted per second is approximately 870000 bytes, and the average transmission speed is V1=870000 bytes / second, which is not limited here.
[0053] The key frame data volume may represent the data size of the key frame. In the embodiment of the present disclosure, the key frame data volume may be the data size of an IDR frame in a media file.
[0054] The amount of key frame data is usually measured in bytes, which can reflect the richness of the image information contained in the frame and the bandwidth resources occupied during the transmission process. If the data of the IDR frame is large, it may mean that the image quality of the frame is high, the resolution is large, or it contains more detailed information, but it may also require more transmission time and storage space.
[0055] For example, the key frame transmission delay T1 can be determined by the following formula:
[0056] T1=I bytes *1000 / V1
[0057] Among them, T1 represents the key frame transmission delay, I bytes represents the amount of key frame data, V1 represents the average transmission speed, and 1000 is used to convert I bytes Convert to kilobytes.
[0058] It should be noted that the average transmission speed affects the transmission time of the key frame. If the network speed is slow, the key frame transmission delay may increase. The size of the key frame data will also affect the transmission delay. A larger amount of data may require a longer transmission time. Accurately determining the key frame transmission delay can help the device better perform buffer management.
[0059] Step 102: Determine the decoding time of the decoder for the frame sequence in the media file.
[0060] The frame sequence may be a frame sequence of video frames in a media file.
[0061] The decoding time may be the time required for the device to decode a frame sequence in a received media file. Different encoding formats and decoding algorithms may affect the length of the decoding time. For example, a video encoded with H.265 may require a longer decoding time than a video encoded with H.264.
[0062] Among them, the decoder can decompress and decode the compressed and encoded media files.
[0063] As a possible implementation method, the maximum buffer size required for the decoder to reorder the frame sequence in the media file can be first calculated. Optionally, the configuration file contained in the media file can be obtained by decapsulating the media file, and then the maximum buffer size corresponding to the media file can be extracted from the configuration file. Optionally, a large number of different types of frames can be decoded in advance, and the decoding time can be recorded, and then the average value can be taken to obtain the average frame decoding time. For example, multiple measurements can be performed under different devices and network environments to obtain a more accurate average frame decoding time. Finally, the decoding time can be determined based on the average frame decoding time and the maximum buffer size. For example, the average frame decoding time can be multiplied by the maximum buffer size to obtain the decoding time, which is not limited here.
[0064] Step 103: Determine the target start buffer time corresponding to the media file according to the key frame transmission delay and decoding time.
[0065] The target start buffering time may be the buffering time required for starting the media file.
[0066] It should be noted that before playing a media file, the device needs to pre-buffer data for a certain period of time to ensure that playback can start smoothly. The target start buffering time is important for achieving fast and stable media playback. By setting the target start buffering time reasonably, playback freezes or interruptions caused by insufficient data can be avoided, and the user's waiting time can be minimized.
[0067] Optionally, the sum of the key frame transmission delay and the decoding time may be used as the target start buffer time corresponding to the media file.
[0068] For example, the target start buffer time may be calculated using the following formula: T3 = T1 + T2.
[0069] Among them, T1 represents the key frame transmission delay, T2 represents the decoding time, and T3 represents the target start buffer time. For example, if the key frame transmission delay is 5 seconds and the decoding time is 2 seconds, then the target start buffer time is 7 seconds.
[0070] It should be noted that by taking the sum of the key frame transmission delay and the decoding time as the target start buffer time corresponding to the media file, the time factors in the data transmission and decoding process can be comprehensively considered. In various media playback scenarios, determining the target start buffer time can help the playback system better adapt to different network conditions and device performance.
[0071] Optionally, when the network condition is poor, the target start buffer time can be appropriately increased to ensure that there is enough data for playback. When the network condition is good and the device performance is high, the target start buffer time can be appropriately reduced to achieve a faster start speed.
[0072] Step 104: Play the media file based on the target start buffer time.
[0073] Optionally, the device may decode the media file when the buffering time corresponding to the media file is greater than or equal to the target start buffering time, and then render the decoded media file to generate data to be played, and then play the data to be played.
[0074] The data to be played is used for playing to the user.
[0075] Specifically, the device can monitor the buffering time of the media file until the buffering time reaches the target start buffering time, and the device can decode the media file, that is, convert the encoded data in the media file into a format that can be understood and processed by the playback device. For example, the decoding process will restore the compressed video data to an image sequence, and the audio encoding data will be converted into a playable sound signal.
[0076] After decoding is complete, the decoded media file can be rendered. The rendering process converts the decoded media data into visual or audible output according to the characteristics and requirements of the playback device. For video, rendering may include operations such as image scaling, color adjustment, and layout displayed on the screen; for audio, rendering may involve volume adjustment, audio effect processing, and playback through speakers.
[0077] After the media file is decoded and rendered, data to be played is generated. This data is ready to be played to the user. The device can present the data to be played in an appropriate manner to provide the user with a smooth audio-visual experience. In this way, the media file can be played as quickly as possible while ensuring that there is enough data buffer to avoid playback interruptions, thereby improving the user's viewing and listening experience.
[0078] In the disclosed embodiment, the key frame transmission delay corresponding to the media file to be played is first determined, and then the decoding time of the frame sequence in the media file by the decoder is determined. Then, based on the key frame transmission delay and the decoding time, the target start buffer time corresponding to the media file is determined, and then the media file is played based on the target start buffer time. Thus, the target start buffer time can be accurately determined through the key frame transmission delay and the decoding time corresponding to the media file, and over-buffering or under-buffering can be avoided. Over-buffering wastes system resources and user time, while under-buffering may cause playback interruptions. By accurately calculating the start buffer time, the normal playback of the media file can be more guaranteed, and the adaptability and stability of the playback can be improved.
[0079] Figure 2 It is a flowchart of a method for playing a media file provided according to the second embodiment of the present disclosure.
[0080] like Figure 2 As shown, the method includes:
[0081] Step 201: Determine the key frame transmission delay corresponding to the media file to be played.
[0082] It should be noted that the specific implementation of step 201 can refer to the above embodiment and will not be described in detail here.
[0083] Step 202: Obtain a first buffer parameter associated with the media file.
[0084] The first buffer parameter indicates the maximum buffer size required by the decoder to reorder the frame sequence in the media file.
[0085] Optionally, the device may first decapsulate the media file to obtain codec data, then parse the codec data to obtain a sequence data set, and then process the sequence data set to obtain the first buffer parameter.
[0086] The codec data is a collection of information describing the video encoding method and parameters, which contains detailed information about how the video is compressed and which codec is used for compression and decompression.
[0087] Among them, the sequence parameter set (SPS) provides the decoder with the global parameters of the video sequence (such as image size information, encoding configuration information and frame rate information). The decoder needs to parse and understand these parameters before decoding the video in order to correctly decode and play the video.
[0088] Specifically, the device can first perform a decontainer (Demux) process on the data of the media file. In this process, the codec data of the video can be obtained from the MOOV atom (Movie Atom), and then the video data of each frame can be parsed from the MDAT atom (Media Data Atom) according to the description of the MOOF atom (Movie Fragment Atom). The device can parse the codec data to obtain the sequence parameter set (SPS) data. Afterwards, the first buffer parameter (num_reorder_frames) can be parsed from the SPS data according to the H.265 specification.
[0089] It is understood that in video encoding, since the encoding order and display order may be different, the decoder needs to reorder the frames to ensure the correct playback order. The first buffer parameter indicates the maximum number of reordered frames that the decoder can temporarily store. The decoder can use these buffers to store reordered frames so that they can be decoded and played at the appropriate time.
[0090] Step 203: Determine the decoding time corresponding to the frame sequence in the media file based on the first buffer parameter and the average frame decoding time.
[0091] The average frame decoding time may be an average frame decoding time determined according to the decoding performance of the decoder, that is, an estimation of the time required for decoding a single frame.
[0092] It should be noted that the decoding time of a large number of frames by the decoder can be measured and averaged in advance to obtain an average frame decoding time. This average frame decoding time can be used as a benchmark to estimate the decoding time of the entire frame sequence.
[0093] The decoding time may be the time required to decode the entire frame sequence in the media file.
[0094] Optionally, the decoding time can be calculated by the following formula:
[0095] T2=(num_reorder_frames+1)*T 平均
[0096] Among them, T2 is the decoder, num_reorder_frames represents the first buffer parameter, T 平均 Indicates the average frame decoding time.
[0097] For example, if the average frame decoding time is 10 milliseconds, the first buffer parameter is 100 frames, and the time for decoding the frame sequence is approximately 1000 milliseconds, which is not limited here.
[0098] It should be noted that in addition to the num_reorder_frames buffers used to store reordered frames, an additional buffer is required to store the frames currently being decoded or to be played. Therefore, the first buffer parameter needs to be increased by one and then multiplied by the average frame decoding time, so as to estimate the decoding time corresponding to the frame sequence in the media file.
[0099] Step 204: Determine the target start buffer time corresponding to the media file according to the key frame transmission delay and decoding time.
[0100] Step 205: Play the media file based on the target start buffer time.
[0101] It should be noted that the specific implementation of steps 203 - 205 may refer to the above embodiment and will not be described in detail here.
[0102] In the disclosed embodiment, the key frame transmission delay corresponding to the media file to be played is first determined, and then the first buffer parameter associated with the media file is obtained, wherein the first buffer parameter represents the maximum buffer size required for the decoder to reorder the frame sequence in the media file, and then the decoding time corresponding to the frame sequence in the media file can be determined based on the first buffer parameter and the average frame decoding time, and then the target start buffer time corresponding to the media file is determined based on the key frame transmission delay and the decoding time, and finally the media file is played based on the target start buffer time. Since the first buffer parameter represents the maximum buffer size required for the decoder to reorder the frame sequence in the media file. Based on this parameter, the device can help manage memory resources more efficiently, and then allocate a buffer of appropriate size, avoid excessive memory allocation that causes performance degradation of other applications, and also ensure that there is enough buffer to handle frame reordering situations, thereby improving resource utilization.
[0103] Figure 3 It is a flowchart of a method for playing a media file provided according to the third embodiment of the present disclosure.
[0104] like Figure 3 As shown, the method includes:
[0105] Step 301: determine the audio segment and video segment associated with the target playback point.
[0106] The target playback point may be a specific time point of the playback position currently selected by the user on the entire media file playback timeline.
[0107] For example, if the user selects the 5 minute 30 second position as the playback point for a 10-minute video, the playback position corresponding to the 5 minute 30 second position is the target playback point.
[0108] The video segment and the audio segment corresponding to the target playback point may be a portion of data that can meet the continuous playback requirement near the target playback point.
[0109] It should be noted that in video encoding, GOP (Group of Pictures) is a group of continuous video frames. A Video fMP4 file can contain multiple GOPs to achieve efficient compression and playback. Figure 4 As shown, a GOP may contain one IDR frame and multiple P frames and B frames.
[0110] For example, the device may first traverse the timestamps (PTS) in the video data and audio data of the media file. Assume that in the video data, the timestamp closest to the target playback point time is found to be milliseconds, Ta milliseconds, and the corresponding video frame is in a certain GOP. In the audio data, the timestamp closest to the target playback point time is found to be Tb milliseconds.
[0111] After that, we can take the location of the timestamp Ta milliseconds found in the video as the center. Expand forward, for example, take a time range of 5 seconds, that is, start from Ta-5000 milliseconds. Expand backward and take the same time range of 5 seconds, that is, end at Ta+5000 milliseconds. The video frames in this time range constitute the video segment corresponding to the target playback point. This segment may contain parts of multiple GOPs, for example, starting from the middle of a GOP and ending at a part of another GOP.
[0112] Similarly, the location of the timestamp Tb milliseconds found in the audio can be used as the center. Similarly, the time range is extended forward and backward by 5 seconds each. The audio samples within this time range constitute the audio clip corresponding to the target playback point. Through the above steps, the audio clip and video clip associated with the target playback point can be determined, and these clips can meet the needs of continuous playback near the target playback point.
[0113] Step 302: When the first time corresponding to the audio segment is greater than the second time corresponding to the video segment, it is determined that a preset condition is met, and the video segment corresponding to the target time period is deleted.
[0114] The first time refers to the time corresponding to the first frame data in the audio segment, and the second time refers to the time corresponding to the first frame data in the video segment.
[0115] The target time period is the time interval between the first time and the second time.
[0116] As a possible implementation method, audio data and video data are transmitted in time-division multiplexed manner in the fMP4 format, which means that the audio and video data are sent alternately in a certain time sequence during the transmission process, such as audio 1-video 1-audio 2-video 2-audio 3-video 3, which is not limited here.
[0117] Since the audio and video data are not sent at the same time, synchronization problems may occur during playback. The device may need to ensure that the audio and video maintain a consistent time relationship during playback to avoid the situation where the sound and picture are out of sync.
[0118] For example, suppose that the duration of the media file is 10 minutes, and the audio and video data are transmitted in time-division multiplexing in the fMP4 format. The user chooses to start playing from the 4th minute as the target playback point. The device determines the audio segment and video segment associated with the target playback point. Assume that the time corresponding to the first frame data in the audio segment is 4 minutes and 10 seconds (first time), and the time corresponding to the first frame data in the video segment is 4 minutes and 5 seconds (second time). The first time is greater than the second time, which meets the preset conditions. The target time period is 4 minutes and 10 seconds minus 4 minutes and 5 seconds, that is, a time interval of 5 seconds. In this case, the device can delete the video segment corresponding to the 5-second time interval from the beginning of the video segment (4 minutes and 5 seconds) to the beginning of the audio segment (4 minutes and 10 seconds). In this way, it can be ensured that the audio and video can be kept synchronized during playback, avoiding the situation where the sound appears later than the picture.
[0119] For example, if you do not delete this part of the video clip during playback, the video may play for 5 seconds but there is no sound, affecting the user's viewing experience. After deleting this part of the video clip, the audio and video can start playing from the synchronized time point, ensuring the continuity and consistency of the playback.
[0120] The preset condition may be a condition for executing the subsequent steps 303 to 306. If the first time is greater than the second time, the subsequent steps may be performed.
[0121] Step 303: In response to satisfying a preset condition, determining a key frame transmission delay corresponding to the media file to be played.
[0122] It should be noted that when playing media files, the synchronization of audio and video is crucial. If the start time of audio and video is inconsistent, the sound and picture may be out of sync during playback, seriously affecting the user experience. By setting this preset condition, the situation of audio and video out of sync can be discovered and handled in time to ensure the smoothness and continuity of playback. When the audio start time is later than the video start time, the specific situation that needs to be adjusted is clarified. This enables the system to take corresponding measures at a specific time point, such as deleting the video clip in the target time period, to achieve synchronous playback of audio and video. Audio and video data are transmitted in time-division multiplexing in the fMP4 format, which means that they are not sent at the same time and time differences may occur. Taking the first time of the audio clip greater than the second time of the video clip as a preset condition can effectively handle this kind of out of sync problem caused by the transmission method.
[0123] Specifically, when the preset condition is met, the next step of determining the key frame transmission delay corresponding to the to-be-played media file can be performed. If the condition is not met, this specific operation will not be triggered, and other processing procedures may be executed or no additional operation may be performed.
[0124] It is understandable that in the case of time-division multiplexing of audio and video data, if the audio start time is less than or equal to the video start time, it means that the audio data arrives relatively early in the transmission process or arrives at the same time as the video data. In this case, since the time difference between audio and video is small, the device is easier to handle synchronization problems and is not prone to serious asynchronism.
[0125] Step 304: Determine the decoding time of the decoder for the frame sequence in the media file.
[0126] Step 305: Determine the target start buffer time corresponding to the media file according to the key frame transmission delay and decoding time.
[0127] Step 306: Play the media file based on the target start buffer time.
[0128] It should be noted that the specific implementation of steps 303 - 306 may refer to the above embodiment and will not be described in detail here.
[0129] In the disclosed embodiment, the audio and video segments associated with the target playback point are first determined, and then, when the first time corresponding to the audio segment is greater than the second time corresponding to the video segment, it is determined that the preset condition is met, and the video segment corresponding to the target time period is deleted. Then, in response to the preset condition being met, the key frame transmission delay corresponding to the media file to be played is determined, and then the decoding time of the frame sequence in the media file by the decoder is determined, and then the target start buffer time corresponding to the media file is determined according to the key frame transmission delay and the decoding time, and then the media file is played based on the target start buffer time. Thus, by determining the audio and video segments associated with the target playback point, the media content near the position where the user expects to play can be accurately located. When the first time corresponding to the audio segment is greater than the second time corresponding to the video segment, the video segment corresponding to the target time period is deleted, which can effectively solve the problem of audio and video being out of sync, which ensures that the sound and the picture can maintain a consistent time relationship during the playback process, and improves the user's viewing experience. In response to determining the key frame transmission delay after the preset condition is met, the playback system can know in advance the time required for the key frame to reach the playback end. This helps the playback system to plan buffering strategies appropriately and prepare enough data before the key frame arrives to reduce the waiting time when playback starts. Playing based on the target start buffering time ensures that the playback system has enough time to receive and process media data. Even in the case of unstable network conditions or data transmission delays, the playback system can maintain playback continuity by buffering data and improve playback stability.
[0130] Figure 5 is a flowchart of the start-up process provided by the embodiment of the present disclosure. Figure 5 As shown, first, the fmp4 file can be parsed through the ROUTER / MMTP protocol, and then the average transmission speed of Video fMP4 is calculated, and then the Codec data is obtained from Video fMP4 MOOV, and the video frame data is obtained from MDAT. After that, the transmission delay of the video IDR frame can be estimated, the maximum number of decoding buffers can be calculated from the SPS of the Video fMP4 data, the frame sequence decoding delay can be estimated, and then the minimum Video buffer level can be calculated based on the IDR transmission delay and the frame sequence decoding delay. If the Video buffer level is greater than the calculated minimum Video buffer level, the audio and video data will be sent to the decoder for decoding, Render, and start playing, otherwise continue to wait.
[0131] In order to better implement the media file playing method of the present disclosure, the present disclosure also provides a media file playing device based on the media file playing method. The meanings of the terms are the same as those in the media file playing method, and the specific implementation details can refer to the description in the method embodiment.
[0132] See also Figure 6 , Figure 6 is a schematic diagram of the structure of a media file playback device provided in an embodiment of the present disclosure, wherein the media file playback device 600 comprises:
[0133] A first determination module 610 is used to determine a key frame transmission delay corresponding to a media file to be played;
[0134] A second determination module 620, configured to determine a decoding time of a decoder for a frame sequence in the media file;
[0135] A third determination module 630 is used to determine a target start buffer time corresponding to the media file according to the key frame transmission delay and the decoding time;
[0136] The playing module 640 is configured to play the media file based on the target start buffer time.
[0137] Optionally, the first determining module is specifically configured to:
[0138] Determine the average transmission speed of the currently received media data;
[0139] Based on the average transmission speed and the amount of key frame data in the media file, a key frame transmission delay corresponding to the media file is determined.
[0140] Optionally, the second determination module includes:
[0141] an acquiring unit, configured to acquire a first buffer parameter associated with the media file,
[0142] The first buffer parameter represents a maximum buffer size required for the decoder to reorder the frame sequence in the media file;
[0143] A first determining unit is configured to determine a decoding time corresponding to a frame sequence in the media file based on the first buffer parameter and an average frame decoding time.
[0144] Optionally, obtain the unit, specifically used to:
[0145] Decapsulating the media file to obtain codec data;
[0146] Parsing the codec data to obtain a sequence data set;
[0147] The sequence data set is processed to obtain the first buffer parameter.
[0148] Optionally, the third determination module is specifically used to:
[0149] The sum of the key frame transmission delay and the decoding time is used as the target start buffer time corresponding to the media file.
[0150] Optionally, the first determining module includes:
[0151] The second determining unit is configured to determine a key frame transmission delay corresponding to the media file to be played in response to satisfying a preset condition.
[0152] Optionally, the second determining unit is further configured to:
[0153] Determining the audio segment and the video segment associated with the target playback point;
[0154] In the case where the first time corresponding to the audio segment is greater than the second time corresponding to the video segment, determining that the preset condition is met, and deleting the video segment corresponding to the target time period,
[0155] The first time refers to the time corresponding to the first frame data in the audio segment, and the second time refers to the time corresponding to the first frame data in the video segment;
[0156] The target time period is the time interval between the first time and the second time.
[0157] Optional, playback module, specifically used for:
[0158] When the buffering time corresponding to the media file is greater than or equal to the target start buffering time, decoding the media file;
[0159] Rendering the decoded media file to generate data to be played;
[0160] Play the data to be played.
[0161] In the disclosed embodiment, the key frame transmission delay corresponding to the media file to be played is first determined, and then the decoding time of the frame sequence in the media file by the decoder is determined. Then, based on the key frame transmission delay and the decoding time, the target start buffer time corresponding to the media file is determined, and then the media file is played based on the target start buffer time. Thus, the target start buffer time can be accurately determined through the key frame transmission delay and the decoding time corresponding to the media file, and over-buffering or under-buffering can be avoided. Over-buffering wastes system resources and user time, while under-buffering may cause playback interruptions. By accurately calculating the start buffer time, the normal playback of the media file can be more guaranteed, and the adaptability and stability of the playback can be improved.
[0162] In addition, the present disclosure also provides an electronic device, such as Figure 7As shown, it shows a schematic diagram of the structure of the electronic device involved in the present disclosure, specifically:
[0163] The electronic device may include components such as a processor 701 with one or more processing cores, a memory 702 with one or more computer-readable storage media, a power supply 703, and an input unit 704. Those skilled in the art will appreciate that Figure 7 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.
[0164] The processor 701 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 702, and calling data stored in the memory 702, it performs various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. Optionally, the processor 701 may include one or more processing cores; preferably, the processor 701 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 701.
[0165] The memory 702 can be used to store software programs and modules. The processor 701 executes various functional applications and data processing by running the software programs and modules stored in the memory 702. The memory 702 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 702 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 702 may also include a memory controller to provide the processor 701 with access to the memory 702.
[0166] The electronic device also includes a power supply 703 for supplying power to each component. Preferably, the power supply 703 can be logically connected to the processor 701 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 703 can also include any components such as one or more DC or AC power supplies, recharging systems, power supply device debugging circuits, power converters or inverters, and power status indicators.
[0167] The electronic device may further include an input unit 704, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.
[0168] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 701 in the electronic device will load the executable files corresponding to the processes of one or more application programs into the memory 702 according to the following instructions, and the processor 701 will run the application programs stored in the memory 702, thereby implementing the steps in any one of the media file playback methods provided in the embodiments of the present disclosure.
[0169] In the disclosed embodiment, the key frame transmission delay corresponding to the media file to be played is first determined, and then the decoding time of the frame sequence in the media file by the decoder is determined. Then, based on the key frame transmission delay and the decoding time, the target start buffer time corresponding to the media file is determined, and then the media file is played based on the target start buffer time. Thus, the target start buffer time can be accurately determined through the key frame transmission delay and the decoding time corresponding to the media file, and over-buffering or under-buffering can be avoided. Over-buffering wastes system resources and user time, while under-buffering may cause playback interruptions. By accurately calculating the start buffer time, the normal playback of the media file can be more guaranteed, and the adaptability and stability of the playback can be improved.
[0170] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.
[0171] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.
[0172] To this end, the present disclosure provides a computer-readable storage medium, on which a computer program is stored. The computer program can be loaded by a processor to execute the steps in any media file playback method provided by the present disclosure.
[0173] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.
[0174] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.
[0175] Since the instructions stored in the computer-readable storage medium can execute the steps in any media file playback method provided in the present disclosure, the beneficial effects that can be achieved by any media file playback method provided in the present disclosure can be achieved. Please refer to the previous embodiments for details, which will not be repeated here.
[0176] The above is a detailed introduction to a media file playback method, device, electronic device and computer-readable storage medium provided by the present disclosure. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as limiting the present invention.
Claims
1. A method for playing a media file, characterized in that: include: Determine the key frame transmission delay corresponding to the media file to be played; Determine a decoding time of a decoder for a frame sequence in the media file; Determining a target start buffer time corresponding to the media file according to the key frame transmission delay and the decoding time; The media file is played based on the target start buffer time.
2. The method according to claim 1, characterized in that The determining of the key frame transmission delay corresponding to the media file to be played includes: Determine the average transmission speed of the currently received media data; Based on the average transmission speed and the amount of key frame data in the media file, a key frame transmission delay corresponding to the media file is determined.
3. The method according to claim 1, characterized in that The determining the decoding time of the frame sequence in the media file by the decoder includes: obtaining a first buffer parameter associated with the media file, The first buffer parameter represents a maximum buffer size required for the decoder to reorder the frame sequence in the media file; Determine a decoding time corresponding to a frame sequence in the media file based on the first buffer parameter and the average frame decoding time.
4. The method according to claim 3, characterized in that The obtaining of a first buffer parameter associated with the media file includes: Decapsulating the media file to obtain codec data; Parsing the codec data to obtain a sequence data set; The sequence data set is processed to obtain the first buffer parameter.
5. The method according to claim 1, characterized in that The determining, according to the key frame transmission delay and the decoding time, a target start buffer time corresponding to the media file includes: The sum of the key frame transmission delay and the decoding time is used as the target start buffer time corresponding to the media file.
6. The method according to claim 1, characterized in that The determining of the key frame transmission delay corresponding to the media file to be played includes: In response to satisfying a preset condition, a key frame transmission delay corresponding to the media file to be played is determined.
7. The method according to claim 6, characterized in that Before determining the key frame transmission delay corresponding to the media file to be played in response to satisfying the preset condition, the method further includes: Determining the audio segment and the video segment associated with the target playback point; In the case where the first time corresponding to the audio segment is greater than the second time corresponding to the video segment, determining that the preset condition is met, and deleting the video segment corresponding to the target time period, The first time refers to the time corresponding to the first frame data in the audio segment, and the second time refers to the time corresponding to the first frame data in the video segment; The target time period is the time interval between the first time and the second time.
8. The method according to claim 1, characterized in that The controlling the playing of the media file based on the target start buffering time includes: When the buffering time corresponding to the media file is greater than or equal to the target start buffering time, decoding the media file; Rendering the decoded media file to generate data to be played; Play the data to be played.
9. A media file playback device, characterized in that: include: A first determination module, used to determine a key frame transmission delay corresponding to a media file to be played; A second determination module, used to determine a decoding time of a decoder for a frame sequence in the media file; A third determination module, configured to determine a target start buffer time corresponding to the media file according to the key frame transmission delay and the decoding time; The playing module is used to play the media file based on the target start buffering time.
10. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in the method according to any one of claims 1 to 8 when executing the computer program.
11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the method according to any one of claims 1 to 8 are implemented.