Multimedia file playing method, readable storage medium and equipment

Abstract

The invention relates to the technical field of multimedia, and discloses a multimedia file playing method, a readable storage medium and equipment. According to the method, when a device plays a multimedia file in a loop mode, after the last frame of data of the multimedia file is read, the first frame of data of the multimedia file continues to be read. Then, before the first output period corresponding to the last frame of data of the multimedia file ends, the equipment processes the first frame of data of the multimedia file to obtain first output data corresponding to the first frame of data; therefore, the device can output the first output data corresponding to the first frame data of the multimedia file in the output period after the first output period. The last frame data and the first frame data of the multimedia file are continuously read, and the output data corresponding to the last frame data and the output data corresponding to the first frame data are continuously output. Therefore, seamless loop playing of the multimedia file can be realized, and the user experience can be improved.

Description

Technical Field

[0001] The present application relates to the field of multimedia technology, and in particular to a multimedia file playing method, a readable storage medium and a device. Background Art

[0002] When a device plays a multimedia file, a frame of data generally needs to be read, parsed, decoded, rendered, and processed in sequence before it can be output. For example, for a video file, the device first reads a frame of data; then parses the frame of data into video data and audio data; secondly, decodes and renders the video data and audio data to obtain image output data and sound output data that can be output to the output device; finally, after synchronizing the image output data and sound output data, the image output data is output to the display device, and the sound output data is output to the playback device. In other words, for any frame of data, after the device reads the frame of data, it needs to go through the aforementioned processing in sequence before it can be output to the corresponding output device, which takes time.

[0003] In general, when a device plays a multimedia file in a loop, such as a video file in a loop, it usually plays the video file again after the last frame of the video file is displayed. Since the first frame of the video file must be processed as described above before it can be output to the device's display screen, the device's display screen has no new images to display from the time the last frame is displayed to the time the first frame is displayed on the display screen. As a result, when the device plays the video file in a loop, the images displayed on the display screen will be discontinuous, affecting the user experience. Summary of the invention

[0004] In view of this, the present application provides a multimedia file playback method, a readable storage medium and a device, which can achieve seamless loop playback of multimedia files.

[0005] A first aspect provides a multimedia file playing method, the method comprising: when a device plays a multimedia file in a loop, the device reads the Kth frame data of the multimedia file in the Kth output cycle, processes the read Kth frame data in the Kth output cycle to the K+Q-1th output cycle, and outputs output data corresponding to the Kth frame data in the K+Qth output cycle, wherein Q is an integer greater than 0, K is an integer greater than 0 and less than N, wherein N is the number of frames of data included in the multimedia file; the device reads the Nth frame data of the multimedia file in the Nth output cycle, and outputs output data corresponding to the NQth frame data of the multimedia file; the device reads the 1st frame data of the multimedia file in the N+1th output cycle, and outputs output data corresponding to the N-Q+1th frame data of the multimedia file; the device reads the Qth frame data of the multimedia file in the N+Qth output cycle, and outputs output data corresponding to the Nth frame data of the multimedia file; the device reads the Qth frame data of the multimedia file in the N+Qth output cycle, and outputs output data corresponding to the Nth frame data of the multimedia file; the device reads the Q+1th frame data of the multimedia file in the N+Q+1th output cycle, and outputs output data corresponding to the 1st frame data of the multimedia file.

[0006] In this method, when the device plays a multimedia file in a loop, it reads the last frame data and the first frame data of the multimedia file in adjacent cycles (the Nth output cycle and the N+1th output cycle), and outputs the output data corresponding to the last frame data of the multimedia file and the output data corresponding to the first frame data in adjacent cycles (the N+Qth output cycle and the N+Q+1th output cycle). Based on the above method, when the device plays a multimedia file in a loop, the output data corresponding to the last frame data and the output data corresponding to the first frame data are output continuously in two adjacent output cycles. In this way, freezes, white screens or black screens are avoided when playing multimedia files in a loop, and seamless loop playback of multimedia files is achieved.

[0007] In a possible implementation of the first aspect above, the multimedia file is a video file, and the output data corresponding to the first frame data includes first image output data and first audio output data; and the device reads the Q+1th frame data of the multimedia file in the N+Q+1th output cycle, and outputs the output data corresponding to the first frame data of the multimedia file, including: after the device outputs the output data corresponding to the Nth frame data of the multimedia file, the synchronization reference time is updated, and the synchronization reference time is used to indicate the starting playback time of the multimedia file; the device synchronizes the first image data and the first audio data based on the synchronization reference time, and outputs the first image output data and the first audio output data in the N+Q+1th output cycle.

[0008] That is, after the device outputs the output data corresponding to the last frame data (Nth frame data) of the multimedia file, it can update the synchronization reference time. In this way, the device can synchronize the first image output data and the first audio output data corresponding to the first frame data based on the updated synchronization reference time to avoid the first image output data and the first audio output data being out of sync.

[0009] In a possible implementation of the first aspect above, the above-mentioned device reads the Nth frame data of the multimedia file in the Nth output cycle, including: after reading the Nth frame data of the multimedia file, the device does not report a reading end message of the multimedia file to the application that plays the multimedia file.

[0010] It should be understood that the multimedia file reading end message is used to instruct the device to stop processing subsequent data after processing the last frame of multimedia file data and end the multimedia file playback. After reading the Nth frame of multimedia file data, the device does not report the multimedia file reading end message to the application that plays the multimedia file, so that the device can continue to process the subsequent data (the first frame of multimedia file data) to achieve seamless loop playback of multimedia files.

[0011] In a possible implementation of the first aspect above, the method above also includes: the device outputs the output data corresponding to the second frame data of the multimedia file to the output data corresponding to the Nth frame data of the multimedia file in sequence from the N+Q+2th output cycle to the N+Q+Nth output cycle.

[0012] In a possible implementation of the first aspect above, the method further includes: the device processes the first frame data of the multimedia file from the N+1th output cycle to the N+Qth output cycle to obtain output data corresponding to the first frame data.

[0013] In a possible implementation of the first aspect above, the above-mentioned device processes the first frame data of the multimedia file from the N+1th output cycle to the N+Qth output cycle, at least including: the device reads the first frame data from the N+1th output cycle to the N+Qth output cycle, parses, decodes and renders the first frame data.

[0014] In a possible implementation of the first aspect above, the device plays a multimedia file through a first application; and the device reads the Nth frame data of the multimedia file in the Nth output cycle, including: the first application reads the Nth frame data of the multimedia file in the Nth output cycle.

[0015] It should be understood that in some implementations, the output data corresponding to each frame of data of the multimedia file may also be output by the first application.

[0016] In a possible implementation of the first aspect, the device reads the first frame data of the multimedia file in the N+1th output cycle, including: the first application reads the first frame data of the multimedia file in the N+1th output cycle.

[0017] That is to say, the instructions of the multimedia file playing method provided in the present application can be embedded in the first application, so that the first application can play the multimedia file through the method provided in the present application.

[0018] In a possible implementation of the first aspect above, the operating system of the above-mentioned device includes a framework layer, the framework layer includes a first module; and the device reads the Nth frame data of the multimedia file in the Nth output cycle, including: the first module reads the Nth frame data of the multimedia file in the Nth output cycle.

[0019] That is to say, the instructions of the multimedia file playing method provided in the present application can be embedded in the first module (such as the reading and parsing module mentioned below), so that the application program that plays multimedia files on the device can play multimedia files through the method provided in the present application through the first module.

[0020] In a possible implementation of the first aspect, the device reads the first frame data of the multimedia file in the N+1th output cycle, including: the first module reads the first frame data of the multimedia file in the N+1th output cycle.

[0021] A second aspect provides a readable storage medium, which includes instructions. When the instructions are executed by a device, the device implements the multimedia file playback method provided by the first aspect and any possible implementation of the first aspect.

[0022] The third aspect provides a device, which includes: a memory for storing instructions; a processor for executing instructions to enable the device to implement the multimedia file playback method provided by the first aspect and any possible implementation of the first aspect.

[0023] A fourth aspect provides a program product, which, when running on a device, enables the device to implement the multimedia file playback method provided by the first aspect and any possible implementation of the first aspect.

[0024] A fifth aspect provides a chip or a chip system, including a processing module and a communication interface, wherein the processing module is used to execute the multimedia file playback method provided by the first aspect and any possible implementation of the first aspect.

[0025] It should be understood that the beneficial effects of the second to fifth aspects mentioned above can be referred to the description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 According to some embodiments of the present application, a schematic diagram of a scenario in which a smart TV 10 plays a video in a loop is shown;

[0027] Figure 2A According to some embodiments of the present application, a timing diagram of a smart TV 10 playing a video in a loop is shown;

[0028] Figure 2B According to some embodiments of the present application, a schematic diagram of a display interface of a smart TV 10 when playing a video in a loop is shown;

[0029] Figure 3A According to some embodiments of the present application, another timing diagram of a smart TV 10 playing a video in a loop is shown;

[0030] Figure 3B According to some embodiments of the present application, another timing diagram of a smart TV 10 playing a video in a loop is shown;

[0031] Figure 4 According to some embodiments of the present application, a schematic diagram of a software architecture of a smart TV 10 is shown;

[0032] Figure 5 According to some embodiments of the present application, another schematic diagram of the software architecture of a smart TV 10 is shown;

[0033] Figure 6 According to some embodiments of the present application, a schematic diagram of a process of playing a video on a smart TV 10 is shown;

[0034] Figure 7 According to some embodiments of the present application, a schematic diagram of a process of looping video playback by a smart TV 10 is shown;

[0035] Figure 8 According to some embodiments of the present application, a flowchart of a multimedia file playing method is shown;

[0036] Fig.9A According to some embodiments of the present application, a schematic diagram of a user triggering a multimedia instruction to play in a loop on a smart TV 10 is shown;

[0037] Fig. 9B According to some embodiments of the present application, a schematic diagram of a user triggering a multimedia instruction to play in a loop on a mobile phone 20 is shown;

[0038] Fig.10 According to some embodiments of the present application, a schematic structural diagram of a smart TV 10 is shown. DETAILED DESCRIPTION

[0039] Illustrative embodiments of the present application include, but are not limited to, multimedia file playback methods, readable storage media, and devices.

[0040] It should be understood that the multimedia file playing method provided in the embodiment of the present application can be applied to any device that can play multimedia files, including but not limited to desktop computers, laptop computers, mobile phones, tablet computers, wearable devices, mobile email devices, portable game consoles, portable music players, AR devices, VR devices, head-mounted display devices, reader devices, smart home devices (such as smart TVs, large screens, projectors, etc.), etc., and the embodiment of the present application does not limit it. For ease of description, the following is an introduction taking smart TV 10 as an example.

[0041] It should be understood that multimedia files may include but are not limited to video files, audio files, etc., which are not limited here. For ease of description, the following is an introduction using a video file as an example.

[0042] It should be understood that the multimedia files played by the device in the embodiment of the present application may be multimedia files stored locally on the device, or may be multimedia files acquired by the device from other devices in real time, which is not limited here.

[0043] The technical solution of the present application is described below in conjunction with the accompanying drawings.

[0044] Figure 1 According to some embodiments of the present application, a schematic diagram of a scenario in which a smart TV 10 plays a video in a loop is shown.

[0045] like Figure 1 As shown, the smart TV 10 can switch to the "picture frame mode" in response to the user operation (or the smart TV automatically switches to the "picture frame mode" when it determines that the preset conditions are met), and the smart TV 10 plays the campfire video in full screen and loops. In some implementations, the "picture frame mode" refers to the function of the smart TV 10 to play a preset video file or a video file selected by the user in full screen and loops through a video playback application.

[0046] It should be understood that Figure 1 The scenario shown is only an example. The technical solution of the present application is applicable to any scenario in which multimedia files are played in a loop, such as a scenario in which video applications, audio playback applications, social applications or other applications play multimedia files in a loop, etc., and is not limited here.

[0047] When the smart TV 10 plays a video in a loop, it usually plays the video file again after the last frame of data of the video file is output (for example, after the image corresponding to the last frame of data of the video file (hereinafter referred to as the last frame of image) is displayed). The first frame of data of the video file needs to be read, parsed, decoded, rendered, etc. before the image corresponding to the first frame of data (hereinafter referred to as the first frame of image) can be obtained and the first frame of image can be output to the display screen of the smart TV 10. Therefore, from the time when the last frame of image is output to the time when the first frame of image is displayed on the display screen, there is no new image to be displayed on the display screen of the device, resulting in the discontinuity of the picture displayed by the smart TV 10.

[0048] For example, suppose Figure 1 The campfire video shown includes N frames of data, refer to Figure 2A , Smart TV 10 in T 0 Time to T 1 During the time period, the Nth frame data is sequentially read, parsed, decoded and rendered to obtain the Nth frame output data corresponding to the Nth frame data (for example, the Nth frame data corresponds to the Nth frame image and the Nth segment sound). 1 Time to T 2 During the time period, the Nth frame of image is displayed and the Nth sound is played (ie, the Nth frame of output data is output).

[0049] Smart TV 10 in T 2 When the Nth frame image is displayed, the bonfire video is triggered to be played again. 3 Complete the initialization of the video playback application at T 3 Time to T 4 During the time period, the first frame data of the campfire video is read, parsed, decoded and rendered in sequence to obtain the first frame output data corresponding to the first frame data (for example, the first frame image and the first audio segment corresponding to the first frame data). 4 Time to T 5 During this time, the first frame of image is displayed and the first sound is played (i.e., the first frame of output data is output).

[0050] That is to say, from T 2 Time to T 4 During this time, the smart TV 10 has no images to display and no sounds to play, which may cause the images displayed or the sounds played by the smart TV 10 to be inconsistent, affecting the user experience.

[0051] For example, refer to Figure 2B If the Smart TV 10 is in T 2 Time to T 4 The Nth frame image continues to be displayed during the time T 4 Time to T5 The first frame of image is displayed during the time period T 2 Time to T 4 The image displayed by the smart TV 10 during the time period is the same, causing the smart TV 10 to stagnate and freeze.

[0052] For another example, if the smart TV 10 is in T 2 Time to T 4 Display background (e.g. white or black) during the time period, and 4 Time to T 5 The first frame of image is displayed during the time period. This will cause the smart TV 10 to 2 Time to T 4 A blank or black screen is displayed for a period of time.

[0053] In order to avoid the problem of discontinuity in the process of looping multimedia files, an embodiment of the present application provides a method for playing multimedia files. In this method, when the device plays a multimedia file in a loop, after reading the last frame of data of the multimedia file, it continues to read the first frame of data of the multimedia file. Then, before the output cycle corresponding to the last frame of data of the multimedia file (hereinafter referred to as the first output cycle) ends, the device processes the first frame of data of the multimedia file to obtain the first output data corresponding to the first frame of data. In this way, the device can output the first output data corresponding to the first frame of data of the multimedia file in the output cycle after the first output cycle (hereinafter referred to as the second output cycle).

[0054] Since the device does not need to wait until the last frame of the multimedia file is output before starting to read the first frame of the multimedia file, the last frame of the multimedia file is read continuously with the first frame of the multimedia file, and the output data corresponding to the last frame of the multimedia file is output continuously with the output data corresponding to the first frame of the multimedia file. In this way, discontinuity can be avoided when the multimedia file is played in a loop, and seamless loop playback of the multimedia file can be achieved, which is conducive to improving the user experience.

[0055] It should be understood that the device processes multimedia files in multiple steps, each step can take up one output cycle, or multiple steps can take up one output cycle. Assume that the device processes one frame of data in Q (Q ≥ 1) output cycles, and the device reads a frame of data in the Kth output cycle, and the device outputs the output data corresponding to the frame of data in the K+Qth output cycle. For example, assume that the device processes one frame of multimedia file data in four steps: reading, parsing, decoding, and rendering, and each step takes up one output cycle, then Q = 4.

[0056] For example, Figure 3AAccording to some embodiments of the present application, another timing diagram of a smart TV 10 playing a video in a loop is shown.

[0057] Assumptions Figure 1 In the illustrated scenario, the campfire video played by the smart TV 10 includes N frames of data. The processing of each frame of data by the smart TV 10 includes four steps: reading, parsing, decoding, and rendering. Each step takes one output cycle, that is, Q=4. Figure 3A As shown:

[0058] In t 0 Time to t 1 During the time period (the Pth output cycle), the smart TV 10 reads the Nth frame data, parses the N-1th frame data, decodes the N-2th frame data, renders the N-3th frame data, and outputs the output data corresponding to the N-4th frame data.

[0059] In t 1 At this moment, the smart TV 10 detects that the last frame of the campfire video has been read, triggering the replay of the video. In some embodiments, the smart TV 10 can detect that the last frame of the campfire video has been read when the end mark of the campfire (such as the end of segment (EOS) mark) is read, and continue to read the first frame of the campfire video. In other embodiments, the smart TV 10 can also detect that the last frame of the campfire video has been read by other means, which are not limited here.

[0060] In t 1 Time to t 2 During the time period (the P+1th output cycle), the smart TV 10 continues to read the 1st frame data, parse the Nth frame data, decode the N-1th frame data, render the N-2th frame data, and output the output data corresponding to the N-3th frame data.

[0061] In t 2 Time to t 3 During the time period (the P+2th output cycle), the smart TV 10 reads the second frame data, parses the first frame data, decodes the Nth frame data, renders the N-1th frame data, and outputs the output data corresponding to the N-2th frame data.

[0062] In t 3 Time to t 4 During the time period (the P+3th output cycle), the smart TV 10 reads the 3rd frame data, parses the 2nd frame data, decodes the 1st frame data, renders the Nth frame data, and outputs the output data corresponding to the N-1th frame data.

[0063] In t 4 Time to t 5During the time period (the P+Qth output cycle (ie, the first output cycle)), the smart TV 10 reads the 4th frame data, parses the 3rd frame data, decodes the 2nd frame data, renders the 1st frame data, and outputs the output data corresponding to the Nth frame data.

[0064] In t 5 At this moment, the smart TV 10 detects that the last frame of video data has been output, and updates the synchronization reference time of the image output data and the audio data, so as to synchronize the output images with the images. It should be understood that the synchronization reference time refers to the start time of the video file playback. If the video file is played from the 0th second, the synchronization reference time is 0 seconds; if the video file is played from the 5th second, the synchronization reference time is 5 seconds. The specific method for the smart TV 10 to synchronize the image output data and the audio output data will be introduced below and is not limited here.

[0065] In t 5 Time to t 6 During the time period (the P+Q+1th output cycle (i.e., the second output cycle)), the smart TV 10 reads the 5th frame data, parses the 4th frame data, decodes the 3rd frame data, renders the 2nd frame data, and outputs the output data corresponding to the 1st frame data.

[0066] It should be understood that when the smart TV 10 outputs the output data corresponding to a certain frame of data, the image output data and the sound output data corresponding to the frame of data may be synchronized before outputting.

[0067] Based on the above method, compared with Figure 2A In the process shown, after the smart TV 10 has read the Nth frame data of the bonfire video, it continues to read the 1st frame data of the bonfire video (that is, the Nth frame data and the 1st frame number of the bonfire video are read in two adjacent input cycles), instead of waiting until the Nth frame data is output and then reading the 1st frame data of the bonfire video. The smart TV 10 can obtain the output data corresponding to the 1st frame data of the bonfire video before the end of the output cycle corresponding to the Nth frame data, so that the smart TV 10 can output the output data corresponding to the Nth frame data and the output number corresponding to the 1st frame data in two adjacent output cycles. In this way, the smart TV 10 can play the video in a seamless loop, and there will be no freeze in the display screen or black screen / white screen during the period, which is conducive to improving the user experience.

[0068] It should be understood that when outputting a multimedia file, the device may output the multimedia file periodically, for example, outputting one frame of data of the multimedia file in one output period.

[0069] Exemplarily, for a video file, one frame of data may include one frame of image data and the audio data corresponding to that frame of image. The output period may be the reciprocal of the frame rate of the video file, or the reciprocal of the frame rate at which the device composes images, or the reciprocal of the refresh rate at which the device displays images. For example, if the frame rate of the video file is 60 Hz (it can also be other data, which is not limited here), or the frame rate at which the device composes images is 60 Hz (it can also be other data, which is not limited here), or the refresh rate at which the device displays images is 60 Hz (it can also be other data, which is not limited here), then the output period may be 1 / 60 = 16.67 ms.

[0070] Exemplarily, for an audio file, one frame of data may include at least one sample data. The output period may be K times (K≥1) the reciprocal of the sampling rate of the audio file. For example, if the sampling rate of the audio file is 44000 Hz (it can also be other data, which is not limited here), then the output period may be K×(1 / 44000) = K×0.0227 ms.

[0071] It should be understood that after the device reads the Mth (1≤M<N, N is the number of frames included in the multimedia file) frame of data of the multimedia file, if the device does not detect an instruction to pause playback or stop playback, it can continue to read the (M + 1)th frame of data of the media file. Then, the device can sequentially parse, decode, and render the (M + 1)th frame of data, so that the device can continuously output the output data corresponding to the Mth frame of data and the (M + 1)th frame of output data of the multimedia file.

[0072] It should be understood that when the device outputs the output data of the multimedia file, it can output the multimedia file through the output device of the device itself (such as the display screen, projection device, speaker, etc. of the device itself), or through other output devices connected to the device, which is not limited here.

[0073] It should be understood that dividing the processing of the multimedia file by the device into four steps: reading, parsing, decoding, and rendering in the embodiments of the present application is only an example. In other embodiments, the processing of the multimedia file by the device can also be divided into more or fewer sequentially executed steps, or some steps can be combined, which is not limited here.

[0074] It should be understood that in the above embodiments, each of the four steps of reading, parsing, decoding, and rendering of one frame of data of the multimedia file occupying one output period is only an example. In other embodiments, each of the four steps may only occupy a part of the duration of one output period; if the total processing duration of two or more steps is less than one output period, two or more steps can also be combined and executed in one output period, which is not limited here.

[0075] For example, Figure 3B According to some embodiments of the present application, another timing diagram of a smart TV 10 playing a video in a loop is shown.

[0076] refer to Figure 3B , compared to Figure 3A In the case shown, the smart TV 10 combines the reading and parsing of a frame of data into one step, that is, the processing of a frame of video file by the smart TV 10 is divided into three steps: reading and parsing, decoding and rendering, and each step takes one output cycle, that is, Q=3. Figure 3B As shown:

[0077] In t 7 Time to t 8 During the time period (the Pth output cycle), the smart TV 10 reads and parses the Nth frame data, decodes the N-1th frame data, renders the N-2th frame data, and outputs the output data corresponding to the N-3th frame data.

[0078] In t 8 At this moment, the smart TV 10 detects that the last frame of the video has been read, triggering the replay of the video.

[0079] In t 8 Time to t 9 During the time period (the P+1th output cycle), the smart TV 10 reads and parses the 1st frame data, decodes the Nth frame data, renders the N-1th frame data, and outputs the output data corresponding to the N-2th frame data.

[0080] In t 9 Time to t 10 During the time period (the P+2th output cycle), the smart TV 10 reads and parses the second frame data, decodes the first frame data, renders the Nth frame data, and outputs the output data corresponding to the N-1th frame data.

[0081] In t 10 Time to t 11 During the time period (the P+Qth output cycle (ie, the first output cycle)), the smart TV 10 reads and parses the third frame data, decodes the second frame data, renders the first frame data, and outputs the output data corresponding to the Nth frame data.

[0082] In t 11 At this moment, the smart TV 10 detects that the last frame of video data has been output, and updates the synchronization reference time of the image output data and the audio data to synchronize the output image and sound.

[0083] In t 11 Time to t 12During the time period (the P+Q+1th output cycle (ie, the second output cycle)), the smart TV 10 reads and parses the 4th frame data, decodes the 3rd frame data, renders the 2nd frame data, and outputs the output data corresponding to the 1st frame data.

[0084] It should be understood that in some embodiments, in the processing of multimedia files:

[0085] Reading refers to reading the data of the multimedia file frame by frame from the memory of the smart TV 10 or from other devices. For example, for a multimedia file stored locally on the smart TV 10, the smart TV 10 can read the data of the multimedia file frame by frame from the location where the multimedia file is stored. For another example, for a multimedia file obtained in real time from other devices, the smart TV 10 can read the data of the multimedia file frame by frame from the cache corresponding to the multimedia file.

[0086] Parsing refers to parsing the data of a multimedia file into video elementary streams (ES) and / or audio ES. If the multimedia file includes video data but not audio data, the video ES can be obtained after parsing the multimedia file; if the multimedia file includes audio data but not video data, the single-frequency ES can be obtained after parsing the multimedia file; if the multimedia file includes video data and audio data, the video ES and audio ES can be obtained after parsing the multimedia file.

[0087] Decoding refers to decoding the multimedia video ES and / or audio ES into coded data that can be rendered. For example, for a video ES, it can be decoded to obtain YUV-encoded (a color encoding method, or other types of encoding methods, not limited here) data. For another example, for an audio ES, it can be decoded into linear pulse code modulation (LPCM) (a non-compressed audio digitization method, or other methods, such as pulse code modulation, etc., not limited here) data.

[0088] Rendering refers to processing the decoded YUV encoded data and LPCM data to obtain output data that can be output to an output device.

[0089] Combine the following Figure 1 and Figure 3A , Figure 3B The situation shown introduces the technical solution of the present application.

[0090] For ease of understanding, the software architecture of the smart TV 10 is first introduced.

[0091] Figure 4According to some embodiments of the present application, a schematic diagram of a software architecture of a smart TV 10 is shown.

[0092] like Figure 4 As shown, the software architecture of the smart TV 10 includes an application layer 01, a framework layer 02, a hardware abstraction layer (HAL) 03, a kernel layer 04, and the like.

[0093] The application layer 01 includes application programs in the smart TV 10 .

[0094] For example, the application layer 01 may include a first application 011. The first application 011 may be an application program capable of playing multimedia files. The first application 011 includes a reading and parsing module 012, a user interface 013 and a playing control 014.

[0095] The reading and parsing module 012 is used to read the data of the multimedia file and parse the read data. For example, when the multimedia file is a video file, the reading and parsing module 012 can be used to read the media stream of the video file and parse the media stream to obtain the video ES and audio ES corresponding to the media stream. Then, the video ES is sent to the video decoder 022, and the audio ES is sent to the audio decoder 023.

[0096] In some embodiments, when the first application 011 plays a multimedia file, the reading and parsing module 012 can sequentially read and parse the first frame data to the last frame data of the multimedia file. For example, the reading and parsing module 012 can read and parse one frame data of the multimedia file in one output cycle, and send the parsed data to the video decoder 022 and / or the audio decoder 023.

[0097] In some embodiments, when the first application 011 plays the multimedia file in a loop, the reading and parsing module 012 may read the first frame of data of the multimedia file in the next output cycle after reading the last frame of data of the multimedia file in one output cycle.

[0098] It should be understood that, in the case where the multimedia file is not played in a loop (for example, when the instruction to play the multimedia file in a loop is not detected), the reading and parsing module 012 can report a reading completion message to the application (for example, the first application 011) that plays the multimedia file after reading the last frame of data of the multimedia file. The reading completion message is used to indicate that the modules such as the video decoder 022, the audio decoder 023, the graphics processing unit (GPU) HAL 032, the hardware composer (HWC) HAL 034, and the audio HAL 033 will no longer process subsequent data after processing the last frame of data of the multimedia file, and end the playing of the multimedia file.

[0099] In other embodiments, the reading and parsing module 012 may also be split into a reading module and a parsing module. The reading module and the parsing module may be set in the same layer at the same time, for example, in each first application 011, in the playback subsystem and service 021, in the hardware abstraction layer 03 or the kernel layer 04. The reading module and the parsing module may also be set in different layers respectively, for example, the reading module may be set in the first application 011, and the parsing module may be set in the playback subsystem and service 021, etc. The embodiment of the present application does not limit the setting position of the reading module and the parsing module.

[0100] The user interface 013 is used to display the interface of the first application 011 on the smart TV 10 .

[0101] The playback control 014 is used to control the playback of multimedia files according to the user's operation. For example, the playback control 014 can perform pause, fast forward, fast rewind, adjust the progress or adjust the playback mode (such as single play, loop play, etc.) according to the user's operation.

[0102] In some embodiments, in response to the user adjusting the playback mode of the multimedia file to loop playback, the playback control 014 may send a loop playback instruction to the reading and parsing module 012. In response to the loop playback instruction, the reading and parsing module 012 may continue to read the first frame of data of the multimedia file after reading the last frame of data of the multimedia file.

[0103] It should be understood that in other embodiments, the application layer 01 may include more application programs, which are not limited here.

[0104] The framework layer 02 can provide an application programming interface (API) and a programming framework for the application in the application layer 01. The framework layer 02 includes some predefined functions. For example, the framework layer can include a playback subsystem and service 021, a graphics rendering 027, an audio rendering 028, and a system clock 029.

[0105] The playback subsystem and service 021 is used to provide the application of the application layer 01 with the ability to play multimedia files. In some embodiments, the playback subsystem and service 021 includes a video decoder 022, an audio decoder 023, synchronization 024, a video output 025 and an audio output 026.

[0106] The video decoder 022 is used to provide video decoding capabilities for the application program of the application layer. For example, the video decoder 022 can call the decoder HAL 031 and / or the graphics processor HAL032 to decode the video ES according to the video ES sent by the reading and parsing module 012 to obtain YUV encoded data. Exemplarily, the video decoder 022 can be used to decode video data in the formats of moving picture experts group (MPEG) 1, MPEG2, MPEG3, MPEG4, audio video interleaved (AVI), H.264, H.265, etc.

[0107] The audio decoder 023 is used to provide audio decoding capabilities for the application of the application layer 01. For example, the audio decoder 023 can call the audio HAL 033 to decode the audio ES according to the audio ES sent by the reading and parsing module 012 to obtain LPCM encoded data. Exemplarily, the audio decoder 023 can be used to decode audio data in formats such as MPEG, MP3, adaptive multi-rate coding (adaptive multi-rate, AMR), and advanced audio coding (advanced audio coding, ACC).

[0108] The video output 025 is used to call the video output 043 in the kernel layer 04 to output the image output data of the multimedia file to the display screen.

[0109] The audio output 026 is used to call the audio output 044 of the kernel layer 04 to output the sound output data of the multimedia file to the sound playback device (such as a sound box, a loudspeaker, etc.).

[0110] Synchronization 024 is used to synchronize the audio output data and the video output data of a multimedia file to ensure the synchronization of the video output data and the audio output data of the output multimedia file. In some embodiments, Synchronization 024 is further used to update the synchronization reference time of the video output data and the audio output data after the last frame of data of the multimedia is output.

[0111] It should be understood that the synchronization reference time may be the time when the multimedia file starts playing. For example, for a video file with a duration of 10s. If the smart TV 10 plays the data from the 0th second to the 10th second of the video file, the synchronization reference time is 0 second; if the smart TV 10 plays the data from the 5th second to the 10th second of the video file, the synchronization reference time is 5 seconds.

[0112] For any frame of data in the multimedia file, its timestamp t_t is the sum of the synchronization reference time t_ref and the played time t_p, that is, t_t = t_ref + t_p. After the last frame of data of the multimedia file is output, Synchronization 024 can update the synchronization reference time to t_ref and set t_p to 0. For example, assuming that the synchronization reference time is 5s, after the last frame of data of the multimedia file is output, Synchronization 024 can update the synchronization reference time to 5s and reset the played time to 0 second.

[0113] It should be understood that for the video output data and the audio output data of a frame of data, the corresponding timestamps are the same. When synchronizing the video output data and the audio output data, Synchronization 024 outputs the video output data and the audio output data with the same timestamp at the same time, or controls the time difference between the output times of the video output data and the audio output data with the same timestamp to be less than a preset time difference.

[0114] In some embodiments, when synchronizing the video output data and the audio output data, Synchronization 024 can adjust the audio output data based on the timestamp of the video output data, or adjust the video output data based on the timestamp of the audio output data, or adjust the video output data and the audio output data simultaneously based on an external time.

[0115] For example, assuming that the timestamp of the audio output data of the smart TV 10 is t_t1 and the timestamp of the video output data is t_t2 based on the timestamp of the audio output data. When t_t2 > t_t1, the output of the video output data can be delayed. For example, when the audio output data with the timestamp t_t2 is output, the video output data with the timestamp t_t2 is output. When t_t2 < t_t1, the output of the video output data with the timestamp t_t2 can be skipped, and the video output data with the timestamp t_t1 is output.

[0116] It should be understood that in other embodiments, synchronization 024 may not synchronize the image output data and the sound output data based on the timestamp of each frame output data, but may synchronize based on other data that can identify the video output data and the audio output data. The embodiment of the present application does not limit the data that synchronization 024 relies on to synchronize the image output data and the sound output data. For example, the image output data and the sound output data corresponding to the same frame data may have the same frame identifier (e.g., a sequence number), so that synchronization 024 can synchronize based on the frame identifier of the image output data and the sound output data.

[0117] The image rendering 027 is used to render the YUV-encoded data into an image. For example, the image rendering 027 can call the graphics processor HAL 032 and / or the hardware hybrid renderer HAL 034 to render the YUV-encoded data and obtain the image output data of the multimedia file.

[0118] The audio rendering 028 is used to render the LPCM-encoded data into sound output data. For example, the audio rendering 028 can call the audio HAL 033 to render the LPCM-encoded data and obtain the sound output data of the multimedia file.

[0119] The system clock 029 is used to provide a clock reference for each module of the smart TV 10.

[0120] It should be understood that in other embodiments, the framework layer 02 may also include more or fewer modules, which is not limited here.

[0121] The hardware abstraction layer 03 is an interface layer between the operating system kernel layer and other layers of the electronic device, and its purpose is to abstract the hardware and provide a virtual hardware platform for the operating system. For example, the hardware abstraction layer 02 may include a decoder HAL 031, a graphics processor HAL 032, an audio HAL 033, and a hardware hybrid renderer HAL 034.

[0122] The decoder HAL 031 is used to provide a video decoding calling interface for the application programs or modules in the application layer 01 and the framework layer 02.

[0123] The graphics processor HAL 032 is used to provide a graphics processor calling interface for application programs or modules in the application layer 01 and the framework layer 02.

[0124] The audio HAL 033 is used to provide a calling interface for audio encoding, audio decoding, audio rendering, etc. for the application programs or modules in the application layer 01 and the framework layer 02.

[0125] The hardware hybrid renderer HAL 034 is used to provide a calling interface for image synthesis and rendering for the application programs or modules in the application layer 01 and the framework layer 02.

[0126] It should be understood that in other embodiments, the hardware abstraction layer 03 may also include more or fewer modules, which is not limited here.

[0127] The kernel layer 04 is a layer between hardware and software. The kernel layer 04 may also be referred to as a driver layer. The kernel layer 04 may include a video driver 041, an audio driver 042, a video output 043, an audio output 044, and the like.

[0128] The video driver 041 is used to respond to the calls of the video decoder 022, the image processor HAL 032, the decoder HAL 031, and the hardware hybrid renderer HAL 034 to implement parsing, decoding, and rendering of video data.

[0129] The audio driver 042 is used to respond to the calls of the audio decoder 023 and the audio HAL 033 to implement parsing, decoding and rendering of audio data.

[0130] Video output 043 is used to respond to the call of video output 025 and output the image output data of the multimedia file to the corresponding display device.

[0131] The audio output 044 is used to respond to the call of the audio output 026 and output the audio output data of the multimedia file to the corresponding playback device.

[0132] It should be understood that in other embodiments, the kernel layer 04 may also include more or fewer modules, which is not limited here.

[0133] It should be understood that Figure 4 The software architecture of the smart TV 10 shown is only an example. In other embodiments, the architecture of the mobile phone 10 may include more or fewer layers / modules, and some modules may be merged or split, and the positions of some modules may be adjusted, which is not limited here.

[0134] For example, Figure 5 According to some embodiments of the present application, another software architecture diagram of a smart TV 10 is shown. Figure 5 As shown, compared to Figure 4 In the software architecture shown, the reading and parsing module 012 is not embedded in the first application 011, but is located in the playback subsystem and service 021 of the framework layer 02. In this way, when any application program of the smart TV 10 plays a multimedia file in a loop, the seamless loop playback of the multimedia file can be realized based on the reading and parsing module 012.

[0135] It should be understood that in other embodiments, the reading and parsing module 012 can also be set in other layers of the software architecture of the smart TV 10, such as the hardware abstraction layer 03, the kernel layer 04, etc., which is not limited here.

[0136] Next, combine Figure 4 The software architecture shown introduces the technical solution of this application.

[0137] For ease of understanding, the process of playing video files on the smart TV 10 is first introduced.

[0138] Figure 6 According to some embodiments of the present application, a schematic diagram of a process of playing a video by a smart TV 10 is shown.

[0139] like Figure 6 As shown, the process includes the following steps:

[0140] First, the reading and parsing module 012 reads the current frame data of the video file.

[0141] In some embodiments, the data read by the reading and parsing module 012 may be local data stored in the smart TV 10 or network data obtained from other devices.

[0142] In some embodiments, the reading and parsing module 012 may store the read current frame data into an input buffer corresponding to the reading and parsing module 012 .

[0143] It should be understood that the current frame data may be any frame data in the video file.

[0144] In some embodiments, when the current frame data is the last frame data of the multimedia file, and the smart TV 10 does not play the multimedia file in a loop, the reading and parsing module 012 can report a reading completion message to the application (e.g., the first application 011) that plays the multimedia file after reading the current frame data. The reading completion message is used to instruct the video decoder 022, the audio decoder 023, the graphics processor HAL 032, the hardware hybrid renderer HAL 034, the audio HAL 033 and other modules to stop processing subsequent data after processing the last frame data of the multimedia file, and end the playback of the multimedia file.

[0145] Then, the reading and parsing module 012 parses the current frame data of the read video file.

[0146] The reading and parsing module 012 parses the read current frame data to obtain the video ES and audio ES corresponding to the current frame data, and sends the video ES and audio ES to the video decoder 022 and the audio decoder 023 respectively.

[0147] In some embodiments, if the smart TV 10 plays an audio file, the reading and parsing module 012 may only generate an audio ES and send the audio ES to the audio decoder 023. If the multimedia file played by the smart TV 10 has only images but no sound, the reading and parsing module 012 may only generate a video ES and send the video ES to the video decoder 022.

[0148] In some embodiments, the reading and parsing module 012 may send the video ES and the audio ES to the video decoder 022 and the audio decoder 023 respectively by storing the video ES and the audio ES in an output buffer corresponding to the reading and parsing module 012 .

[0149] In some embodiments, the reading and parsing module 012 can store the video ES and audio ES corresponding to the current frame data into the output cache corresponding to the stream parsing module 012 after the video ES and audio ES corresponding to the previous frame data stored in the reading and parsing module 012 have been processed by the video decoder 022 and the audio decoder 023.

[0150] In some embodiments, the smart TV 10 plays a multimedia file in a loop. When the current frame data is the first frame data of the multimedia file, the previous frame data may be the last frame data of the multimedia file.

[0151] It should be understood that when the smart TV 10 is not playing a multimedia file in a loop, when the current frame data is the first frame data of the multimedia file, the output cache corresponding to the parsing module 012, the output cache corresponding to the video decoder 022, the output cache corresponding to the audio decoder 023, the image output cache, the audio output cache, and the data in the output cache of the module are empty.

[0152] Next, the video ES and the audio ES are decoded.

[0153] Exemplarily, the video decoder 022 decodes the video ES to obtain corresponding YUV encoded data, and the audio decoder 023 decodes the audio ES to obtain LPCM encoded data.

[0154] In some embodiments, the video decoder 022 can read the video ES from the output buffer corresponding to the reading and parsing module 012 for decoding, and store the decoded YUV coded data in the output buffer corresponding to the video decoder 022. The audio decoder 023 can read the audio ES from the output buffer corresponding to the reading and parsing module 012 for decoding, and store the decoded LPCM coded data in the output buffer corresponding to the audio decoder 023.

[0155] In some embodiments, the video decoder 022 may store the YUV coded data corresponding to the current frame data to the output buffer corresponding to the video decoder 022 after the graphics processor HAL 032 and / or the hardware hybrid renderer HAL 034 have completed rendering the YUV coded data corresponding to the previous frame data. The audio decoder 023 may store the LPCM coded data corresponding to the current frame data to the output buffer corresponding to the audio decoder 023 after the audio HAL 033 has completed rendering the LPCM coded data corresponding to the previous frame data.

[0156] In some other embodiments, the video decoder 022 may decode the video ES to obtain data encoded in other forms, which is not limited here.

[0157] In other embodiments, the audio decoder 023 may decode the audio ES to obtain data encoded in other forms, which is not limited here.

[0158] Again, render the YUV encoded data and LPCM data.

[0159] Exemplarily, the graphics processor HAL 032 and / or the hardware hybrid renderer HAL 034 renders the YUV encoded data to obtain image output data, and the audio HAL 033 renders the LPCM encoded data to obtain sound output data.

[0160] In some embodiments, the graphics processor HAL 032 and / or the hardware hybrid renderer HAL 034 can read the YUV encoded data from the output buffer of the video decoder 022, render the YUV encoded data, and then store the image output data in the video output buffer. The audio HAL 033 can read the LPCM encoded data from the output buffer corresponding to the audio decoder 023, render the LPCM encoded data, and then store the sound output data in the audio output buffer.

[0161] In some embodiments, the graphics processor HAL 032 and / or the hardware hybrid renderer HAL 034 may store the image output data of the current frame in the video output buffer after the image output data corresponding to the previous frame data stored in the video output buffer has been output. The audio HAL 033 may store the sound output data of the current frame in the video output buffer after the sound output data corresponding to the previous frame data stored in the audio output buffer has been output.

[0162] Finally, after synchronization 024 synchronizes the image output data and the audio data, the image output data and the sound output data are output to corresponding output devices by video output 025 and audio output 026 respectively.

[0163] For example, video output 025 can call video driver 041 and video output 043 to output image output data to the display screen of smart TV 10 or other image output devices connected to smart TV 10. Audio output 026 can call audio driver 042 and audio output 044 to output sound output data to the speaker of smart TV 10 or audio output devices connected to smart TV 10.

[0164] based on Figure 6 The embodiment shown, referring to Figure 7 When the smart TV 10 loops and plays a multimedia file including N frames of data (for example, when an instruction to loop and play a multimedia file including N frames of data is detected), the smart TV 10 can implement seamless loop playback of the multimedia file through the following steps:

[0165] S1, the reading and parsing module 012 continues to read the 1st frame of data after finishing reading the Nth frame of data.

[0166] When the reading and parsing module 012 plays the multimedia file in a loop (for example, when an instruction to play the multimedia file including N frames of data in a loop is detected), the reading and parsing module 012 continues to read the first frame of data after reading the Nth frame of data.

[0167] In some embodiments, after reading the Nth frame data, the reading and parsing module 012 does not report the reading completion message to the first application 011. In this way, after the reading and parsing module 012 reads the first frame data, the video decoder 022, the audio decoder 023, the graphics processor HAL 032, the hardware hybrid renderer HAL 034, the audio HAL 033 and other modules will continue to process the first frame data after processing the Nth frame data, and obtain the output data corresponding to the first frame data. The specific processing process can be referred to Figure 6 The illustrated embodiments are not described in detail here.

[0168] S2, after synchronization 024 outputs the Nth frame image, the synchronization reference time is updated.

[0169] After outputting the Nth frame of image, the synchronization 024 updates the synchronization reference time so as to regenerate the timestamp of the output data of each frame of data. Thus, after the image output data (the first frame of image) and the audio output data (the first sound) corresponding to the first frame of data are generated, the synchronization 024 can be synchronized based on the timestamp of the first frame of image and the timestamp of the first sound.

[0170] It should be understood that the synchronization reference time may be the time when the multimedia file starts playing. In some embodiments, the timestamp t_t of any frame data in the multimedia file is the sum of the synchronization reference time t_ref and the played time t_p, that is, t_t=t_ref+t_p. After outputting the last frame data of the multimedia file, the synchronization 024 may update the synchronization reference time to t_ref and set t_p to 0. For example, assuming that the synchronization reference time is 5s, the synchronization 024 may update the synchronization reference time to 5s and reset the played time to 0 seconds after outputting the last frame data of the multimedia file.

[0171] S3, the video output 025 outputs the first frame of image in the second output cycle after outputting the Nth frame of image in the first output cycle; the audio output 026 outputs the first segment of sound in the second output cycle after outputting the Nth segment of sound in the first output cycle.

[0172] After synchronization 024 updates the synchronization reference time, video output 025 and audio output 026 can respectively call video output 043 and audio output 044 in the second output cycle to output the first frame image to the display screen of smart TV 10 and the first sound to the speaker of smart TV 10 respectively.

[0173] Based on the above method, the smart TV 10 can achieve seamless loop playback of multimedia files, avoiding the inconsistency of the images displayed or the sounds played by the smart TV 10.

[0174] Next, combine Figure 1 , Figure 3A and Figure 3B The situation shown, Figure 4 The software architecture shown, Figure 6 and Figure 7 The process of playing a video file on the smart TV 10 shown introduces the technical solution of the present application.

[0175] Figure 8 According to some embodiments of the present application, a flowchart of a multimedia file playing method is shown. Figure 8 As shown, the method comprises the following steps:

[0176] S801: An instruction to loop-play a multimedia file is detected.

[0177] When the smart TV 10 detects an instruction to play a multimedia file in a loop, the multimedia file playing method provided in the embodiment of the present application is triggered.

[0178] Exemplarily, when the smart TV 10 detects that the user turns on the aforementioned “picture frame mode”, or when the smart TV 10 automatically switches to the “picture frame mode”, the smart TV 10 can detect an instruction to play multimedia files in a loop.

[0179] For example, after the user selects the operation of looping the video in the display interface of the first application 011 on the smart TV 10, the smart TV 10 can detect the instruction of looping the multimedia file. Fig.9A , the smart TV 10 can display a video setting window in response to a user operation, and display a play mode window after the user selects the "play mode" option 91. After the smart TV 10 detects the user's selection operation on the "single loop" option 92, the smart TV 10 can detect an instruction to play the multimedia file in a loop.

[0180] It should be understood that in other embodiments, the device may also detect instructions for looping multimedia files in other situations, which are not limited here.

[0181] For example, in some applications, when a certain interface is displayed, the multimedia file of the interface is automatically played in a loop by default. When the user switches to the interface of the application, the device can detect the instruction to play the multimedia file in a loop. Fig. 9B The social information of "user A" displayed on the display interface of the social application of the mobile phone 20 includes a video playback control 93. When the user switches to the display interface, the smart TV 10 can detect an instruction to loop the multimedia file.

[0182] S802: After reading the last frame of data of the multimedia file, continue to read the first frame of data of the multimedia file.

[0183] Exemplarily, the smart TV 10 can use the aforementioned reading and parsing module 012 to continue reading the first frame of the multimedia file after reading the last frame of the multimedia file. In other words, the last frame of the multimedia file and the first frame of the multimedia file are read continuously.

[0184] For example, corresponding to Figure 3A In the case shown, the reading and parsing module 012 reads 1 After the Nth frame of data (ie, the last frame of data) of the multimedia file is read, the first frame of data of the multimedia file is read.

[0185] For another example, corresponding to the situation shown in 3B, the reading and parsing module 012 reads 8 After the Nth frame of data (ie, the last frame of data) of the multimedia file is read, the first frame of data of the multimedia file is read.

[0186] In some embodiments, after the smart TV 10 uses the aforementioned reading and parsing module 012 to read the first frame data of the multimedia file, it can store the video ES and audio ES corresponding to the first frame data in the output buffer corresponding to the reading and parsing module 012.

[0187] In some embodiments, after reading the EOS mark of the multimedia file, the smart TV 10 may determine that the last frame of data of the multimedia file has been read and continue to read the first frame of data of the multimedia file.

[0188] It should be understood that, in the case of loop playback of multimedia files (for example, in the case of detecting a loop playback instruction of multimedia files), after reading the last frame of data of the multimedia file, the reading and parsing module 012 does not report a message of the end of reading to the application program that plays the multimedia file. After the reading and parsing module 012 reads the first frame of data of the multimedia file, the video decoder 022, the audio decoder 023, the graphics processor HAL 032, the hardware hybrid renderer HAL 034, the audio HAL 033 and other modules will continue to process the first frame of data of the multimedia file after processing the last frame of data of the multimedia file.

[0189] It should be understood that after reading the first frame of the multimedia file, the smart TV 10 will continue to read, parse, decode, render and output other frames of the multimedia file in sequence. The processing process of each frame of data by the smart TV 10 can be referred to Figure 6 The embodiments shown are not limited here.

[0190] S803: outputting the last frame data of the multimedia file in a first output cycle, and processing the first frame data before the first output cycle ends to obtain first output data corresponding to the first frame data.

[0191] The smart TV 10 outputs the last frame data corresponding to the output data of the multimedia file in the first output cycle, and decodes and renders the first frame data before the end of the first output cycle to obtain the first output data corresponding to the first frame data.

[0192] Exemplarily, after the smart TV 10 uses the reading and parsing module 012 to store the video ES and the audio ES of the first frame data of the multimedia file into the output buffer corresponding to the reading and parsing module 012, the first frame data may be processed in the following manner to obtain the first output data corresponding to the first frame data:

[0193] For the video ES, the video decoder 022 reads the video ES from the output buffer corresponding to the reading and parsing module 012 for decoding, and stores the decoded YUV coded data in the output buffer corresponding to the video decoder 022. The graphics processor HAL 032 and / or the hardware hybrid renderer HAL 034 reads the YUV coded data from the output buffer of the video decoder 022, renders the YUV coded data to obtain image output data corresponding to the multimedia file, and stores the image output data in the video output buffer.

[0194] For the audio ES, the audio decoder 023 reads the audio ES from the output buffer corresponding to the reading and parsing module 012 for decoding, and stores the decoded LPCM encoded data in the output buffer corresponding to the audio decoder 023. The audio HAL 033 reads the LPCM encoded data from the output buffer corresponding to the audio decoder 023, renders the LPCM encoded data to obtain sound output data, and stores the sound output data in the audio output buffer.

[0195] That is, if the multimedia file includes video data and audio data, the first output data includes image output data stored in the video output buffer and sound output data stored in the audio output buffer. If the multimedia file includes video data but does not include audio data, the first output data includes image output data stored in the video output buffer. If the multimedia file includes audio data but does not include video data, the first output data includes sound output data stored in the audio output buffer.

[0196] For example, for Figure 3A In the situation shown, the smart TV 10 can 4 Time to t 5 During the time period (i.e., the first output cycle), the Nth frame of data is output. 1 Time to t 5 During the time period, the first frame data of the multimedia file is read, parsed, decoded and rendered in sequence to obtain first output data corresponding to the first frame data.

[0197] For example, for Figure 3B In the situation shown, the smart TV 10 can 10 Time to t 11 During the time period (i.e., the first output cycle), the Nth frame of data is output. 8 Time to t 11 During the time period, the first frame data of the multimedia file is read, parsed, decoded and rendered in sequence to obtain output data corresponding to the first frame data.

[0198] It should be understood that after the smart TV 10 outputs the last frame of the multimedia file, it can also update the synchronization reference time so that the synchronization 024 can synchronize the image output data and the sound output data corresponding to the first frame of the multimedia file with the correct reference time. For example, assuming that the synchronization reference time is 5s, the synchronization 024 can update the synchronization reference time to 5s and reset the played time to 0 seconds after outputting the last frame of the multimedia file.

[0199] S804: Output the first output data in a second output cycle, wherein the second output cycle is an output cycle after the first output cycle.

[0200] The smart TV 10 outputs the first output data in an output cycle following the first output cycle.

[0201] For example, for Figure 3A In the situation shown, the smart TV 10 can 5 Time to t 6 During the time period (ie, the second output cycle), output data corresponding to the first frame data (first output data) is output.

[0202] For example, for Figure 3B In the situation shown, the smart TV 10 can 11 Time to t 12 During the time period (ie, the second output cycle), output data corresponding to the first frame data (first output data) is output.

[0203] It should be understood that after the smart TV 10 outputs the first output data in the second output cycle, it can also output other frame data of the multimedia file in sequence in the following output cycle.

[0204] By implementing the method provided by the present application, when the smart TV 10 plays a video in a loop, after reading the last frame of data of the multimedia file, it continues to read the first frame of data of the multimedia file. Then, before the end of the first output cycle, the smart TV 10 processes the first frame of data of the multimedia file to obtain the first output data corresponding to the first frame of data, and outputs the first output data in the second output cycle. In this way, seamless loop playback of multimedia files can be achieved.

[0205] further, Fig.10 According to some embodiments of the present application, a schematic structural diagram of a smart TV 10 is shown.

[0206] like Fig.10 As shown, the smart TV 10 may include a processor 110 , a memory 120 , a display screen 130 , a communication module 140 , a power switch 150 , an input / output interface 160 , an audio module 170 , and the like.

[0207] The processor 110 may include one or more processing units. For example, the processor 110 may include a central processing unit (CPU), an application processor (AP), a modem processor, a GPU, an image signal processor (ISP), a microcontroller unit (MCU), a video codec, a digital signal processor (DSP), a baseband processor, a neural-network processing unit (NPU), a field programmable gate array (FPGA), etc. In some embodiments, different processing units may be independent devices or integrated into one or more processors.

[0208] In some embodiments, the processor 110 may be used to execute instructions of the multimedia file playing method provided in the aforementioned embodiments. For example: when playing a multimedia file in a loop (for example, when a multimedia file playing in a loop instruction is detected), an instruction to continue reading the first frame data of the multimedia file after reading the last frame data of the multimedia file, an instruction to render the first frame data before the output cycle of the last frame data ends to obtain output data corresponding to the first frame data, an instruction to synchronize the image output data and the sound output data of the multimedia file after the last frame data of the multimedia file is output, etc.

[0209] The memory 120 can be used to store data and instructions. For example, the memory can be used to store instructions of the multimedia file playing method provided in the above embodiments, store multimedia files, etc.

[0210] The display screen 130 is used to display images, videos, etc. The display screen 130 includes a display panel. In some embodiments, the display screen 130 can be used to output image output data of a multimedia file, such as displaying an image corresponding to the multimedia file.

[0211] The communication module 140 is used to realize the communication between the smart TV 10 and other devices. For example, the communication module 140 can be used to obtain the media stream corresponding to the multimedia file from other devices, transmit the output data corresponding to the multimedia file to other devices, etc.

[0212] In some embodiments, the communication module may include a mobile communication module and a wireless communication module.

[0213] The mobile communication module can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the smart TV 10. The mobile communication module may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module can receive electromagnetic waves by an antenna, and filter, amplify, and process the received electromagnetic waves, and transmit them to a modulation and demodulation processor for demodulation. The mobile communication module can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna. In some embodiments, at least some of the functional modules of the mobile communication module may be arranged in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module may be arranged in the same device as at least some of the modules of the processor 110.

[0214] The wireless communication module can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the smart TV 10. The wireless communication module can be one or more devices integrating at least one communication processing module. The wireless communication module receives electromagnetic waves via an antenna, modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor. The wireless communication module can also receive the signal to be sent from the processor 110, modulate the frequency, amplify it, and convert it into electromagnetic waves for radiation through the antenna.

[0215] The power switch 150 is used to turn the smart TV 10 on or off.

[0216] The input / output interface 160 includes interfaces for receiving data from other devices or outputting data (such as audio, video, etc.) to other devices, including but not limited to high definition multimedia interface (HDMI), S / PDIF (Sony / Philips digital interconnect format, a standard format used by Sony TM and Philips TMDigital audio interface protocol developed by the company) interface, local area network (LAN) interface, universal serial bus (USB) interface, etc.

[0217] In some embodiments, the smart TV 10 can output the output data (such as image output data and sound output data) corresponding to the multimedia file to other output devices connected to the input and output interface 160 (such as a display device, a sound device, etc. connected to the input and output interface) through the input and output interface 160.

[0218] The audio module 170 is used to implement the audio function of the smart TV 10. The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. For example, the audio module 170 may include a speaker, a microphone, etc. The speaker, also known as a "speaker", is used to convert an audio electrical signal into a sound signal. The microphone 170, also known as a "microphone" or "microphone", is used to convert a sound signal into an electrical signal.

[0219] In some embodiments, the smart TV 10 can output the sound output data of the multimedia file through the audio module 170, for example, playing the sound corresponding to the multimedia file through a speaker.

[0220] In some embodiments, the smart TV 10 may also be configured with a remote controller so that the user can control the smart TV 10 to play multimedia files in a loop through the remote controller.

[0221] The structure of the smart TV 10 shown in the embodiment of the present application is only an example. In other embodiments, the smart TV 10 may include more or fewer modules, and some modules may be merged or split, which is not limited here.

[0222] The embodiment of the present application also provides a program product, which, when executed on a device, can enable the device to implement the multimedia file playback method provided by the aforementioned embodiments.

[0223] An embodiment of the present application further provides a readable storage medium, in which instructions are stored. When the instructions are executed by a device, the device implements the multimedia file playback method provided by the aforementioned embodiments.

[0224] An embodiment of the present application further provides a chip or a chip system, including a processing module and a communication interface, wherein the processing module is used to implement the multimedia file playback method provided in the aforementioned embodiments.

[0225] In the accompanying drawings, some structural or method features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be required. Instead, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. In addition, the inclusion of structural or method features in a particular figure does not mean that such features are required in all embodiments, and in some embodiments, these features may not be included or may be combined with other features.

[0226] It should be noted that in the examples and description of this patent, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "includes" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "includes one" do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Claims

1. A multimedia file playing method, characterized in that: include: When the device plays the multimedia file in a loop, the device reads the Kth frame data of the multimedia file in the Kth output cycle, processes the read Kth frame data in the Kth output cycle to the K+Q-1th output cycle, and outputs the output data corresponding to the Kth frame data in the K+Qth output cycle, wherein Q is an integer greater than 0, the multimedia file includes N frames of data, and K is an integer greater than 0 and less than N; The device reads the Nth frame data of the multimedia file in the Nth output cycle, and outputs the output data corresponding to the NQth frame data of the multimedia file; The device reads the first frame data of the multimedia file in the N+1th output cycle, and outputs output data corresponding to the N-Q+1th frame data of the multimedia file; The device reads the Qth frame data of the multimedia file in the N+Qth output cycle, and outputs the output data corresponding to the Nth frame data of the multimedia file; The device reads the Q+1th frame data of the multimedia file in the N+Q+1th output cycle, and outputs output data corresponding to the 1st frame data of the multimedia file.

2. The method according to claim 1, characterized in that The multimedia file is a video file, and the output data corresponding to the first frame data includes first image output data and first audio output data; and The device reads the Q+1th frame data of the multimedia file in the N+Q+1th output cycle, and outputs the output data corresponding to the first frame data of the multimedia file, including: After the device outputs the output data corresponding to the Nth frame data of the multimedia file, the device updates the synchronization reference time, where the synchronization reference time is used to indicate the start playback time of the multimedia file; The device synchronizes the first image data and the first audio data based on the synchronization reference time, and outputs the first image output data and the first audio output data in the N+Q+1th output period.

3. The method according to claim 1 or 2, characterized in that: The device reads the Nth frame data of the multimedia file in the Nth output cycle, including: After reading the Nth frame of data of the multimedia file, the device does not report a reading completion message of the multimedia file to the application program that plays the multimedia file.

4. The method according to any one of claims 1 to 3, characterized in that The method further comprises: The device sequentially outputs output data corresponding to the second frame data of the multimedia file to output data corresponding to the Nth frame data of the multimedia file in the N+Q+2th output cycle to the N+Q+Nth output cycle.

5. The method according to claim 1, characterized in that The method further comprises: The device processes the first frame data of the multimedia file from the N+1th output cycle to the N+Qth output cycle to obtain output data corresponding to the first frame data.

6. The method according to claim 5, characterized in that The device processes the first frame data of the multimedia file from the N+1th output cycle to the N+Qth output cycle, at least comprising: The device parses, decodes and renders the first frame data during the N+1th output cycle to the N+Qth output cycle.

7. The method according to any one of claims 1 to 6, characterized in that The device plays the multimedia file through a first application; and the device reads the Nth frame data of the multimedia file in the Nth output cycle, including: The first application reads the Nth frame data of the multimedia file in the Nth output cycle.

8. The method according to claim 7, characterized in that The device reads the first frame data of the multimedia file in the N+1th output cycle, including: The first application reads the first frame data of the multimedia file in the N+1th output cycle.

9. A readable storage medium, characterized in that: The readable storage medium includes instructions, and when the instructions are executed by the device, the device implements the multimedia file playing method according to any one of claims 1 to 8.

10. A device, characterized in that: include: A memory for storing instructions; A processor, configured to execute the instruction so that the device implements the multimedia file playing method according to any one of claims 1 to 8.

Images