Audio track switching method and device

Abstract

The present application discloses a method and device for switching audio tracks, which are used to solve the problem that the mute mode affects the user experience when audio is switched. The method provided in the present application includes: receiving an audio track switching instruction, wherein the audio track switching instruction indicates that the currently played audio data is switched from the data of the first audio track to the data of the second audio track; obtaining the first maximum timestamp of the data of the first audio track included in the decoding buffer, wherein the decoding buffer includes decoded and unrendered audio data; according to the first maximum timestamp, decoding the audio data in the data of the second audio track whose timestamp is greater than the first maximum timestamp and saving it to the decoding buffer.

Description

Technical Field

[0001] The present application relates to the field of audio processing technology, and in particular to a method and device for switching audio tracks. Background Art

[0002] When switching audio tracks, if the encoding formats of the audio data before and after the switch are different, it will involve encoder switching, that is, the release of the old decoder and the creation of a new decoder, which takes a certain amount of time. The decoder decodes the audio data and sends it to the decoding buffer. The rendering module obtains the audio data from the decoding buffer and renders it. At present, when switching audio tracks, all the decoded data of the previous audio track cached in the decoding buffer is discarded at the beginning of the switch. Because the data of the new audio track takes time to complete the creation and decoding of the decoder, and it takes a certain amount of time for the new data to be sent to the rendering module. At present, mute is directly set during this time, and the mute is unmute after the new data arrives at the rendering module. However, this muting method will affect the user experience to a certain extent. Summary of the invention

[0003] The present application proposes a method and device for switching audio tracks, so as to solve the problem that the mute mode affects the user experience during audio switching.

[0004] In a first aspect, an embodiment of the present application provides a method for switching audio tracks, including:

[0005] Receive a track switching instruction, the track switching instruction instructing to switch the currently played audio data from the data of the first track to the data of the second track; obtain a first maximum timestamp of the data of the first track included in a decoding buffer, the decoding buffer including decoded and unrendered audio data; according to the first maximum timestamp, decode the audio data in the data of the second track whose timestamp is greater than the first maximum timestamp and save it to the decoding buffer.

[0006] Based on the above scheme, when an audio switching instruction is received to switch the currently played audio data from the data of the first audio track to the data of the second audio track, the decoded audio data in the first audio track is no longer discarded, but the decoded audio data is saved in the decoding buffer. By obtaining the first maximum timestamp of the decoded data of the first audio track, the audio data in the data of the second audio track that is greater than the first maximum timestamp is decoded and saved in the decoding buffer, so as to achieve continuous output of data and avoid the problem of poor user experience caused by sudden silence.

[0007] In a possible implementation, the method further includes: receiving a video stream; and upon receiving an audio track switching instruction, synchronously rendering the video data corresponding to the audio data with the minimum timestamp included in the video stream when rendering the audio data with the minimum timestamp in the decoding buffer.

[0008] Based on the above solution, when switching the audio track, the mute mode is no longer set, but the audio data is continuously output. Therefore, when switching the audio track data, the video stream does not need to be paused or frame-tracked to synchronize the video stream with the audio data, which improves the user's viewing experience.

[0009] In one possible implementation, when it is determined that the first decoder does not support decoding of the data of the second audio track, a second decoder is created; the first decoder supports decoding of the data of the first audio track; and the audio data in the data of the second audio track whose timestamp is greater than the first maximum timestamp is decoded, including: decoding the audio data in the data of the second audio track whose timestamp is greater than the first maximum timestamp through the decoder.

[0010] Based on the above solution, the old decoder is released and a new decoder is created to facilitate decoding of the data of the switched second audio track.

[0011] In a possible implementation, after the switching is completed, a second maximum timestamp of the rendered audio data is obtained; and the data of the first audio track is synchronized with the rendered audio data according to the second maximum timestamp.

[0012] In a possible implementation, after the switching is completed, the audio data in the data of the first audio track whose timestamp is smaller than the first maximum timestamp is discarded.

[0013] Based on the above solution, the audio data with a timestamp smaller than the first maximum timestamp in the data of the first audio track is discarded to facilitate synchronization with the rendered audio data of the second audio track.

[0014] In a second aspect, an embodiment of the present application provides a display device, including a communication interface, a processor, an audio output interface, and a display screen;

[0015] The communication interface is used to receive an audio track switching instruction, wherein the audio track switching instruction instructs to switch the currently played audio data from the data of the first audio track to the data of the second audio track;

[0016] The processor is used to obtain a first maximum timestamp of data of a first audio track included in a decoding buffer, wherein the decoding buffer includes decoded and unrendered audio data. The processor is also used to decode, according to the first maximum timestamp, audio data in the data of the second audio track whose timestamp is greater than the first maximum timestamp, and save the data to the decoding buffer, and render the audio data in the decoding buffer.

[0017] The audio output interface is used to output the audio data after rendering.

[0018] The display screen is used to display video data corresponding to the audio data.

[0019] In some embodiments, the display device further includes a speaker, and the speaker is used to play audio data output by the audio output interface.

[0020] In some embodiments, the communication interface is further used to receive a video stream;

[0021] The processor, upon receiving the audio track switching instruction, is further configured to synchronously render the video data corresponding to the audio data with the minimum timestamp included in the video stream when rendering the audio data with the minimum timestamp in the decoding buffer.

[0022] In some embodiments, the processor is further configured to: create a second decoder when it is determined that the first decoder does not support decoding the data of the second audio track; the first decoder supports decoding the data of the first audio track;

[0023] The processor, when decoding the audio data in the data of the second audio track whose timestamp is greater than the first maximum timestamp, is specifically configured to: decode the audio data in the data of the second audio track whose timestamp is greater than the first maximum timestamp.

[0024] In some embodiments, the processor is further used to: after completing the switching, obtain a second maximum timestamp of the rendered audio data; and synchronize the data of the first audio track with the rendered audio data according to the second maximum timestamp.

[0025] In a third aspect, an embodiment of the present application provides a device for switching audio tracks, including a memory and a processor;

[0026] A memory for storing program instructions;

[0027] The processor is used to call the program instructions stored in the memory and execute the method described in the first aspect and different implementation methods of the first aspect according to the obtained program.

[0028] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions. When the computer instructions are executed on a computer, the computer executes the method described in the first aspect and different implementation methods of the first aspect.

[0029] In addition, the technical effects brought about by any implementation method of the second to fourth aspects can refer to the technical effects brought about by the first aspect and different implementation methods of the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0031] Figure 1 A schematic diagram of a usage scenario of a display device provided in an embodiment of the present application;

[0032] Figure 2 A configuration block diagram of a control device provided in an embodiment of the present application;

[0033] Figure 3 A schematic diagram of the hardware structure of a display device provided in an embodiment of the present application;

[0034] Figure 4 A schematic diagram of a system of a display device provided in an embodiment of the present application;

[0035] Figure 5 A schematic diagram of the logical structure of a media player provided in an embodiment of the present application;

[0036] Figure 6 A flowchart of a method for switching audio tracks provided in an embodiment of the present application;

[0037] Figure 7 A schematic diagram of audio track switching provided in an embodiment of the present application;

[0038] Figure 8 A schematic diagram of audio track switching provided in an embodiment of the present application;

[0039] Fig. 9 An example diagram of a scene for switching audio tracks provided in an embodiment of the present application;

[0040] Fig.10 A schematic diagram of a display device provided in an embodiment of the present application;

[0041] Fig.11 A schematic diagram of a device for switching audio tracks provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0043] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0044] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0045] When the audio data in the track queue is output from the decoder, it needs to be decoded by the decoder and sent to the decoding buffer. The rendering module reads the audio data from the decoding buffer, performs rendering processing and outputs. The decoding buffer can cache the audio data that has been decoded and not rendered. Therefore, at a certain moment, the timestamp of the audio data being decoded by the decoder is not the same as the timestamp of the audio data rendered by the rendering module. At a certain moment, the timestamp of the audio data decoded by the decoder will be later than the timestamp of the audio data rendered by the rendering module. In the process of switching tracks, if the encoding format of the currently played audio data is different from that of the audio data to be switched, and the current encoder cannot decode the audio data to be switched, it is necessary to switch the decoder, that is, release the current decoder and create a new decoder. When releasing the current decoder and creating a new decoder, it takes a certain amount of time. In the existing technical solution, when switching tracks, the audio data that has been decoded in the track being played is all discarded, that is, the audio data in the decoding buffer is all discarded. However, it takes a certain amount of time for the audio data in the new track to be switched to be sent to the rendering module. Therefore, the audio is usually muted at the beginning of the switch until the rendering module can read the audio data in the track to be switched from the decoded buffer. However, this sudden muting method affects the user experience.

[0046] In view of the above problems, the embodiment of the present application provides a data processing method and device for switching audio tracks. When switching audio tracks, the audio data in the decoding buffer are no longer discarded. Instead, the audio data that has been decoded in the currently playing audio track is sent to the decoding buffer, and the audio data of the audio track queue to be switched is synchronized according to the timestamp (T) of the audio data that has been decoded in the currently playing audio track queue, and the audio data after the timestamp (T) in the audio track queue to be switched is output. During the switching process, there is no need to set mute, and the audio data before and after the switching is continuous.

[0047] Below, in order to facilitate understanding of the solution of the present application, the solution of the present application will be described with multiple embodiments in different scenarios. The solution proposed in the present application can be applied to a terminal device, and the terminal device involved in the present application has a track switching function, such as a radio, MP4, speaker, computer, TV, smart phone or tablet computer. In some embodiments, the terminal device involved in the embodiments of the present application may also have a display function. For example, the terminal device may be an audio and video playback device (or display device), such as a TV.

[0048] Figure 1 Schematic diagram of a usage scenario of a display device according to an embodiment. Figure 1As shown, the display device 200 also performs data communication with the server 400 , and the user can establish a Bluetooth connection or an infrared connection with the display device 200 through the smart device 300 or the control apparatus 100 to operate the display device 200 .

[0049] In some embodiments, the control device 100 may be a remote controller, and the communication between the remote controller and the display device 200 includes infrared protocol communication or Bluetooth protocol communication, and at least one of other short-range communication methods, and the display device 200 is controlled wirelessly or wired. The user may input user commands through at least one of buttons on the remote controller, voice input, and control panel input to control the display device 200. For example, the user may input corresponding control commands through the volume plus and minus keys, channel control keys, up / down / left / right movement keys, voice input keys, menu keys, power on / off keys, etc. on the remote controller to realize the function of controlling the display device 200.

[0050] In some embodiments, the smart device 300 may include any one of a mobile terminal, a tablet computer, a computer, a laptop computer, an augmented reality (AR) or virtual reality (VR) device, etc. In some embodiments, the smart device 300 may also be used to control the display device 200. For example, the display device 200 is controlled using an application running on the smart device.

[0051] In some embodiments, the display device 200 can also be controlled in a manner other than the control device 100 and the smart device 300. For example, the display device 200 can directly receive the user's voice command control through a module for acquiring voice commands configured inside the display device 200, or can receive the user's voice command control through a voice control device provided outside the display device 200. In some scenarios, the display device 200 is configured with a touch screen panel, and a plurality of virtual keys are provided on the touch screen panel. The user can input corresponding control commands through the plurality of virtual keys on the touch screen panel to realize the function of controlling the display device 200.

[0052] In some embodiments, the display device 200 also communicates data with the server 400. The display device 200 may be allowed to communicate via a local area network (LAN), a wireless local area network (WLAN), and other networks. The server 400 may provide various content and interactions to the display device 200. The server 400 may be a cluster or multiple clusters, and may include one or more types of servers. The server 400 may provide audio data streams and video streams for the display device 200.

[0053] Figure 2 Schematically shows a block diagram of a configuration of the control device 100 according to an exemplary embodiment. Figure 2 As shown, the control device 100 includes a processor 110, a memory 120, a communication interface 130, a user input / output interface 140, and a power supply 150. The control device 100 can receive the user's input operation instructions, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, and play the role of an interactive intermediary between the user and the display device 200. In some embodiments, the communication interface 130 is used to communicate with the outside, including at least one of a wireless fidelity (Wireless Fidelity, Wi-Fi) chip, a Bluetooth component, a near field communication (NearField Communication, NFC) or an alternative module. The Bluetooth component includes an antenna for sending and receiving Bluetooth signals, and may also include a controller for receiving instructions sent by the processor 110. The Bluetooth component can be used to send Bluetooth signals to the outside or receive external Bluetooth signals under the control of the processor 110. For example, the processor 110 can establish a Bluetooth connection with the display device 200 through the Bluetooth component, thereby controlling the display device 200. In some embodiments, the user input / output interface 140 includes at least one of a microphone, a touchpad, a sensor, a button or an alternative module.

[0054] Figure 3 The hardware structure diagram of the display device 200 is exemplarily shown. In some embodiments, the display device 200 includes: a radio frequency (RF) circuit 210, a memory 220, a display unit 230, a camera 240, a sensor 250, an audio circuit 260, a speaker 261, a microphone 262, a wireless fidelity (Wi-Fi) module 270, a processor 280, a Bluetooth component 281, and a power supply 290 and other components.

[0055] The RF circuit 210 can be used to receive and send signals during the process of sending and receiving information or making calls. It can receive downlink data from the base station and hand it over to the processor 280 for processing; it can send uplink data to the base station. The memory 220 can be used to store software programs and data, an operating system that enables the display device 200 to run, and code for executing the method of the embodiment of the present application. The processor 280 executes various functions and data processing of the display device 200 by running the software programs or data stored in the memory 220.

[0056] The display unit 230 can be used to receive input digital or character information and generate signal input related to the user settings and function control of the display device 200. Specifically, the display unit 230 may include a touch screen 231 arranged on the front of the display device 200, which can collect the user's touch operations on or near it, such as clicking a button, dragging a scroll box, etc. The display unit 230 can also be used to display information input by the user or information provided to the user and a graphical user interface (GUI) of various menus of the display device 200. Specifically, the display unit 230 may include a display screen 232 arranged on the front of the display device 200. Among them, the display screen 232 can be configured in the form of a liquid crystal display, a light emitting diode, etc. The display unit 230 can be used to display various graphical user interfaces in the present application.

[0057] The camera 240 can be used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element converts the optical signal into an electrical signal, which is then transmitted to the processor 280 for conversion into a digital image signal. The display device 200 may also include at least one sensor 250, such as an acceleration sensor 251, a distance sensor 252, a fingerprint sensor 253, and a temperature sensor 254.

[0058] The audio circuit 260, the speaker 261, and the microphone 262 can provide an audio interface between the user and the display device 200. The audio circuit 260 can transmit the electrical signal converted from the received audio data to the speaker 261, which is converted into a sound signal for output. The display device 200 can also be configured with a volume button for adjusting the volume of the sound signal. On the other hand, the microphone 262 converts the collected sound signal into an electrical signal, which is received by the audio circuit 260 and converted into audio data, and then the audio data is output to the RF circuit 210 to be sent to, for example, another display device, or the audio data is output to the memory 220 for further processing. In this application, the microphone 262 can obtain the user's voice.

[0059] The display device 200 also includes a Wi-Fi module 270, which provides users with wireless broadband Internet access. The processor 280 is the control center of the display device 200, which uses various interfaces and lines to connect the various parts of the entire display device, and executes various functions and processes data of the display device 200 by running or executing software programs stored in the memory 220, and calling data stored in the memory 220. In some embodiments, the processor 280 may include one or more processing units. In the present application, the processor 280 can run an operating system, an application program, a user interface display and touch response, and a processing method of an embodiment of the present application. In addition, the processor 280 is coupled to the display unit 230.

[0060] The Bluetooth component 281 is used to exchange information with other Bluetooth devices having Bluetooth components through the Bluetooth protocol. The display device 200 also includes a power supply 290 (such as a battery) for supplying power to each component. The power supply can be logically connected to the processor 280 through a power management system, so that the power management system can manage functions such as charging, discharging, and power consumption.

[0061] In some embodiments, from a software perspective, the system of the display device may include a kernel, a command parser (shell), a file system, and an application. The kernel, shell, and file system together form the basic operating system structure, which allows users to manage files, run programs, and use the system. After power-on, the kernel starts, activates the kernel space, abstracts the hardware, initializes hardware parameters, etc., runs and maintains virtual memory, schedulers, signals, and inter-process communication (IPC). After the kernel starts, the shell and user applications are loaded. After startup, the application is compiled into machine code to form a process, see Figure 4 .

[0062] It should be noted that the hardware configuration of different display devices may be different, so the above Figure 1-Figure 4 This is just an exemplary description.

[0063] In some embodiments, in order to realize the audio or video playback function, the terminal device is deployed with a media player (Media Player) 500. From the perspective of logical functions, the media player 500 may include a playback control module 310, a track selector 320, a decoder 330, a decoding buffer 340 and a rendering module 350. Figure 5 As shown. Among them, the function of the media player 500 can be implemented by one or more processors described above. The media player 500 itself has a demultiplexing (demux) function, which can separate the audio data stream in the audio and video file and save multiple audio data in the track queue. The broadcast control module 310 is used to receive the signal of switching the audio track sent by the user through the control device 100, and switch the track queue according to the received signal of switching the audio track. The track selector 320 is used to receive the instruction of switching the audio track queue sent by the broadcast control module 310, switch to the track queue corresponding to the audio track set by the user according to the instruction, associate the track queue with the decoder 330, and send the audio data in the track queue to the decoder 330. The decoder 330 decodes the received audio data and sends the decoded audio data to the decoding buffer 340. The rendering module 350 can obtain audio data from the decoding buffer 340, render the obtained audio data and output it.

[0064] In some embodiments, when the data played by the media player includes both audio data and video data, after determining the playback audio track queue, the playback clock of the media player can be associated with the playback audio track queue, that is, when playing video data, the playback progress of the playback audio track queue is used as a reference, so that the played audio data is synchronized with the video data.

[0065] It should be noted that Figure 5 The logical structure of the media player shown is only an example and is not limited to this in the embodiments of the present application.

[0066] The solution provided by the present application is described in detail below in conjunction with the embodiments. For the convenience of description, the identification of each component of the terminal device will not be exemplified in the following.

[0067] The present application embodiment provides a method for switching audio tracks. Figure 6 The flow of the audio track switching method is exemplarily shown, and the method can be executed by a terminal device, or implemented by a media player in the terminal device, or by a playback control module in the media player. The method is implemented by a media player in the terminal device as an example for description.

[0068] 601, the media player receives a track switching instruction, the track switching instruction indicating track switching. In the embodiment of the present application, the currently played track is the first track, and the track after switching is the second track. That is, the track switching instruction is used to instruct to switch the currently played audio data from the first track to the second track.

[0069] In some embodiments, the demultiplexing function of the media player can separate the audio data stream from the audio and video file, and then save the multiple audio data. As an example, the audio data separated by the media player can be output through three tracks. The data of the three tracks correspond to audio data of different playback language types, such as the data of track 1 is the audio data stream of the playback language type of English, and the data of tracks 2 and 3 are the audio data streams of the playback language types of Chinese and French, respectively. Figure 7 When a track switching instruction is received, the track switching instruction indicates that the language type of the current playback is switched from English to Chinese, and the media player controls the currently played audio data to switch from the data of track 1 to the data of track 2. Exemplarily, after receiving the track switching instruction, the playback module in the media player can control the track selector to switch the data of the selected track 1 to the data of track 2.

[0070] In some scenarios, when a media player receives media data, the media data may include audio data of multiple tracks, each track corresponds to an audio data stream, and an audio data stream may include multiple segments of audio data. The data of different tracks may be audio data streams of different language types, such as Chinese, English, or French, etc.; or the data of different tracks may also be audio data streams of different sound qualities, such as Dolby sound, AAC, stereo, etc. Users can switch tracks through the media player to meet different needs.

[0071] In some scenarios, the audio data played may be recorded audio data or live audio data, which is not limited here.

[0072] 602, the media player obtains a maximum timestamp of the data of the first audio track included in the decoding buffer. The decoding buffer includes decoded and unrendered audio data. In order to distinguish it from the maximum timestamp described later, the maximum timestamp of the data of the first audio track included in the decoding buffer is referred to as the first maximum timestamp.

[0073] In some scenarios, the audio data also includes timestamp information, which is parsed and generated from the audio data stream in the demultiplexing stage of the media player. Exemplarily, the data of the output audio track can be multiple segments of audio data, and the timestamp of each segment of audio data can be calculated based on the time base of the audio data information of the audio track.

[0074] In some embodiments, after receiving the audio data of the first audio track, the decoder decodes the audio data of the first audio track, and after completing the decoding, sends the decoded audio data to the decoding buffer. The decoder can obtain the timestamp of the decoded audio data during the decoding process. The broadcast control module can obtain the timestamp of the audio data that has been decoded from the decoder. When a track switching instruction is received, the broadcast control module can obtain the first maximum timestamp of the decoded data of the first audio track in the decoding buffer from the decoder. For example, the data of track 1 is decoded by the decoder and sent to the decoding buffer, and the decoding buffer can cache multiple decoded audio data of track 1. When the track switching instruction is received, the decoder has just completed the decoding of the audio data before the T1 timestamp in the data of track 1 and has sent it to the decoding buffer. The broadcast control module can obtain the first maximum timestamp T1 of the audio data of track 1 that has been decoded included in the decoding buffer, see Figure 7 .

[0075] 603 , the media player decodes the audio data in the second audio track whose timestamp is greater than the first maximum timestamp according to the first maximum timestamp, and saves the decoded data into a decoding buffer.

[0076] In some embodiments, the media player synchronizes the data of the inactive audio tracks with the audio data being rendered by the rendering module in real time. An inactive audio track refers to an audio track for which audio data is not currently being decoded. For example, the media player receives the data of three audio tracks, namely Track 1 - 3. If the audio data of Track 1 is currently being played, then the other audio tracks (Track 2 and Track 3) except Track 1 are inactive audio tracks. It can be understood that when the media player synchronizes the data of the inactive audio tracks with the audio data being rendered by the rendering module, it can synchronize the data of the received inactive audio tracks according to the timestamps of the audio data rendered by the rendering module. For example, if the timestamp of the audio data rendered by the rendering module is T1, then the media player discards the audio data in the data of the inactive audio tracks with timestamps less than or equal to T1. Based on this, the minimum timestamp of the audio data of the second audio track is less than the first maximum timestamp. When the currently played audio data switches to the data of the second audio track, it is necessary to discard the audio data in the second audio track with timestamps less than or equal to the first maximum timestamp, and decode and save the audio data in the second audio track with timestamps greater than the first maximum timestamp to the decoding buffer. For example, when the currently played audio data switches from the data of Track 1 to the data of Track 2, the maximum timestamp of the audio data that has been decoded for Track 1 is T1. At this time, the minimum timestamp of the data of Track 2 is T2, and T2 < T1. Therefore, when switching the currently played audio data to the data of Track 2, the audio data in the T2 - T1 time period of Track 2 is discarded, the audio data in Track 2 with timestamps greater than T1 is output, and it is decoded and saved to the decoding buffer.

[0077] In some embodiments, when the currently played audio data needs to switch from the data of the first audio track to the data of the second audio track, the playback control module sends a switching instruction to the audio track selector. The switching instruction instructs the audio track selector to stop sending the data of the first audio track to the decoder and send the data of the second audio track to the decoder. When the audio track selector sends the audio data in the second audio track with timestamps greater than the first timestamp to the decoder, and the current decoder supports decoding the data of the second audio track, the decoder decodes the received data of the second audio track and saves it to the decoding buffer.

[0078] In some other embodiments, after obtaining the first maximum timestamp, the playback control module can carry the first maximum timestamp in the switching instruction, so that the audio track selector switches the audio tracks according to the first maximum timestamp. For example, discard the audio data in the second audio track earlier than the first maximum timestamp, and send the audio data in the first audio track earlier than or equal to the first maximum timestamp to the decoder.

[0079] In some embodiments, when the media data includes both audio data and video data, the media player receives a video stream, and when receiving a track switching instruction, when rendering the audio data with the minimum timestamp in the decoding buffer, the video data corresponding to the audio data with the minimum timestamp included in the video stream is synchronously rendered. For example, the audio data currently being played is the data of track 1, the timestamp of the audio data decoded in track 1 is T1, and the timestamp of the audio data being rendered is T2. The media player uses the timestamp of the audio data output by the rendering module as the reference clock, and also renders the video data with a timestamp of T2. When the track switching instruction is received, the played track data is switched from the data of track 1 to the data of track 2. At this time, the rendering module reads the audio data with the minimum timestamp T5 decoded in the data of track 1 stored in the decoding buffer from the decoding buffer, and renders it. At the same time, the rendering module of the video stream synchronously renders the video data with a timestamp of T5.

[0080] Based on the above solution, when the audio track is switched, the decoded data of the currently playing audio track is no longer discarded, that is, the decoded audio data in the decoding buffer is no longer discarded, but the audio data in the switched audio track with a timestamp greater than the timestamp of the decoded audio data is sent to the decoder for decoding, and the value is cached in the decoding buffer to achieve continuous data output. In addition, when switching the playing audio track, the video data does not need to be paused or frame-chased, which improves the user's viewing experience.

[0081] In some embodiments, the data of different audio tracks can be exactly the same or completely different. For example, the encoding formats of the audio data of different audio tracks are different, such as the encoding formats can be MP3, AAC, WMA, AC-3, APE, FLAC, etc.; or the data encapsulation formats of different audio tracks are different, such as: FLV, MP4, MP3, etc.; or the audio data sampling rate, sampling bit depth, number of channels, etc., transmission protocols like file, http (Hypertext Transfer Protocol), rtmp, etc. Therefore, when decoding the audio data of an audio track, it is necessary to consider whether the decoder can decode the input audio data. In some embodiments, the currently played audio data is the data of the first audio track, and the decoder decodes the data of the first audio track. When the currently played audio data is switched to the data of the second audio track, if the current decoder can decode the audio data of the second audio track, the received audio data of the second audio track is directly decoded and saved to the decoding buffer. In some other embodiments, when it is determined that the current decoder cannot decode the audio data of the second audio track, the current decoder is released, and a new decoder is created so that the created new decoder can decode the audio data of the second audio track. For the sake of easy distinction, the released decoder is called the first decoder, and the created new decoder is called the second decoder. Among them, releasing the current decoder and creating a new decoder can be understood as creating the second decoder and disconnecting the association relationship between the first decoder and the decoding buffer and the audio track selector, and associating the second decoder with the decoding buffer and the audio track selector. The audio track selector sends the audio data of the second audio to the second decoder, so that the second decoder decodes the audio data in the data of the second audio track whose timestamp is greater than the first maximum timestamp and saves it to the decoding buffer.

[0082] In some embodiments, after the switching is completed, the second maximum timestamp of the rendered audio data is obtained, and the data of the first audio track and the rendered audio data are synchronized according to the second maximum timestamp, so that the maximum timestamp of the discarded data of the first audio track is the same as the timestamp of the rendered audio data. For example, after the currently played audio data is switched from the data of audio track 1 to the data of audio track 2, the played audio data is the data of audio track 2. After that, when the maximum timestamp T3 of the rendered audio data of audio track 2 is obtained, the minimum timestamp of the data of audio track 1 and audio track 3 is T4, see Figure 8 . When T4 < T3, the audio data in the data of audio track 1 and audio track 3 whose timestamp is less than T3 is discarded, so that the maximum timestamp of the discarded data of audio track 1 and audio track 3 is the same as the timestamp of the rendered audio data of audio track 2.

[0083] In some embodiments, the audio data played may be audio data of any one of the multiple audio tracks. In some scenarios, when the media player starts playing audio data, any audio track may be set as the default playback audio track and the data of the audio track may be output, or the first audio track in the audio track may be set as the default playback audio track and the data of the audio track may be output, which is not limited in the present application.

[0084] In some embodiments, the display device is used to play the video stream and the audio data corresponding to the video stream, and the audio data currently being played is the audio track data where the stereo audio data is located. The audio data is output through the audio output interface in the display device, and the audio data can be played through the speaker in the display device, and the video stream is output through the display screen on the display device. The user can switch the audio data being played through the control device. For example, the user can switch the audio data currently being played from stereo to Dolby sound through the remote control, see Fig. 9 . When the user switches the stereo to Dolby sound by clicking a button on the remote control, the media player receives the track switching instruction and switches the currently playing track from the track where the stereo audio data is located to the track where the Dolby sound audio data is located. The media player obtains the maximum timestamp of the stereo audio data included in the decoding buffer, and decodes the audio data with a timestamp greater than the maximum timestamp in the Dolby sound audio data according to the maximum timestamp and saves it to the decoding buffer. When the encoding format, sampling rate and sampling bit number of the stereo audio data and the Dolby sound audio data are different, the decoder that decodes the stereo audio data cannot decode the Dolby sound audio data, releases the current decoder, and creates a new decoder so that the created new decoder can decode the Dolby sound audio data. The new decoder receives the audio data with a timestamp greater than the maximum timestamp in the Dolby sound audio data, decodes it and saves it to the decoding buffer. The rendering module reads the decoded audio data from the decoding buffer and renders it. The rendered audio data can be sent to the speaker through the audio output interface and played through the speaker. The solution provided in the embodiment of the present application enables the playback of audio data to be continuous in time when switching audio tracks, so the video stream played on the display screen does not need to be paused or frame-tracked, which can reduce the complexity of video processing and improve user experience.

[0085] Based on the same technical concept, see Fig.10 As shown, the embodiment of the present application provides a display device 900, which can execute each step in the above method. To avoid repetition, the control device 900 includes a communication interface 901 and a processor 902.

[0086] The communication interface 901 is used to receive an audio track switching instruction, where the audio track switching instruction indicates to switch the currently played audio data from the data of the first audio track to the data of the second audio track.

[0087] The processor 902 is configured to obtain a first maximum timestamp of data of a first audio track included in a decoding buffer, wherein the decoding buffer includes decoded and unrendered audio data. The processor 902 is further configured to decode, according to the first maximum timestamp, audio data in the data of the second audio track whose timestamp is greater than the first maximum timestamp, and save the decoded audio data in the decoding buffer, and render the audio data in the decoding buffer.

[0088] In some embodiments, the device 900 further includes an audio output interface 903 and a display screen 904, wherein the audio output interface 903 is used to output the rendered audio data, and the display screen 904 is used to display video data corresponding to the audio data.

[0089] In some embodiments, the device 900 may also include a speaker ( Fig. 9 The speaker may be used to play the audio data output by the audio output interface 903. In other embodiments, the speaker may be disposed outside the device 900, and the device 900 is electrically connected to the speaker via the audio data output interface 903.

[0090] In some embodiments, the communication interface 901 is further configured to receive a video stream. The processor 902, upon receiving the audio track switching instruction, is further configured to synchronously render the video data corresponding to the audio data with the minimum timestamp included in the video stream when rendering the audio data with the minimum timestamp in the decoding buffer.

[0091] In some embodiments, the processor 902 is further configured to: create a second decoder when it is determined that the first decoder does not support decoding of the data of the second audio track; the first decoder supports decoding of the data of the first audio track. When the processor 902 decodes the audio data in the data of the second audio track whose timestamp is greater than the first maximum timestamp, it is specifically configured to: decode the audio data in the data of the second audio track whose timestamp is greater than the first maximum timestamp through the decoder.

[0092] In some embodiments, the processor 902 is further used to: after completing the switching, obtain a second maximum timestamp of the rendered audio data; and synchronize the data of the first audio track with the rendered audio data according to the second maximum timestamp.

[0093] Based on the same technical concept, such as Fig.11As shown, an embodiment of the present application provides a device 1000 for switching audio tracks, and the control device 1000 includes: a memory 1001 and a processor 1002.

[0094] Memory 1001, used for storing program instructions;

[0095] Processor 1002 is used to call the program instructions stored in the memory and execute the above-mentioned audio track switching method according to the obtained program.

[0096] In the embodiments of the present application, the processor may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and may implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application may be directly embodied as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0097] As a non-volatile computer-readable storage medium, the memory can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory can include at least one type of storage medium, for example, it can include flash memory, hard disk, multimedia card, card-type memory, random access memory (Random Access Memory, RAM), static random access memory (Static Random Access Memory, SRAM), programmable read-only memory (Programmable Read Only Memory, PROM), read-only memory (Read Only Memory, ROM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), magnetic memory, disk, CD, etc. The memory is any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0098] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0099] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0100] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0101] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0102] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for switching audio tracks, characterized in that: include: Receive an audio track switching instruction, wherein the audio track switching instruction instructs to switch the currently played audio data from the first audio track to the second audio track; Obtaining a first maximum timestamp of data of a first audio track included in a decoding buffer, wherein the decoding buffer includes decoded and unrendered audio data, and the audio data in the decoding buffer is sent to the decoding buffer after the decoder decodes the data of the first audio track; When it is determined that the first decoder does not support decoding of the data of the second audio track, creating a second decoder; the first decoder supports decoding of the data of the first audio track; According to the first maximum timestamp, the audio data in the data of the second audio track whose timestamp is greater than the first maximum timestamp is decoded by the second decoder and then saved in the decoding buffer.

2. The method according to claim 1, characterized in that The method further comprises: Receive video stream; When receiving the audio track switching instruction, when rendering the audio data with the minimum timestamp in the decoding buffer, the video data corresponding to the audio data with the minimum timestamp included in the video stream is synchronously rendered.

3. The method according to claim 1 or 2, characterized in that The method further comprises: After the switching is completed, the second maximum timestamp of the rendered audio data is obtained; The data of the first audio track is synchronized with the rendered audio data according to the second maximum timestamp.

4. The method according to claim 1 or 2, characterized in that: The method further comprises: The audio data with a timestamp smaller than the first maximum timestamp in the data of the first audio track is discarded.

5. A display device, characterized in that: Includes a communication interface, a processor, an audio output interface and a display screen; The communication interface is used to receive an audio track switching instruction, wherein the audio track switching instruction instructs to switch the currently played audio data from the data of the first audio track to the data of the second audio track; The processor is configured to obtain a first maximum timestamp of data of a first audio track included in a decoding buffer, the decoding buffer including decoded and unrendered audio data, the audio data in the decoding buffer being sent to the decoding buffer after the decoder decodes the audio data; when it is determined that the first decoder does not support decoding the data of the second audio track, create a second decoder; the first decoder supports decoding the data of the first audio track; according to the first maximum timestamp, decode the audio data in the data of the second audio track whose timestamp is greater than the first maximum timestamp through the second decoder, save the decoded audio data to the decoding buffer, and render the audio data in the decoding buffer; The audio output interface is used to output the audio data after rendering; The display screen is used to display video data corresponding to the audio data.

6. The display device according to claim 5, characterized in that The communication interface is also used to receive a video stream; The processor, upon receiving the audio track switching instruction, is further configured to synchronously render the video data corresponding to the audio data with the minimum timestamp included in the video stream when rendering the audio data with the minimum timestamp in the decoding buffer.

7. The display device according to claim 5 or 6, characterized in that: The processor is further configured to: After the switching is completed, the second maximum timestamp of the rendered audio data is obtained; The data of the first audio track is synchronized with the rendered audio data according to the second maximum timestamp.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 4.

Images