Playing device and method for automatically switching audio and video mode

Abstract

This application discloses a method for automatically switching playback devices and audio / video modes, including: a controller and an audio / video transmission interface and a speaker connected via communication; receiving an input search command for searching media assets; when connected to a display device, sending a first media asset request containing keywords, receiving first audio / video data fed back by a server based on the first media asset request, and sending the first audio / video data to the display device through the audio / video transmission interface; when the audio / video transmission interface is detected to be disconnected from the display device, sending a second media asset request containing keywords; receiving second audio data fed back by the server based on the second media asset request, and controlling the speaker to play the second audio data. When the playback device is disconnected from the display device, this application allows the playback device to continue searching for audio with the same name to continue playing the video the user is currently watching, or in other words, to continue playing the audio data from the current video, thus improving the user experience.

Description

Technical Field

[0001] This application relates to the field of playback technology, and in particular to a playback device and a method for automatically switching audio and video modes. Background Technology

[0002] A smart TV box is a device that connects a television to the internet. It has a built-in operating system that allows users to install various applications. Currently, users can connect their smart TV boxes to monitors via HDMI to watch media on the display.

[0003] Typically, smart TV boxes feature OTT (Over-The-Top TV) functionality, also known as OTT mode. In OTT mode, after connecting the smart TV box to a monitor via HDMI, users can enjoy video services based on the open internet. Users can watch online videos such as live HD TV, dramas, movies, and variety shows. However, if the connection between the smart TV box and the monitor is lost, the smart box cannot play media, resulting in a poor user experience. Summary of the Invention

[0004] This application provides a method for automatically switching playback devices and audio / video modes to solve the technical problem in the prior art where a smart TV box cannot play media resources to the user after the connection between the smart TV box and the display is lost, resulting in a poor user experience.

[0005] To address the aforementioned technical problems, some embodiments of this application disclose the following technical solutions:

[0006] In a first aspect, some embodiments of this application disclose a playback device, the playback device comprising:

[0007] The audio / video transmission interface is configured to output audio / video data to an external display device;

[0008] speaker;

[0009] The controller is communicatively connected to the audio / video transmission interface and the speaker, and is configured to:

[0010] Receive input search instructions for searching media assets;

[0011] When the display device is connected to the audio / video transmission interface, a first media resource request containing a keyword is sent, wherein the keyword is obtained by parsing the search instruction, and the first media resource request is used to enable the server to find the first audio / video data based on the keyword;

[0012] The system receives the first audio and video data from the server and sends the first audio and video data to the display device through the audio and video transmission interface, so that the display device can display the first audio and video data.

[0013] When the audio / video transmission interface is detected to be disconnected from the display device, a second media asset request containing the keyword is sent, wherein the second media asset request is used to enable the server to search for second audio data based on the keyword, but not video data;

[0014] The system receives the second audio data from the server and controls the speaker to play the second audio data.

[0015] Secondly, some embodiments of this application disclose a playback device, the playback device comprising:

[0016] The audio / video transmission interface is configured to output audio / video data to an external display device;

[0017] speaker;

[0018] The controller is communicatively connected to the audio / video transmission interface and the speaker, and is configured to:

[0019] Receive input voice commands for searching media assets;

[0020] When the display device is connected to the audio / video transmission interface, a first media asset request containing a keyword is sent; first audio / video data is received from the server based on the first media asset request, and the first audio / video data is sent to the display device through the audio / video transmission interface so that the display device displays the first audio / video data; wherein, the keyword is obtained by parsing the voice command, and the first media asset request is used to enable the server to find the first audio / video data based on the keyword;

[0021] When the audio / video transmission interface is disconnected from the display device, a second media asset request containing the keyword is sent; and after receiving the second audio data searched by the server according to the second media asset request, the speaker is controlled to play the second audio data; wherein, the second media asset request is used to enable the server to search for the second audio data according to the keyword, without searching for video data.

[0022] Thirdly, some embodiments of this application disclose a method for automatically switching audio and video modes, the method comprising:

[0023] Receive input search instructions for searching media assets;

[0024] When a display device is connected to the audio / video transmission interface, a first media resource request containing a keyword is sent, wherein the keyword is obtained by parsing the search instruction, and the first media resource request is used to enable the server to find the first audio / video data based on the keyword;

[0025] The system receives the first audio and video data from the server and sends the first audio and video data to the display device through the audio and video transmission interface, so that the display device can display the first audio and video data.

[0026] When the audio / video transmission interface is detected to be disconnected from the display device, a second media asset request containing the keyword is sent, wherein the second media asset request is used to enable the server to search for second audio data based on the keyword, but not video data;

[0027] The system receives the second audio data from the server and controls the speaker to play the second audio data.

[0028] Compared with the prior art, the beneficial effects of this application are as follows:

[0029] This application provides a method for automatically switching playback devices and audio / video modes. The playback device can be connected to a display device via an audio / video transmission interface. Users can search for media assets through the playback device. Based on the user's search, the playback device requests corresponding first audio / video data from a server for playback and display on the display device. After the playback device disconnects from the display device, it can output audio data through a speaker. If the user is watching the first audio / video data being played by the playback device on the display device, and the connection between the playback device and the display device is broken, the playback device sends a media asset request containing the same keyword to the server to obtain second audio data, which is then played through the speaker. In this application, even after the playback device disconnects from the display device, it can still search for audio data with the same name to continue playing the currently viewed video / film media asset, thus improving the user experience.

[0030] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0031] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1The diagram illustrates an operational scenario between a playback device and a control device according to some embodiments;

[0033] Figure 2 The diagram illustrates, by way of example, a hardware configuration block diagram of a playback device 200 according to some embodiments;

[0034] Figure 3 The diagram illustrates a flowchart of a method for automatically switching audio and video modes according to some embodiments;

[0035] Figure 4 The diagram illustrates, for example, the display effect of search media resources according to some embodiments;

[0036] Figure 5 The diagram above exemplifies another display effect of a search media asset according to some embodiments;

[0037] Figure 6 The diagram illustrates a timing diagram of a method for automatically switching audio and video modes according to some embodiments. Detailed Implementation

[0038] To make the objectives and implementation methods of this application clearer, the exemplary implementation methods of this application will be clearly and completely described below with reference to the accompanying drawings of the exemplary embodiments of this application. Obviously, the exemplary embodiments described are only some embodiments of this application, and not all embodiments.

[0039] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0040] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0041] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0042] The term "module" refers to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0043] Figure 1 The diagram illustrates an operational scenario between a playback device and a control device according to some embodiments. Figure 1 As shown, the playback device 200 can connect to the server 400 via the Internet and can also connect to the display device 500. Users can operate the display device 500 via the smart device 300 or the control device 100 to control the playback device 200.

[0044] In some embodiments, the control device 100 may be a remote control. Communication between the remote control and the display device includes infrared protocol communication, Bluetooth protocol communication, and other short-range communication methods, controlling the display device 500 wirelessly or via wired means. Users can control the display device 500 by inputting user commands through buttons on the remote control, voice input, control panel input, etc.

[0045] In some embodiments, a smart device 300 (such as a mobile terminal, tablet computer, computer, laptop computer, etc.) may also be used to control the display device 500. For example, an application running on the smart device may be used to control the display device 500.

[0046] In some embodiments, the display device 500 can also be controlled in ways other than the control device 100 and the smart device 300. For example, it can be controlled by directly receiving the user's voice commands through a module configured inside the display device 500 for acquiring voice commands, or it can be controlled by receiving the user's voice commands through a voice control device set outside the display device 500.

[0047] In some embodiments, the playback device 200 communicates with the server 400. The playback device 200 may communicate via a local area network (LAN), wireless local area network (WLAN), and other networks. The server 400 may provide various content and interactive features to the playback device 200. The server 400 may be a cluster or multiple clusters, and may include one or more types of servers.

[0048] Figure 2 The diagram illustrates, by way of example, a hardware configuration block diagram of a playback device 200 according to some embodiments.

[0049] In some embodiments, the playback device 200 includes at least one of a controller 210, a tuner / demodulator 220, an audio / video transmission interface 230, an audio output interface 240, and a power supply 250.

[0050] In some embodiments, the tuner 220 receives broadcast television signals via wired or wireless reception and demodulates audio and video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.

[0051] In some embodiments, the playback device 200 can be connected to the display device 500 via the audio / video transmission interface 230. The audio / video output interface can be an HDMI interface (High-Definition Multimedia Interface) or a TYPE-C interface (USB Type-C).

[0052] In some embodiments, the audio data requested by the playback device 200, after being processed by the audio processor, can be output through the audio output interface 240. The audio output interface can be connected to a speaker, which can be built into the playback device 200 or located outside the playback device 200.

[0053] In some embodiments, under the control of the controller 210, power input from an external power source is used to provide power to the playback device 200. The power supply 250 may include a built-in power circuit installed inside the playback device 200, or it may be an external power source installed in the playback device 200, providing an external power interface within the playback device 200.

[0054] In some embodiments, the controller 210 controls the operation of the playback device and responds to user operations through various software control programs stored in memory. The controller 210 controls the overall operation of the playback device 200. For example, in response to receiving a user command input by the user through the display device 500, the controller 210 can perform an operation related to the object selected by the user command.

[0055] In some embodiments, the controller 210 includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), RAM (random access memory), ROM (read-only memory), a first to an nth interface for input / output, a communication bus, etc.

[0056] A CPU (CPU) processor is used to execute operating system and application instructions stored in memory, as well as various interactive instructions received from external input, to execute various applications, data, and content, ultimately for the display and playback of various audio and video content. A CPU processor can include multiple processors, such as a main processor and one or more sub-processors.

[0057] In some embodiments, a graphics processor is used to generate various graphical objects, such as icons, operation menus, and graphics displayed based on user input commands. The graphics processor includes an arithmetic logic unit (ALU) that performs calculations based on various user-input interactive commands and displays various objects according to display attributes; it also includes a renderer that renders the various objects obtained from the ALU, and the rendered objects are used to display on a monitor.

[0058] In some embodiments, the video processor is configured to receive external video signals and perform video processing such as decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, and image synthesis according to the standard encoding and decoding protocol of the input signals, so as to obtain signals that can be directly displayed or played on the display device 500.

[0059] In some embodiments, the audio processor is configured to receive external audio signals, and according to the standard codec protocol of the input signals, perform decompression and decoding, as well as noise reduction, digital-to-analog conversion, and amplification processing, to obtain a sound signal that can be played in the audio output interface 240.

[0060] In some embodiments, the system of the playback device 200 may include a kernel, a command interpreter (shell), a file system, and applications. The kernel, shell, and file system together form the basic operating system structure, allowing users to manage files, run programs, and use the system. Upon power-up, the kernel starts, activates the kernel space, abstracts hardware, initializes hardware parameters, and runs and maintains virtual memory, the scheduler, signals, and inter-process communication (IPC). After the kernel starts, the shell and user applications are loaded. The applications are compiled into machine code after startup, forming a process.

[0061] In some embodiments, the hardware or software architecture can be based on the description in the above embodiments, and in some embodiments it can be based on other similar hardware or software architectures, as long as the technical solution of this application can be implemented.

[0062] Based on the aforementioned playback device 200, depending on whether a display device 500 is connected, it can be divided into OTT mode and speaker mode. OTT mode refers to outputting audio and video data to the external display device 500 through the audio and video transmission interface. In this mode, both video and audio data can be played, and it can also be called video mode.

[0063] In some embodiments, after the playback device 200 is started, it detects whether a display device 500 is connected.

[0064] In some embodiments, users can search for media assets via remote control commands or voice commands.

[0065] In some embodiments, the playback device is connected to the display device 500. After the playback device 200 displays the search results on the display device 500 in the form of a UI, playback begins after the user clicks to confirm.

[0066] In some embodiments, after the current media asset has finished playing, other related recommended media assets will only be played if the user actively clicks on them. Speaker mode refers to the playback device 200 being disconnected from the display device 500, and only audio media assets can be played; this can also be called audio mode.

[0067] In some embodiments, users can search for media resources via voice commands. For the returned search results, the playback device 200 directly begins playback through the speaker connected to the audio output interface, without user confirmation. After playing the current media resource, the playback device 200 can automatically play other related recommended media resources.

[0068] In some embodiments, audio and video functions are processed using the same audio and video application.

[0069] In some embodiments, the voice command processing function of the playback device 200 can be integrated into the application or can be relatively independent of each application.

[0070] In some embodiments, taking the voice command processing function and each application as relatively independent as an example, after receiving a voice command, the voice command processing function transmits the voice command to the audio / video application. The audio / video application sends different requests to the server 400 based on the voice command and the current connection status of the audio / video transmission interface. For example, when a display device 500 is currently connected, it requests the video data corresponding to the voice command, and after receiving the video data, it displays the image and outputs the audio through the display device 500 based on the video data. When no display device 500 is connected, it requests the audio data corresponding to the voice command, and the playback device receives and processes the audio data and plays it through its speaker.

[0071] In some embodiments, video functionality uses a video application, and audio functionality uses an audio application.

[0072] In some embodiments, taking the voice command processing function and each application as relatively independent as an example, after receiving a voice command, the voice command processing function determines the connection status of the audio / video transmission interface and sends the command to different applications depending on the status. When the audio / video transmission interface is connected to a display device 500, the voice command is sent to the video application, which requests the server 400 based on the voice command. After receiving the video data, the video application displays the image and outputs the audio through the display device 500 based on the video data. When the audio / video transmission interface is not connected to a display device 500, the voice command is sent to the audio application, requesting the corresponding audio data. The playback device receives and processes the audio data and plays it through its speaker.

[0073] In some embodiments, when displaying on a display device 500 connected via an audio / video transmission interface, the application can also be controlled via a remote control, and the interface controlled by the remote control can be refreshed and displayed on the display device 500.

[0074] In some embodiments, voice commands sent by the voice command processing function can be broadcast through the framework layer, and the video / audio application can be configured to respond based on the connection status of the audio / video transmission interface. For example, the video application responds to the voice command when a display device 500 is connected to the audio / video transmission interface, and does not respond otherwise. The audio application does the opposite.

[0075] In some embodiments, the voice command sent by the voice command processing function may be a received voice command or a new voice command generated after processing the received voice command.

[0076] In some embodiments, when playback device 200 receives a first voice command input by the user, such as "play XZ", if a display device 500 is connected at this time, the video application responds to the voice command and sends a request containing keywords to server 400. Server 400 performs a search based on the keywords and returns the search results to the video application. In some embodiments, the video application directly transmits the returned search results to display device 500. Display device 500 has an application corresponding to the video application, which parses the search results and displays / plays them. In some embodiments, the video application parses the search results and sends the parsed search results to display device 500 for display / play. In some embodiments, playback / or switching playback is performed in response to the user's input selection of the search results. In some embodiments, when the search results are transmitted first, the selection command also needs to be transmitted. In some embodiments, when playback device parses the search results, the selection command can be transmitted to the TV and executed by the TV, which then returns the execution result to playback device 200. Alternatively, playback device 200 can execute the command directly and send the executed interface to display device 500 for display.

[0077] In some embodiments, the connection between the playback device 200 and the display device 500 may be interrupted in the event of a hardware malfunction or perceived failure.

[0078] In some embodiments, when the connection between the playback device 200 and the display device 500 is disconnected, the playback device 200 switches from OTT mode to speaker mode. How to handle the media resources the user was previously playing and subsequent playback behavior is a problem that needs to be solved. To improve the user experience, some embodiments of this application provide a method for automatically switching playback devices and audio / video modes.

[0079] The automatic audio / video mode switching process provided in some embodiments of this application will be described below with reference to the accompanying drawings.

[0080] Figure 3 The diagram illustrates a flowchart of a method for automatically switching audio and video modes according to some embodiments. In some embodiments, this application provides a playback device 200 including an audio and video transmission interface, a speaker, and a controller 210. The controller 210 is communicatively connected to the audio and video transmission interface and the speaker, and is configured to perform... Figure 3 The automatic switching process between audio and video modes. Combined with... Figure 3 The automatic switching process for this audio / video mode is as follows:

[0081] In some embodiments, taking a display device 500 connected before switching as an example, the video application before switching plays the media asset selected by the user with the corresponding media asset identifier or searches and displays the media asset based on the media asset identifier entered / selected by the user. Specifically,

[0082] S301: Receives input search instructions for searching media assets.

[0083] In some embodiments, users can search for media assets to watch via the playback device 200. Users can send search commands for media assets to the playback device 200 via remote control commands or voice commands.

[0084] S302: When the display device is connected to the audio / video transmission interface, a first media resource request containing a keyword is sent, wherein the keyword is obtained by parsing the search instruction, and the first media resource request is used to enable the server to search for first audio / video data based on the keyword.

[0085] In some embodiments, when the playback device 200 detects that a display device 500 is connected via an audio / video transmission interface, the playback device 200 enters OTT mode. A video application (video app) can be installed on the playback device 200, allowing for UI interaction with the user through the display on the display device 500. For example, the user can select a video application control displayed on the display using a remote control to launch the video application. When the playback device 200 is in OTT mode, it can accept search commands for media resources input by the user through the display device 500. The playback device 200 parses the search command to obtain keywords and sends a first media resource request containing the keywords to the server 400. Here, the requested media resource is typically a video resource. It should be noted that the display device 500 here refers not only to a television but also to a computer, tablet computer, etc., or simply a monitor.

[0086] Figure 4 The diagram illustrates, for example, the display effect of search media resources according to some embodiments. Figure 4 As shown, the user selects a search control on the display device 500 via a remote control. The playback device 200 receives the user's selection of the search control and displays a search interface on the display device 500, which includes a virtual keyboard control. The user can select each letter in the virtual keyboard control sequentially based on the first letter or full spelling of the title of the media asset they want to search for. The playback device 200 receives the user's selection of the virtual keyboard control and determines the target search term. Here, the target search term can be the title of the media asset, and the keyword can be the target search term. That is, the playback device 200 sets the title of the media asset to be searched in the first media asset request. The playback device 200 sends a first media asset request containing the title of the media asset to the server 400.

[0087] Figure 5 The image below exemplifies another display effect diagram of search media assets according to some embodiments. For example... Figure 5 As shown, a user can input voice commands to the playback device 200, which contain a target search term. For example, after the user inputs the wake word "Hi! XX", they can input the voice command "I want to watch XXX" to the playback device 200, and the playback device 200 can extract the target search term "XXX". The playback device 200 receives the voice command input by the user containing the target search term. Here, the target search term can be the title of the video or the keyword can be the target search term. That is, the playback device 200 sets the title of the media asset to be searched in the first media asset request. The playback device 200 sends the first media asset request containing the title of the video or the keyword to the server 400.

[0088] For example, when a user wants to watch skit A, they can search for the program through playback device 500. Playback device 200 adds the program name containing skit A to the first media asset request and sends it to server 400, so that server 400 can find the audio and video data of skit A based on the keywords and feed it back to playback device 200.

[0089] In some embodiments, the request for the first audio and video data can be displayed directly based on the user's selection of media asset controls in the interface or other processes.

[0090] S303: Receive the first audio and video data fed back by the server, and send the first audio and video data to the display device through the audio and video transmission interface, so that the display device displays the first audio and video data.

[0091] In some embodiments, after receiving a first media asset request, the server 400 searches for corresponding media asset data in the media asset database using the keywords carried in the first media asset request, and then sends the media asset data back to the playback device 200. Here, the media asset data is generally audio or video data. For ease of description, the media asset data obtained by the server 400 according to the first media asset request is referred to as the first audio or video data.

[0092] In some embodiments, the first audio and video data may further include first audio data or first video data.

[0093] In some embodiments, the first audio / video data refers to data that a video application can play.

[0094] In some embodiments, when the server 400 receives a first media asset request, it can search for the first audio and video data corresponding to the first media asset request, as well as other audio and video data related to the first audio and video data, in order to provide the user with more media asset resources. The server 400 feeds back all the audio and video data related to the first audio and video data that it can find to the playback device 200, so that the user can make a selection. The server 400 collects all the found audio and video data into an audio and video data list and sends the audio and video data list to the playback device 200.

[0095] In some embodiments, the playback device 200 may send a first media asset request via a video application, for example, when a user launches a video application via the display device 500 and selects a media asset control from the video application. In response to the user's selection of the media asset control, the playback device 200 sends a media asset request containing the media asset indicated by the media asset control to the server 400.

[0096] In some embodiments, the playback device 200 may receive, via a video application, a list of audio and video data retrieved by the server 400 according to the first media asset request, wherein the list of audio and video data includes the first audio and video data indicated by the keyword and audio and video data related to the first audio and video data. The playback device 200 displays the list of audio and video data via a display device 500.

[0097] In some embodiments, after browsing the audio and video data list through the display device 500, the user selects a first audio and video data item in the list. The playback device 200 receives the user's selection of the first audio and video data item and controls the video application to play the first audio and video data item. The playback device 200 sends the first audio and video data item to the display device 500 through the audio and video transmission interface, so that the display device 500 displays the first audio and video data item.

[0098] S304: When it is detected that the audio / video transmission interface is disconnected from the display device, a second media asset request containing the keyword is sent, wherein the second media asset request is used to enable the server to search for second audio data based on the keyword, but not video data.

[0099] In some embodiments, when the playback device 200 detects that the audio / video transmission interface is disconnected from the display device 500, the playback device 200 enters speaker mode. An audio application (audio app) may be installed on the playback device 200, and the user can input voice commands to the playback device 200 to open the audio application. For example, after the user inputs the wake word "Hi! XX", and then inputs the voice command "Open XX audio application" to the playback device 200, the playback device 200 will launch the corresponding audio application.

[0100] In some embodiments, after the playback device 200 starts the audio application, it can continue to request media data upon receiving a voice command input from the user. Here, media data typically refers to real-time audio data. For example, if the playback device 200 inputs the voice command "I want to listen to song XX", the playback device 200 will request the data of song XX from the server 400 for playback.

[0101] In some embodiments, a user can disconnect the connection between the playback device 200 and the display device 500 by unplugging the HDMI cable between them.

[0102] In some embodiments, the user can turn off the display of the display device 500 by pressing the "standby" button on the remote control. At this time, the playback device 200 also cannot display video content.

[0103] In some embodiments, a user can disconnect the playback device 200 from the display device 500 by inputting a voice command. For example, after the user inputs the wake-up word "Hi! XX" to the display device 500, they can further input the voice command "Turn off screen" to the display device 500.

[0104] In some embodiments, when playing media assets corresponding to a keyword, after detecting a disconnection from the display device 500, the playback device 200 sends a second media asset request containing the keyword. For example, if a user is watching a skit program (likely a program title) on the display device 500 and wants to listen to it but not watch it, the user will disconnect the playback device 200 from the display device 500. Upon detecting a disconnection from the display device 500, the playback device 200 sends a second media asset request containing the same keyword as the currently viewed program to request audio media assets corresponding to the same keyword. For example, it could send a second media asset request containing the program name / media asset ID of the skit program (likely a program title).

[0105] In some embodiments, after receiving a second media asset request, the server 400 searches for corresponding media asset data in the media asset database using the keywords carried in the second media asset request, and then sends the media asset data back to the playback device 200. Here, since the playback device 200, after being disconnected from the display device 500, cannot output video but only audio, the media asset data is generally audio data, and this audio data has the same name as the audio and video data of the program the user was previously watching. For ease of description, the media asset data obtained by the server 400 according to the second media asset request is referred to as the second audio data, which is the same as the first audio data in the first audio and video data.

[0106] In some embodiments, if the audio data and video are played using the same audio / video application, the audio / video application that plays the media asset data corresponding to the keyword may send a second media asset request.

[0107] In some embodiments, the application playing the first audio / video data is different from the application playing the second audio data. Upon detecting a connection loss, the video application playing the first audio / video data is paused, and a second media asset request is sent via the audio application.

[0108] S305: Receive the second audio data fed back by the server, and control the speaker to play the second audio data.

[0109] In some embodiments, the media asset database may or may not contain second audio data with the same name as the first audio / video data. Therefore, when server 400 searches for second audio data in the media asset database according to the second media asset request, it may find the second audio data or it may not. Server 400 will send the query result based on the second media asset request back to playback device 200. If server 400 finds the second audio data, the query result can be the second audio data.

[0110] In some embodiments, the playback device 200 can control an audio application to receive the second audio data retrieved by the server 400 according to the second media asset request. Before playing the second audio data, the playback device 200 needs to control the video application to stop playing the first audio and video data and mark the current playback progress of the first audio and video data. The playback device 200 switches the playback program to the audio application, and based on the current playback progress of the first audio and video data, the playback device 200 controls the audio application to continue playing the second audio data. For example, if the playback device 200 is watching a skit A at 5 minutes and 35 seconds when it receives the second audio data, then the playback device 200 controls the second audio data to start playing from 5 minutes and 35 seconds.

[0111] In some embodiments, when using the same application, the application itself can record playback progress and switch between different playback data.

[0112] In some embodiments, if the same application is not used, the playback progress of the original application and the management of the foreground and background applications can be passed through other modules.

[0113] In some embodiments, if the second audio data is not received from the server, the video application is reactivated and the video application is controlled to play the first audio data through the speaker, wherein the first audio data is the audio data in the first audio and video data and is the same as the second audio data.

[0114] In some embodiments, if the server 400 does not find the second audio data, the query result may be that the second audio data does not exist. After receiving the query result that the second audio data does not exist, the playback device 200 controls the video application to continue playing the first audio data through the audio output interface. Here, after the playback device 200 disconnects from the display device 500, it does not mean that the playback of skit A stops, but rather that skit A continues to play. In this case, only the audio data of skit A is output through its own speaker, and the video data cannot be fed back to the user. For the user, they can only listen to the program, but cannot watch it.

[0115] In some embodiments, to ensure the continuity of the user's viewing of the program before and after disconnection, upon receiving feedback indicating the absence of second audio data, the playback device 200 plays the first audio data from the first audio and video data. For example, if the video application only transmitted the audio and video data before disconnection, then the video application's parsing module needs to be activated to parse the audio and video data, and the audio data is played through the speaker of the playback device 200, without playing the video data. If the video application parsed the first audio and video data before disconnection, then parsing continues, but the parsed data is not transmitted to the display device 500; instead, the audio data is played through the speaker of the playback device 200, without playing the video data.

[0116] In some embodiments, after the video application finishes playing the first audio data, or when it receives an input control command to play the next track, the playback device 200 switches the playback program to the audio application, so that the playback device 200 can continue to listen to other audio data through the audio application after playing the first audio data.

[0117] If the playback device 200 can find an audio program with the same name as the video program the user is currently watching, the playback device 200 will control the video application to stop playing the video program and, based on the playback progress of the video program, control the audio application to continue playing its corresponding audio program. If the playback device 200 cannot find an audio program with the same name as the video program the user is currently watching, the playback device 200 will continue to control the video application to play the video program, and the audio data in the video program will be output through the built-in speaker of the playback device 200.

[0118] In some embodiments, the playback progress of the audio of the first audio / video data in the video application that has been brought back to the foreground can be detected, and the application can be switched to the audio application when the audio playback is complete, or when a switch input is received.

[0119] In some embodiments, if the audio application finishes playing the second audio data, or if the audio application receives a user-input control command to play the next track while playing the second audio data, or if the video application finishes playing the first audio data and the playback program switches to the audio application, or if the video application receives a user-input control command to play the next track while playing the first audio data and the playback program switches to the audio application, the playback device 200 may also send a third media asset request containing keywords to the server 400. Here, the third media asset request is used to enable the server to expand based on the keywords and perform an audio search based on the expanded keywords. That is, the third media asset request is not used to request the first audio / video data or the second audio data, but to obtain other audio data related to the first audio / video data and the second audio data.

[0120] In some embodiments, when the server 400 receives a third media asset request, it can search for audio data corresponding to the third media asset request. The server 400 then sends the search results back to the display device 500. Here, the search results can be one or multiple; if multiple results are found, they can be a list of all audio data related to previously played media assets.

[0121] In some embodiments, the audio application switched to the foreground can also play system-recommended audio data.

[0122] In some embodiments, the playback device 200 receives the search results obtained by the server 400 based on the third media resource request. If the search results are an audio data list, the playback device 200 can control the audio application to play the audio data in the audio data list sequentially through the audio output interface.

[0123] In some embodiments, since the video application and the audio application are different applications, and the audio application requires a certain amount of time to start, the audio application is started before the playback progress of the first audio / video data in the video application, which has been brought back to the foreground, is completed. This allows the audio application to finish playing as quickly as possible after the video application finishes playing. The advance time can be preset based on empirical values.

[0124] The following description, in conjunction with the accompanying drawings, further illustrates the automatic switching process of audio and video modes in some embodiments.

[0125] Figure 6 The image illustrates a timing diagram of a method for automatically switching audio and video modes according to some embodiments. Figure 6As shown, the playback device 200 has video and audio applications installed, and the server 400 includes a search subsystem and a recommendation subsystem. It should be noted that the search and recommendation subsystems can be integrated into one device or deployed on multiple devices. First, the user connects the playback device 200 and the display device 500 via an HDMI cable to put the playback device into OTT mode. Then, the user starts the video application on the playback device 200 using a remote control and searches for first audio and video data through the video application. The playback device 200 sends a first media asset request containing keywords to the search subsystem. After obtaining the first audio and video data based on the first media asset request, the search subsystem sends a list of audio and video data containing the first audio and video data and related audio and video data back to the video application. After the user selects the specific audio and video data to play, the video application plays the selected first audio and video data. Subsequently, if the user disconnects the HDMI cable between the playback device 200 and the display device 500, the playback device 200 switches to speaker mode. The playback device 200 sends a second media asset request to the search subsystem, wherein the keywords in the second media asset request are the same as those in the first media asset request, and the second media asset request is used to obtain second audio data with the same name as the first audio and video data. The search subsystem returns the search results to the playback device 200.

[0126] When the search results indicate the existence of second audio data, the playback device 200 controls the video application to stop playing the first audio / video data, records the playback progress of the first audio / video data, and switches the playback program to the audio application, which then resumes playing the second audio data based on the playback progress of the first audio / video data. After the second audio data finishes playing, or during its playback, the playback device 200 receives a voice command from the user to play the next track. At this time, the audio application retrieves audio data related to the second audio data through the recommendation subsystem. The recommendation subsystem compiles all audio data related to the second audio data into an audio data list and sends the audio data list back to the audio application. The audio application then plays the audio data sequentially according to the audio data list.

[0127] When the search results indicate that the second audio data does not exist, the playback device 200 controls the video application to continuously play the first audio and video data, primarily through the speaker. After the first audio data finishes playing, or during playback, if the playback device 200 receives a voice command from the user to play the next track, the playback device 200 switches the playback program to the audio application. The audio application retrieves audio data related to the second audio data through a recommendation subsystem. The recommendation subsystem compiles all audio data related to the second audio data into an audio data list and sends this list back to the audio application. The audio application then plays the audio data sequentially according to the audio data list.

[0128] In some embodiments, when the audio / video transmission interface is detected to be reconnected to the display device 500, that is, when the playback device 200 is reconnected to the display device 500, the playback device 200 needs to determine whether the previously played audio data is the audio data referred to by the keyword or other audio data. Here, other audio data can be audio data related to the audio data indicated by the keyword, or audio data re-requested based on the user's search operation.

[0129] In some embodiments, when the currently playing audio data is the first audio data or the second audio data, that is, before the playback device 200 reconnects to the display device 500, the playback device is still playing the media data indicated by the previous keyword. At this time, the playback device 200 controls the video application to continue playing the first audio and video data based on the playback progress of the currently playing audio data, and displays it on the display device.

[0130] Here, when the playback device 200 and display device 500 are disconnected, if the first audio / video data is still being played by a video application, there is no need to switch playback programs; the video application will continue playing the first audio / video data. If the second audio data is already being played by an audio application, the playback device 200 controls the audio application to stop playing the second audio data and records the current playback progress of the second audio data. The playback device 200 then switches the playback program to the video application, reactivates the video application, and, based on the current playback progress of the second audio data, controls the video application to resume playing the first audio / video data.

[0131] In some embodiments, when the currently played audio data is not the first audio data or the second audio data, that is, before the playback device 200 reconnects to the display device 500, the playback device has already finished playing the media data indicated by the previous keyword. At this time, the playback device 200 sends a fourth media data request containing the same keyword as the currently played audio data to the server 400, and uses the fourth media data request to obtain the third audio and video data, wherein the third audio and video data is audio and video data with the same name as the currently played audio data.

[0132] In some embodiments, when the server 400 searches the media asset database according to the fourth media asset request, it may find audio and video data with the same name as the currently playing audio data, or it may not find any. The server 400 will send the query result according to the fourth media asset request back to the playback device 200.

[0133] In some embodiments, when the server 400 receives audio and video data in response to the fourth media asset request, the playback device 200 controls the video application to play the audio and video data and displays it on the display device 500.

[0134] In some embodiments, when no audio or video data is received from the server 400 in accordance with the fourth media asset request, the playback device 200 controls the video application to display the main interface of the video application through the display device 500, so that the user can browse and search for other movies and TV shows he or she wants to watch through the main interface of the video application.

[0135] Based on the same inventive concept as the aforementioned playback device, some embodiments of this application also provide a playback device, comprising: an audio / video transmission interface configured to output audio / video data to an external display device; a speaker; and a controller communicatively connected to the audio / video transmission interface and the speaker, the controller being configured to receive input voice commands for searching media assets. When the display device 500 is connected to the audio / video transmission interface, the controller 210 sends a first media asset request containing keywords. The controller 210 receives first audio / video data fed back by the server 400 according to the first media asset request, and sends the first audio / video data to the display device 500 through the audio / video transmission interface, so that the display device 500 displays the first audio / video data; wherein, the keywords are obtained by parsing the voice commands, and the first media asset request is used to enable the server to search for the first audio / video data based on the keywords. When the audio / video transmission interface is disconnected from the display device 500, the controller 210 sends a second media data request containing the keyword; and after receiving the second audio data searched by the server 400 according to the second media data request, controls the speaker to play the second audio data; wherein, the second media data request is used to enable the server to search for the second audio data according to the keyword, without searching for video data.

[0136] The playback device described in this application, after being disconnected from the display device, can continue to search for audio data with the same name to continue playing the audio data in the current film and television media asset being viewed by the user, thereby improving the user experience.

[0137] Based on the same inventive concept as the aforementioned playback device, some embodiments of this application also provide a method for automatically switching audio and video modes. The method includes: receiving an input search command for searching media assets. When a display device is connected to the audio / video transmission interface, the playback device 200 sends a first media asset request containing a keyword, wherein the keyword is obtained by parsing the search command, and the first media asset request is used to instruct the server to search for first audio and video data based on the keyword. The playback device 200 receives the first audio and video data fed back by the server 400 and sends the first audio and video data to the display device 500 through the audio / video transmission interface, so that the display device 500 displays the first audio and video data. When it is detected that the audio / video transmission interface is disconnected from the display device 500, the playback device 200 sends a second media asset request containing the keyword, wherein the second media asset request is used to instruct the server to search for second audio data based on the keyword, without searching for video data. The playback device 200 receives the second audio data fed back by the server 400 and controls the speaker to play the second audio data.

[0138] In some embodiments, the above process is basically repeated when the connection with the display device is re-established. When playing audio data before switching, the same keyword is sent after switching to request the audio and video data that can be displayed on the display device. If the request is received, the audio data is played through the display device. If the request is not received, the playback of the audio data on the playback device is canceled, and the audio data is played through the display device.

[0139] In some cases, if the audio application plays audio data before the connection is established, after the connection is established, the audio application is set to a background application, and the video application sends a request for audio and video resources by sending a keyword containing the previous audio data. If there is a resource response, the audio and video resources are played through the display device. If there is no resource response, the audio application is brought back to the foreground application, and the previously paused audio data is played through the display device.

[0140] The progress of replaying audio and video can also be referenced to the progress of audio data playback in audio applications.

[0141] In some embodiments, when a display device plays audio data from an audio application, it can display an interface showing information about the audio data, such as lyrics, composer, singer, etc.

[0142] In some embodiments, when an audio application is playing audio data on a display device, after the audio playback is complete or the user inputs an operation command, the playback device switches the application that provides data to the display device to a video application.

[0143] In some embodiments, video applications may also be launched in advance based on experience values.

[0144] Since the above embodiments are all described in conjunction with other methods, and different embodiments have the same parts, the same or similar parts between the various embodiments in this specification can be referred to mutually. They will not be described in detail here.

[0145] It should be noted that in this specification, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a circuit structure, article, or device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a circuit structure, article, or device. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the circuit structure, article, or device that includes the element.

[0146] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the content of the claims.

[0147] The above embodiments of this application do not constitute a limitation on the scope of protection of this application.

Claims

1. A playback device, characterized by, The playback device includes: The audio / video transmission interface is configured to output audio / video data to an external display device; speaker; The controller is communicatively connected to the audio / video transmission interface and the speaker, and is configured to: Receive input search instructions for searching media assets; When the display device is connected to the audio and video transmission interface, the control video application sends a first media resource request containing keywords, wherein the keywords are obtained by parsing the search command, and the first media resource request is used to enable the server to find the first audio and video data based on the keywords; The control video application receives the first audio and video data fed back by the server, and sends the first audio and video data to the display device through the audio and video transmission interface, so that the display device displays the first audio and video data; When the audio / video transmission interface is detected to be disconnected from the display device, a second media asset request containing the keyword is sent. The second media asset request is used to enable the server to search for second audio data based on the keyword, but not video data. The second audio data is the same as the audio data in the first audio / video data. The audio application is controlled to receive the second audio data fed back by the server, and the video application is controlled to stop sending the first audio and video data to the display device. Switch the playback program to the audio application and control the audio application to play the second audio data through the speaker.

2. The playback device according to claim 1, characterized in that, The control video application receives the first audio and video data fed back by the server, and sends the first audio and video data to the display device through the audio and video transmission interface, including: The control video application receives a list of audio and video data fed back by the server, wherein the list of audio and video data includes the first audio and video data and audio and video data related to the first audio and video data; The system receives an input selection operation for the first audio and video data and controls the video application to send the first audio and video data to the display device through the audio and video transmission interface.

3. The playback device of claim 1, wherein, The controller is also configured to: The current playback progress of the first audio and video data is marked, and the audio application is controlled to continue playing the second audio data based on the current playback progress of the first audio and video data.

4. The playback device of claim 1, wherein, After sending a second media asset request containing the keyword, the controller is further configured to: If the second audio data is not received from the server, the video application is reactivated and controlled to play the first audio data through the speaker. The first audio data is the audio data in the first audio and video data and is the same as the second audio data.

5. The playback device of claim 4, wherein, After the control video application plays the first audio data through the speaker, the controller is further configured to: When the video application finishes playing the first audio data, or when a control command to play the next track is received, the playback program is switched to the audio application.

6. The playback device of claim 1, wherein, The controller is also configured to: Send a third media asset request containing the keyword, wherein the third media asset request is used to enable the server to expand based on the keyword and perform an audio search based on the expanded keyword; Receive search results from the server, wherein the audio data in the search results is related to the second audio data; The audio application is controlled to play audio data from the search results through the speaker.

7. The playback device of claim 4, wherein, When the controller detects that the audio / video transmission interface has been reconnected to the display device, it is further configured to: If the currently playing audio data is the first audio data or the second audio data, the control video application sends the first audio and video data to the display device based on the playback progress of the currently playing audio data, so that the display device can display it; If the currently playing audio data is not the first audio data or the second audio data, the control video application sends a fourth media asset request containing the same keywords as the currently playing audio data; upon receiving the third audio and video data fed back by the server according to the fourth media asset request, the control video application sends the third audio and video data to the display device through the audio and video transmission interface so that the display device displays the third audio and video data.

8. The playback device of claim 7, wherein, The controller is also configured to: When no third audio or video data is received from the server in response to the fourth media asset request, the control video application displays the main interface of the video application through the display device.

9. A playback device, characterized by comprising: The playback device includes: The audio / video transmission interface is configured to output audio / video data to an external display device; speaker; The controller is communicatively connected to the audio / video transmission interface and the speaker, and is configured to: Receive input voice commands for searching media assets; When the display device is connected to the audio / video transmission interface, the control video application sends a first media asset request containing a keyword; the control video application receives first audio / video data fed back by the server according to the first media asset request, and sends the first audio / video data to the display device through the audio / video transmission interface, so that the display device displays the first audio / video data; wherein, the keyword is obtained by parsing the voice command, and the first media asset request is used to enable the server to find the first audio / video data according to the keyword; When the audio / video transmission interface is disconnected from the display device, a second media asset request containing the keyword is sent; and after the audio application receives the second audio data searched by the server according to the second media asset request, the video application is controlled to stop sending the first audio / video data to the display device; the playback program is switched to the audio application, and the audio application is controlled to play the second audio data through the speaker; wherein, the second media asset request is used to enable the server to search for the second audio data according to the keyword, without searching for video data, and the second audio data is the same as the audio data in the first audio / video data.

10. A method for automatically switching audio and video modes, characterized in that, The method includes: Receive input search instructions for searching media assets; When a display device is connected to the audio / video transmission interface, the control video application sends a first media resource request containing keywords, wherein the keywords are obtained by parsing the search command, and the first media resource request is used to enable the server to find the first audio / video data based on the keywords; The control video application receives the first audio and video data fed back by the server, and sends the first audio and video data to the display device through the audio and video transmission interface, so that the display device displays the first audio and video data; When the audio / video transmission interface is detected to be disconnected from the display device, a second media asset request containing the keyword is sent. The second media asset request is used to enable the server to search for second audio data based on the keyword, but not video data. The second audio data is the same as the audio data in the first audio / video data. The audio application is controlled to receive the second audio data fed back by the server, and the video application is controlled to stop sending the first audio and video data to the display device. Switch the playback program to the audio application and control the audio application to play the second audio data through the speaker.

Images