Display device and voice interaction method

Abstract

The invention relates to display equipment and a voice interaction method, and relates to the technical field of display equipment. The display device comprises a Bluetooth module configured to be in communication connection with a Bluetooth earphone; a display configured to display content; the audio output device is configured to carry out audio output; the at least one controller is connected with the display and is configured to receive a callback instruction sent by the Bluetooth earphone; obtaining a virtual callback number in response to the callback instruction; based on the virtual callback number, starting a voice assistant application, and establishing a voice channel with the Bluetooth headset; receiving voice data transmitted by the Bluetooth earphone through the voice channel; and calling the voice assistant application, determining response data of the voice data, and controlling the display equipment according to the response data. By adopting the technical scheme, the voice interaction between the user and the display equipment can be realized without the help of a voice remote controller.

Description

Technical Field

[0001] The present application relates to the technical field of display devices, and in particular to a display device and a voice interaction method. Background Art

[0002] With the rapid development of intelligent voice technology, display devices such as televisions can respond to user voice data and play response data corresponding to the above voice data on the basis of playing the audio they obtain themselves.

[0003] In traditional technology, when voice interaction between a user and a display device (such as a smart TV) is realized based on a voice remote control, the interaction process is usually that the voice remote control responds to the triggering operation of the voice button, collects voice data, encodes the voice data, and sends it to the display device via Bluetooth; the display device parses the received voice data, responds according to the parsing results, and broadcasts the response data corresponding to the voice data.

[0004] However, the above-mentioned traditional technology requires a voice remote control that supports Bluetooth and a mic (microphone) in hardware. If the voice remote control is damaged or lost, voice interaction between the user and the display device will not be possible. Summary of the Invention

[0005] The present application provides a display device and a voice interaction method to enable voice interaction between a user and a display device without the aid of a voice remote control.

[0006] In a first aspect, some embodiments provide a display device, including:

[0007] A Bluetooth module is configured to communicate with a Bluetooth headset;

[0008] a display configured to display content;

[0009] an audio output device configured to perform audio output;

[0010] At least one controller, connected to the display and configured to:

[0011] Receive a call back instruction sent by the Bluetooth headset;

[0012] In response to the call-back instruction, obtaining a virtual call-back number;

[0013] Based on the virtual callback number, start the voice assistant application and establish a voice channel with the Bluetooth headset; receive voice data transmitted by the Bluetooth headset through the voice channel;

[0014] The voice assistant application is called to determine response data to the voice data, and the display device is controlled according to the response data.

[0015] In some embodiments, the Bluetooth module of the display device can be connected to a Bluetooth headset for communication, so that the display device can receive a callback signal sent by the Bluetooth headset, and in response to the callback signal, obtain a virtual callback number, after which the display device can start the voice assistant application based on the virtual callback number and establish a voice channel with the Bluetooth headset. Furthermore, the display device can receive voice data transmitted by the Bluetooth headset through the voice channel, call the voice assistant application, determine the response data of the voice data, and then control the display device according to the response data. Based on this, without using a voice remote control, by setting a virtual callback number in the display device and utilizing the callback function of the Bluetooth headset, the display device responds to the callback instruction of the Bluetooth headset, starts the voice assistant application based on the virtual callback number, and establishes a voice channel with the Bluetooth headset, thereby realizing voice interaction between the user and the display device. In this way, when the voice remote control is damaged or lost, the user can continue to use the Bluetooth headset to realize voice interaction with the display device. Moreover, since the call-back function is a basic function of the Bluetooth headset, when the voice remote control is damaged or lost, the ordinary Bluetooth headset used in daily life can be directly used to continue to realize voice interaction between the user and the display device without the need to purchase other hardware devices, thus avoiding waste and improving the convenience of controlling the display device.

[0016] In a second aspect, some embodiments further provide a voice interaction method, applied to a display device, comprising:

[0017] Receive the callback command sent by the Bluetooth headset;

[0018] In response to the call-back instruction, obtaining a virtual call-back number;

[0019] Based on the virtual callback number, start the voice assistant application and establish a voice channel with the Bluetooth headset; receive voice data transmitted by the Bluetooth headset through the voice channel;

[0020] The voice assistant application is called to determine response data to the voice data, and the display device is controlled according to the response data.

[0021] In some embodiments, the Bluetooth module of the display device can be connected to a Bluetooth headset for communication, so that the display device can receive a callback signal sent by the Bluetooth headset, and in response to the callback signal, obtain a virtual callback number, after which the display device can start the voice assistant application based on the virtual callback number and establish a voice channel with the Bluetooth headset. Furthermore, the display device can receive voice data transmitted by the Bluetooth headset through the voice channel, call the voice assistant application, determine the response data of the voice data, and then control the display device according to the response data. Based on this, without using a voice remote control, by setting a virtual callback number in the display device and utilizing the callback function of the Bluetooth headset, the display device responds to the callback instruction of the Bluetooth headset, starts the voice assistant application based on the virtual callback number, and establishes a voice channel with the Bluetooth headset, thereby realizing voice interaction between the user and the display device. In this way, when the voice remote control is damaged or lost, the user can continue to use the Bluetooth headset to realize voice interaction with the display device. Moreover, since the call-back function is a basic function of the Bluetooth headset, when the voice remote control is damaged or lost, the ordinary Bluetooth headset used in daily life can be directly used to continue to realize voice interaction between the user and the display device without the need to purchase other hardware devices, thus avoiding waste and improving the convenience of controlling the display device.

[0022] In a third aspect, some embodiments further provide a voice interaction device, applied to a display device, comprising:

[0023] The command receiving module is used to receive the callback command sent by the Bluetooth headset;

[0024] A number acquisition module, configured to acquire a virtual callback number in response to the callback instruction;

[0025] A channel establishment module is used to start the voice assistant application based on the virtual callback number and establish a voice channel with the Bluetooth headset;

[0026] A voice receiving module, configured to receive voice data transmitted by the Bluetooth headset through the voice channel;

[0027] A voice response module is used to call the voice assistant application, determine response data for the voice data, and control the display device according to the response data.

[0028] In some embodiments, the Bluetooth module of the display device can be connected to a Bluetooth headset for communication, so that the display device can receive a callback signal sent by the Bluetooth headset, and in response to the callback signal, obtain a virtual callback number, after which the display device can start the voice assistant application based on the virtual callback number and establish a voice channel with the Bluetooth headset. Furthermore, the display device can receive voice data transmitted by the Bluetooth headset through the voice channel, call the voice assistant application, determine the response data of the voice data, and then control the display device according to the response data. Based on this, without using a voice remote control, by setting a virtual callback number in the display device and utilizing the callback function of the Bluetooth headset, the display device responds to the callback instruction of the Bluetooth headset, starts the voice assistant application based on the virtual callback number, and establishes a voice channel with the Bluetooth headset, thereby realizing voice interaction between the user and the display device. In this way, when the voice remote control is damaged or lost, the user can continue to use the Bluetooth headset to realize voice interaction with the display device. Moreover, since the call-back function is a basic function of the Bluetooth headset, when the voice remote control is damaged or lost, the ordinary Bluetooth headset used in daily life can be directly used to continue to realize voice interaction between the user and the display device without the need to purchase other hardware devices, thus avoiding waste and improving the convenience of controlling the display device.

[0029] In a fourth aspect, some embodiments further provide a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the method provided in some embodiments of the second aspect are implemented.

[0030] In a fifth aspect, some embodiments further provide a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the method provided in some embodiments of the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments of the present application or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0032] Figure 1 A schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application;

[0033] Figure 2 A schematic diagram of the hardware configuration of a display device provided in some embodiments of the present application;

[0034] Figure 3A schematic diagram of the hardware configuration of a control device provided in some embodiments of the present application;

[0035] Figure 4 A schematic diagram of software configuration of a display device provided in some embodiments of the present application;

[0036] Figure 5 A flowchart of a voice interaction method provided in some embodiments of the present application;

[0037] Figure 6 A flowchart illustrating steps for adding a virtual call record according to some embodiments of the present application;

[0038] Figure 7 A flowchart illustrating steps for adding virtual call records provided in other embodiments of the present application;

[0039] Figure 8 A flowchart illustrating steps for establishing a voice channel according to some embodiments of the present application;

[0040] Figure 9 A signaling diagram for establishing a voice channel between a Bluetooth headset and a display device provided in some embodiments of the present application;

[0041] Figure 10 A flowchart of a voice interaction method provided in some other embodiments of the present application;

[0042] Figure 11 A schematic flow chart of steps for determining the audio playback status provided in some embodiments of the present application;

[0043] Figure 12 A flowchart of a voice interaction method provided in some other embodiments of the present application;

[0044] Figure 13 A signaling diagram of a voice interaction method provided in some embodiments of the present application;

[0045] Figure 14 A schematic diagram of the structure of a voice interaction device provided in some embodiments of the present application;

[0046] Figure 15 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0047] The following embodiments are described in detail, with examples illustrated in the accompanying drawings. When the following description refers to the drawings, identical numbers in different figures represent identical or similar elements unless otherwise indicated. The embodiments described in the following embodiments are not intended to represent all possible implementations consistent with the present application. They are merely examples of systems and methods consistent with certain aspects of the present application, as detailed in the claims.

[0048] It should be noted that the brief descriptions of terms in this application are only for the purpose of facilitating the understanding of the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their ordinary and usual meanings.

[0049] In the specification and claims of this application and the accompanying drawings, the terms "first," "second," "third," etc. are used to distinguish similar or similar objects or entities, and are not necessarily intended to limit a particular order or sequence, unless otherwise noted. It should be understood that the terms used in this manner are interchangeable under appropriate circumstances.

[0050] The terms "comprise," "comprises," and "comprising," and any variations thereof, are intended to cover but not exclude inclusion; for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0051] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functionality associated with that element.

[0052] In the embodiments of the present application, the display device 200 generally refers to a device capable of displaying images and processing data. For example, the display device 200 includes but is not limited to a smart TV, a mobile terminal, a computer, a monitor, an advertising screen, a wearable device, a virtual reality device, an augmented reality device, etc.

[0053] Figure 1 This is a schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application. Figure 1 As shown in FIG, a user can operate the display device 200 through touch operation, the mobile terminal 300 and the control device 100. For example, the control device 100 can be a remote controller, a stylus pen, a handle, etc.

[0054] The mobile terminal 300 can function as a control device for performing human-computer interaction between a user and the display device 200. The mobile terminal 300 can also function as a communication device for establishing a communication connection with the display device 200 for data exchange. In some embodiments, the mobile terminal 300 can install software applications with the display device 200, enabling communication via a network communication protocol for one-to-one control and data communication. Audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 for synchronized display.

[0055] like Figure 1 As shown in FIG, the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate via a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0056] The display device 200 may provide a broadcast receiving television function, and may also additionally provide an intelligent network television function with a computer support function, including but not limited to network television, smart TV, Internet Protocol television (IPTV), etc.

[0057] Figure 2 Some embodiments of this application provide Figure 1 2 is a block diagram of the hardware configuration of the display device 200.

[0058] In some embodiments, the display device 200 may include at least one of a tuner 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface.

[0059] In some embodiments, detector 230 is used to collect signals from the external environment or external interactions. For example, detector 230 may include a light receiver, such as a sensor for collecting ambient light intensity; or an image collector, such as a camera, for collecting external environmental scenes, user attributes, or user interaction gestures; or a sound collector, such as a microphone, for receiving external sounds.

[0060] In some embodiments, the display 260 includes a display component for presenting images and a driver component for driving image display. The display 260 is configured to receive image signals output from the controller 250 for display. For example, the display 260 can be used to display video content, image content, menu control interface components, and user control UI interfaces.

[0061] In some embodiments, the communication device 220 is a component used to communicate with an external device or server 400 according to various communication protocol types. The display device 200 can be provided with multiple communication devices 220 depending on the supported communication methods. For example, if the display device 200 supports wireless network communication, the display device 200 can be provided with a communication device 220 including WiFi functionality. If the display device 200 supports Bluetooth connection communication, the display device 200 needs to be provided with a communication device 220 including Bluetooth functionality.

[0062] The communication device 220 can establish a communication connection between the display device 200 and an external device or server 400 via a wireless or wired connection. A wired connection can connect the display device 200 to an external device via a data cable, an interface, or other components. A wireless connection can connect the display device 200 to an external device via a wireless signal or wireless network. The display device 200 can establish a connection with an external device directly or indirectly through a gateway, router, or connection device.

[0063] In some embodiments, the controller 250 may include at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processor, and a power processor, and first to nth interfaces for input / output. The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in a memory. The controller 250 controls the overall operation of the display device 200.

[0064] In some embodiments, the controller 250 and the tuner 210 may be located in different separate devices, that is, the tuner 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0065] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface receives the user input command through the graphical user interface (GUI).

[0066] In some embodiments, the audio output device 270 may be a local speaker of the display device 200, or an external audio output device connected to the display device 200. For the external audio output device connected to the display device 200, the display device 200 may further be provided with an external audio output terminal, through which the audio output device may be connected to the display device 200 to output the sound of the display device 200.

[0067] In some embodiments, the user input interface 280 may be configured to receive instructions from a user.

[0068] Figure 3 Some embodiments of this application provide Figure 1 The hardware configuration diagram of the control device in the figure. Figure 3 As shown, the control device 100 may include: a controller 110, a communication interface 130, a user input / output interface, a memory, and a power supply.

[0069] The control device 100 is configured to control the display device 200 , and can receive user input operation instructions, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, playing the role of an interactive intermediary between the user and the display device 200 .

[0070] In some embodiments, the control device 100 may be a smart device. For example, the control device 100 may be installed with various applications for controlling the display device 200 according to user needs.

[0071] In some embodiments, as Figure 1 As shown, the mobile terminal 300 or other intelligent electronic devices can play a similar function as the control device 100 after installing the application for controlling the display device 200 .

[0072] Controller 110 includes a processor 112, random access memory (RAM) 113, read-only memory (ROM) 114, a communication interface 130, and a communication bus. Controller 110 is used to control the operation and functionality of control device 100, as well as communication and coordination between internal components and external and internal data processing.

[0073] Under the control of the controller 110, the communication interface 130 communicates control signals and data signals with the display device 200. The communication interface 130 may include at least one of a WiFi chip 131, a Bluetooth module 132, an NFC (Near Field Communication) module 133, or other near field communication modules.

[0074] The user input / output interface 140 includes at least one of a microphone 141 , a touch panel 142 , a sensor 143 , a button 144 and other input interfaces.

[0075] In some embodiments, the control device 100 includes at least one of a communication interface 130 and an input / output interface 140. The control device 100 is configured with the communication interface 130, such as a WiFi, Bluetooth, or NFC module, to encode user input commands via the WiFi protocol, Bluetooth protocol, or NFC protocol and transmit them to the display device 200.

[0076] The memory 190 is used to store various operating programs, data and applications for driving and controlling the control device 100 under the control of the controller. The memory 190 can store various control signal instructions input by the user.

[0077] The power supply 180 is used to provide operating power support for various components of the control device 100 under the control of the controller.

[0078] To facilitate user interaction, in some embodiments, the display device 200 may run an operating system. The operating system is a computer program for managing and controlling the hardware and software resources in the display device 200. The operating system may provide a user interface (control the display device), allow the user to interact with the display device 200, and support the running of various application programs.

[0079] It should be noted that the operating system may be a native operating system based on a specific operating platform, or a third-party operating system deeply customized based on a specific operating platform, or an independent operating system specially developed for the display device.

[0080] The operating system can be divided into different modules or layers according to the functions implemented.

[0081] For example, Figure 4 As shown, in some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (abbreviated as "application layer"), the application framework layer (abbreviated as "framework layer"), the system library layer and the kernel layer.

[0082] In some embodiments, the application layer provides services and interfaces for applications, enabling the display device 200 to run applications and interact with the user based on the applications. The application layer can host at least one application, which can include built-in window programs, system settings programs, clock programs, and the like, or applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the examples above.

[0083] The framework layer provides applications with an application programming interface (API) and programming framework. The application framework layer includes predefined functions. The application framework layer acts as a processing center, determining the actions taken by applications in the application layer. Through the API, applications can access system resources and services during execution.

[0084] like Figure 4As shown, in the embodiment of the present application, the application framework layer includes a view system, managers, content providers, etc., wherein the view system can design and implement the interface and interaction of the application, and the view system includes lists, grids, text boxes, buttons, etc. The manager includes at least one of the following modules: an activity manager for interacting with all activities running in the system; a location manager for providing system services or applications with access to the system location service; a package manager for retrieving various information related to the application packages currently installed on the device; a notification manager for controlling the display and clearing of notification messages; and a window manager for managing icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0085] In some embodiments, the activity manager is used to manage the lifecycle of each application and common navigation back functions, such as controlling application exit, opening, and back. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, taking screenshots, and controlling changes in display windows, such as shrinking, shaking, or distorting the display window.

[0086] In some embodiments, the system runtime layer can provide support for the framework layer. When the framework layer is used, the operating system will run the instruction library contained in the system runtime layer, such as the C / C++ instruction library, to implement the functions to be implemented by the framework layer.

[0087] In some embodiments, the kernel layer is a functional layer between the hardware and software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, and memory management. Figure 4 As shown, the kernel layer can be configured with hardware drivers, and the drivers included in the kernel layer can be at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB (Universal Serial Bus) driver, HDMI (High Definition Multimedia Interface) driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0088] It should be noted that the above example is only a simple division of the operating system functions and does not constitute a limitation on the specific operating system form of the display device 200 in the embodiment of the present application. Depending on factors such as the function of the display device and the type of operating system, the number of levels and specific level types contained in the operating system may be expressed in other forms.

[0089] With the rapid development of intelligent voice technology, display devices such as televisions can respond to the user's voice data on the basis of playing the audio they have acquired themselves, and play the response data corresponding to the above voice data. In traditional technology, when realizing voice interaction between users and display devices (such as smart TVs) based on voice remote controls, the interaction process is usually that the voice remote control responds to the triggering operation of the voice button, collects voice data, encodes the voice data, and sends it to the display device via Bluetooth; the display device parses the received voice data, and responds according to the parsing results, broadcasting the response data corresponding to the voice data. However, the above traditional technology requires a voice remote control that supports Bluetooth and mic in hardware. In the event that the voice remote control is damaged or lost, voice interaction between the user and the display device will not be possible.

[0090] In order to overcome the above problems, in some optional embodiments, see Figure 5 , provides a voice interaction method, which is applied to at least one controller 250 in a display device 200, hereinafter referred to as the controller. Wherein, the display device has a Bluetooth module, and the Bluetooth module is configured to communicate with a Bluetooth headset. Then, when the Bluetooth module is communicated with the Bluetooth headset, the display device can communicate with the Bluetooth headset to exchange information (such as instructions, data). Accordingly, when the Bluetooth module of the display device is communicated with the Bluetooth headset, the voice interaction method includes the following steps:

[0091] S510: Receive a call-back instruction sent by the Bluetooth headset.

[0092] Typically, a Bluetooth headset has a call back function, and the so-called call back function refers to a convenient operation of quickly calling back the most recently called number through a headset button or voice command, thereby triggering the instruction for the call back function, which is the call back instruction.

[0093] The callback instruction can be generated in a variety of ways. For example, double-clicking either earphone, long-pressing the left or right earphone, double-clicking the left or right earphone, long-pressing the multi-function key, long-pressing any button, double-clicking the side button of the earphone, etc. In this way, when Bluetooth detects a user operation for generating a callback instruction, the Bluetooth earphone can generate a callback instruction and send the callback instruction to the controller of the display device through a communication connection with the Bluetooth module of the display device, so that the controller of the display device receives the callback instruction sent by the Bluetooth earphone.

[0094] S520: Responding to the call-back instruction, obtaining a virtual call-back number.

[0095] Since display devices generally do not have the ability to make phone calls, they generally do not store actual call records and call numbers. Therefore, to enable the display device to respond to a callback instruction and simulate the callback function of a Bluetooth headset, a virtual callback number can be set in the display device. Thus, upon receiving the callback instruction, the controller of the display device can respond to the callback instruction and obtain the virtual callback number, thereby simulating the operation of a mobile terminal with a phone call function, such as a mobile phone, obtaining the most recent call number upon receiving the callback instruction.

[0096] Considering that display devices usually do not have the function of making calls, display devices usually do not have core classes or interfaces (i.e., call record classes) for managing, storing, and querying device call histories. Therefore, in order to store virtual callback numbers, a number database can be built in the display device to store virtual callback numbers.

[0097] Based on this, in some embodiments, the above S520 may include accessing a number database corresponding to the preset access address and reading the virtual callback number from the number database. In these embodiments, the number database may be set in the display device and the preset access address corresponding to the number database may be recorded.

[0098] The so-called access address usually refers to a specific path or identifier used to access network resources, equipment or systems. In this embodiment, the preset access address refers to a specific path used to access the number database. Therefore, the number database can be accessed according to the above preset access address.

[0099] In this way, when the controller of the display device receives the above-mentioned callback instruction, it can access the number database corresponding to the recorded preset access address and read the virtual callback number from the number database. Optionally, the number database can record one or more virtual callback numbers. Among them, when the number database records multiple virtual callback numbers, the controller of the display device can respond to the callback instruction and read a virtual callback number from the number database in various ways. For example, it can randomly read a virtual callback number from the number database, or read the last number written from the number database, etc. There is no specific limitation on this.

[0100] Among them, the virtual callback number is a pre-set multi-digit character sequence used to simulate a call number, which may include at least one of a variety of contents such as numbers, letters, and symbols; and the above-mentioned virtual callback number can be set in a variety of ways, for example, generating and displaying the serial number of the device as the virtual callback number at any time, etc., and there is no specific limitation on this.

[0101] S530: Based on the virtual callback number, start the voice assistant application and establish a voice channel with the Bluetooth headset.

[0102] It can be understood that in the interaction between a Bluetooth headset and a mobile terminal with a calling function such as a mobile phone, the mobile terminal can dial back the call number after obtaining the most recent call number in response to the call-back instruction sent by the Bluetooth headset, and establish a voice channel between the Bluetooth headset and the mobile terminal to realize voice transmission between the Bluetooth headset and the mobile terminal.

[0103] Accordingly, since display devices generally do not have a phone call function, and during the voice interaction between the user and the display device, it is necessary to implement voice transmission between the Bluetooth headset and the mobile terminal, therefore, similar to the most recent call number in the interaction between the Bluetooth headset and the mobile terminal, the virtual callback number can be used to start the voice assistant application of the display device and establish a voice channel between the display device and the Bluetooth headset for voice transmission between the display device and the Bluetooth headset. Based on this, after the controller of the display device reads the virtual callback number in response to the callback instruction sent by the Bluetooth headset, it can start the voice assistant application based on the virtual callback number and establish a voice channel with the Bluetooth headset.

[0104] Among them, the so-called voice assistant application has the intelligent function of allowing users to control the TV and obtain services in natural language through voice interaction technology. It usually includes speech recognition (ASR, Automatic Speech Recognition), natural language understanding (NLU, Natural Language Understanding), as well as command execution and feedback functions.

[0105] In some embodiments, in order to realize voice transmission between the above-mentioned Bluetooth headset and the display device, the Bluetooth headset of the display device has an HFP (Hands-Free Profile) function, which is a standardized protocol in Bluetooth technology. It is mainly used to realize the hands-free call function between Bluetooth devices (such as car systems, headsets, etc.) and mobile phones, which is actively initiated by the Bluetooth headset.

[0106] S540: Receive voice data transmitted by the Bluetooth headset through the voice channel.

[0107] After establishing a voice channel between the display device and the Bluetooth headset, the display device controller can receive voice data transmitted by the Bluetooth headset through the voice channel. The voice data is the audio data in the current environment collected by the Bluetooth headset, such as the user's voice commands and voice query requests.

[0108] In some embodiments, the voice data is obtained by encoding the collected original voice by the Bluetooth headset based on a preset Bluetooth protocol.

[0109] Specifically, the Bluetooth headset microphone converts the sound wave vibrations of the original voice into an analog electrical signal and completes the digitization through an analog-to-digital converter (ADC); then the digital signal processing (DSP) chip is used to remove the environmental noise (such as wind and keyboard sounds) in the digitized original voice, and suppress the echo during the call (such as the sound of the display device speaker picked up by the Bluetooth headset microphone), and then identify the starting and ending positions of the original voice, filter out the silent segments to reduce the amount of data transmission. Finally, the original voice presented in PCM (Pulse Code Modulation) data is compressed and encoded into a format suitable for Bluetooth transmission through the Bluetooth protocol as voice data to be transmitted to the display device. Among them, the above-mentioned preset Bluetooth protocol can be the HFP protocol.

[0110] In some embodiments, the Bluetooth headset packages the collected raw voice data and sends it to the display device via a voice channel in the form of BLE packets. Compared to traditional technologies that use SCO (Synchronous Connection-Oriented) or enhanced eSCO links to transmit audio to the display device, this embodiment uses LBE packets to transmit voice data to the display device, which can reduce the power consumption of the Bluetooth headset.

[0111] In some embodiments, the Bluetooth headset can collect audio data when triggered by the user, and according to different triggering methods, the playback device of the response data corresponding to the voice data can be determined. Optionally, the Bluetooth headset includes at least one voice button, and different button instruction identifiers can be set for different operation modes of triggering the voice button, and different button instruction identifiers are used to instruct the display device or the Bluetooth headset to play the response data corresponding to the voice data. Among them, for the voice button of the Bluetooth headset, the voice button can be triggered by a variety of operation methods, for example, double-click, single-click, long press, etc. Optionally, the Bluetooth headset includes at least one voice button, and different button instruction identifiers can be set for different voice buttons, and different button instruction identifiers are used to instruct the display device or the Bluetooth headset to play the response data corresponding to the voice data.

[0112] S550: Call the voice assistant application, determine response data for the voice data, and control the display device according to the response data.

[0113] As mentioned above, voice assistant applications generally include functions such as speech recognition, natural speech understanding, and command execution and feedback. Therefore, after receiving voice data, the controller of the display device can call the voice assistant application to perform speech recognition and natural speech understanding on the voice data to determine the response data of the voice data.

[0114] Among them, the controller of the display device calls the voice assistant application to convert the voice data into text content, and responds according to the text content to obtain response data of the voice data. For example, the controller of the display device can perform semantic analysis on the above text content to obtain the meaning expressed by the above text content, so that the controller of the display device can perform corresponding actions according to the above meaning, and use the execution results of the above actions as the response data of the voice data. For example, if the meaning expressed by the above text content is to provide information about Person A, the controller of the display device can call a search engine to search for Person A, and use the information obtained from the search as the response data of the voice data.

[0115] After obtaining the response data of the voice data, the controller of the display device can control the display device according to the response data.

[0116] It is understandable that during the voice interaction between a user and a display device, the user's voice command can be a command for controlling the display device, such as controlling the display device to switch display content, controlling the volume of the display device, etc. The user's voice command can also be a command for inquiring about information, such as inquiring about the day's weather conditions, querying personnel information, and querying technical knowledge. Based on this, the type of response data of the voice data obtained varies depending on the user's voice command. Therefore, after obtaining the response data of the voice data, the controller of the display device can adopt different control methods for the display device based on the different types of the response data.

[0117] For example, when the user's voice command is an instruction for performing display control on a display device, the response data of the voice data obtained is control data for performing display control on the display device, such as increasing the volume, decreasing the volume, switching the content, etc. Thus, the controller of the display device performs display control on the display device according to the response data of the voice data, so that the display device executes the control action represented by the above control data.

[0118] For another example, when the user's voice command is an instruction for inquiring about information, the response data of the voice data obtained is the response data of the information inquired by the user, such as specific weather information, personnel information, technical knowledge content, etc. Thus, the controller of the display device controls the playback of the audio output device or display of the display device according to the response data of the voice data, so that the audio output device or display of the display device plays the response data of the above-mentioned voice data.

[0119] Based on this, in some embodiments, the above S550 may include, when the response data is inquiry response data, outputting the response data; when the response data is control data, controlling the playback of the display or audio output device according to the response data.

[0120] As mentioned above, in some embodiments, the Bluetooth headset can collect audio data under user triggering, and according to different triggering methods, the playback device of the response data corresponding to the voice data can be determined. Therefore, the Bluetooth device can respond to the user triggering operation, obtain the operation instruction identifier, and after collecting the voice data, encode the above-mentioned operation instruction identifier and voice data to obtain the voice instruction. For example, the Bluetooth headset can use the NEC (Network Equipment Control) protocol of PWM (Pulse Width Modulation) to encode the above-mentioned operation instruction identifier and voice data. In this way, the Bluetooth device sends the voice instruction to the controller of the display device through the above-mentioned voice channel. After receiving the voice instruction, the controller of the display device can parse the voice instruction and obtain the operation instruction identifier and voice data. The display device calls the voice assistant application, determines the response data of the voice data, and determines the playback device of the above-mentioned response data according to the operation instruction identifier. When the operation instruction identifier indicates that the above-mentioned response data is output by the display device, the controller of the display device controls the audio output device of the display device to play the above-mentioned response data; when the operation instruction identifier indicates that the above-mentioned response data is output by the Bluetooth headset, the controller of the display device keeps the display displaying the currently displayed content and / or keeps the audio output device playing the currently played audio, and transmits the above-mentioned response data to the Bluetooth headset through the above-mentioned voice channel, so that the Bluetooth headset plays the above-mentioned response data after receiving the above-mentioned response data.

[0121] In some optional embodiments, when the display device is connected to multiple playback devices, such as Bluetooth headsets, voice remote controls, etc., the controller of the display device can control the corresponding playback device to play the response data according to the source of the voice data. For example, if the voice data is sent by a Bluetooth headset, the controller of the display device can transmit the response data to the Bluetooth headset through the above-mentioned voice channel, so that the Bluetooth headset plays the above-mentioned response data after receiving the above-mentioned response data. For another example, if the voice data is sent by a voice remote control, the controller of the display device can transmit the response data to the voice remote control, so that the voice remote control plays the above-mentioned response data through the audio playback device of the voice remote control after receiving the above-mentioned response data.

[0122] In the above embodiment, the Bluetooth module of the display device can be connected to the Bluetooth headset for communication, so that the display device can receive the callback signal sent by the Bluetooth headset, and obtain a virtual callback number in response to the callback signal. After that, the display device can start the voice assistant application based on the virtual callback number and establish a voice channel with the Bluetooth headset. In addition, the display device can receive the voice data transmitted by the Bluetooth headset through the voice channel, call the voice assistant application, determine the response data of the voice data, and then control the display device according to the response data. Based on this, without using a voice remote control, by setting a virtual callback number in the display device and utilizing the callback function of the Bluetooth headset, the display device responds to the callback instruction of the Bluetooth headset, starts the voice assistant application based on the virtual callback number, and establishes a voice channel with the Bluetooth headset, thereby realizing voice interaction between the user and the display device. In this way, when the voice remote control is damaged or lost, the user can continue to use the Bluetooth headset to realize voice interaction with the display device. Moreover, since the call-back function is a basic function of the Bluetooth headset, when the voice remote control is damaged or lost, the ordinary Bluetooth headset used in daily life can be directly used to continue to realize voice interaction between the user and the display device without the need to purchase other hardware devices, thus avoiding waste and improving the convenience of controlling the display device.

[0123] On the basis of the technical solution of the above embodiment, in some optional embodiments, a virtual call record adding step is added, such as Figure 6 As shown, the steps of adding the virtual call record may include:

[0124] S610 , in response to a power-on instruction for the display device, initialize a voice assistant application.

[0125] To implement a voice assistant application on a display device, upon detecting a power-on command for the display device, the voice assistant application is initialized in response to the power-on command. That is, the voice assistant application on the display device is initialized simultaneously with the power-on command for the display device. The power-on command for the display device can be generated in a variety of ways, such as by pressing a physical power button on the display device or by the user shouting a voice wake-up word for the display device.

[0126] Initializing the voice assistant application refers to the process of setting up the voice interaction system built into the display device for the first time or restoring its initial state. The core of the process is to allow the voice assistant application to enter the preparation stage of "normal use" from "not enabled" or "abnormal state", involving a series of basic operations such as hardware detection, parameter configuration, and function activation. For example, it detects whether the microphone array, Bluetooth module, chip computing power, etc. of the display device are normal, selects the trigger conditions for starting the voice assistant application, connects to Wi-Fi, and authorizes the voice assistant to access the display device functions (such as controlling the volume, switching display content, etc.).

[0127] As mentioned above, in this embodiment, the trigger condition for starting the voice assistant application is that the controller of the display device responds to the callback instruction sent by the Bluetooth headset instruction and obtains the virtual callback number.

[0128] S620: During the initialization of the voice assistant application, a virtual call record is added to a number database corresponding to a preset access address.

[0129] The virtual call record includes a virtual callback number.

[0130] As mentioned above, in order to store the virtual callback number, a number database can be set up in the display device in advance, and the preset access address of the above number database can be recorded in advance. In order to ensure that the controller of the display device can read the virtual callback number from the number database corresponding to the preset access address when responding to the callback instruction sent by the Bluetooth headset in the subsequent response, the virtual callback number can be added to the number database corresponding to the preset access address during the initialization process of the voice assistant application.

[0131] Specifically, during the initialization of the voice assistant application, the display device can generate a virtual call record including a virtual callback number, thereby adding the virtual call record to the number database corresponding to the preset access address to store the virtual callback number in the number database. The virtual call record in the number database is written according to a preset format so that the virtual call records stored in the number database have a unified standardized format, thereby improving the efficiency of obtaining the virtual callback number by reading the virtual call record. Furthermore, the above-mentioned preset format can be set according to the device system of the display device and the requirements of the actual application scenario, and there is no specific limitation on this.

[0132] Among them, the so-called call records refer to a collection of information automatically recorded and stored by communication devices (such as mobile phones, landlines, walkie-talkies, etc.) on calls that have been dialed, answered or not answered, which is used to trace communication history, manage contact interactions and query call details.

[0133] Accordingly, since display devices generally do not have the ability to make calls, they generally do not have real call records. However, in order to enable the display device to simulate the operation of a mobile terminal with a call-making function, such as a cell phone, in response to a call-back instruction sent by a Bluetooth headset and obtain the most recent call number, a virtual call record can be stored in the display device to simulate the real call record of a mobile terminal with a call-making function, such as a cell phone. Furthermore, in order to read a virtual call-back number in the virtual call record, the virtual call record includes a virtual call-back number.

[0134] In the above embodiment, in response to a power-on command for the display device, the voice assistant application is initialized, and during the initialization of the voice assistant application, a virtual call record including a virtual callback number is added to the number database of the display device. In this way, when the display device is initially powered on, the preparations for the voice assistant application and the virtual callback number can be further completed based on the communication connection between the Bluetooth module of the display device and the Bluetooth headset. This prepares the user for subsequent voice interaction with the display device using the callback function of the Bluetooth headset, and allows the user to continue voice interaction with the display device using the Bluetooth headset in the event that the voice remote control is damaged or lost.

[0135] Based on the technical solutions of the above embodiments, in some optional embodiments, the steps of adding virtual call records are refined. Figure 7 The steps for adding a virtual call record include:

[0136] S710: If the number database does not exist in the preset access address, construct a number database.

[0137] As mentioned above, the virtual call records are stored in a number database. Therefore, during the initialization process of the voice assistant application of the display device, the controller of the display device can first access the storage space corresponding to the preset access address to determine whether the number database exists. If the number database does not exist in the preset access address, the controller of the display device first constructs the number database. The constructed number database can be equivalent to the core class or interface (i.e., call record class) in the display device for managing, storing, and querying the device call history.

[0138] Optionally, the database type can be first defined based on the device system of the display device; then, based on the content of the "virtual call records" to be stored, the data structure of the database storage content can be designed. Furthermore, based on the above database type and data structure, a database file can be created to construct a number database in the storage space of the display device. Depending on the device system of the display device, the above database type can be various types of databases, such as SQLite (SQLite Database, an embedded relational database) and MySQL (My Structured Query Language, a relational database). Furthermore, the above data results are the format of the virtual call records stored in the number database designed based on the core fields of the virtual call records, which can include basic information fields (such as call ID, call number, contact name, etc.), call attribute fields (such as call type, call time, call duration, etc.), and additional information fields (such as call status, call notes, call recording path, etc.). SQL is the abbreviation for Structured Query Language.

[0139] S720: Call the calendar to generate a virtual call time.

[0140] It can be understood that call records usually include basic information, contact identification, additional information, as well as grouping and marking information; among which, basic information usually includes call time (date, hour, minute, second), call duration, call type (voice / video, outgoing / incoming / missed), etc.; contact identification usually includes contact name (if stored), phone number, call location, etc.; additional information usually includes call status (whether connected), network, call recording link, etc.; grouping and marking usually include missed call mark, frequently used contacts pinned to the top, call record notes (such as work communication, express call), etc.

[0141] After the number database is constructed, at least one virtual call record may be further constructed. When constructing the virtual call record, since the virtual call record may include at least the call time and the phone number, the calendar implementation may be called first to generate the virtual call time.

[0142] Optionally, the controller of the display device can call the calendar API (Application Programming Interface) of the display device to obtain the recorded schedule events, thereby automatically generating the call time of the virtual call record based on the event time of the schedule event as the virtual call time. For example, a simulated incoming call is performed ten minutes before the start of the schedule event, and the time ten minutes before the start of the schedule event is used as the virtual call time.

[0143] Optionally, the controller of the display device may call a calendar API of the display device to randomly generate virtual call times according to time periods (eg, 9:00 a.m. every day).

[0144] S730: Construct a virtual call record including the virtual call time and the preset callback number.

[0145] After generating the virtual call time, for each virtual call time generated, a virtual number, virtual contact, virtual call duration and other data can be added to the array where the virtual call time is located to construct a virtual call record including the virtual call time and the above-mentioned virtual number, virtual contact, virtual call duration and other data. The above-mentioned added virtual number can be used as a preset callback number, so that when the controller of the display device receives the callback instruction sent by the Bluetooth headset, it can be read as a virtual callback number by the controller of the display device. In this way, a virtual call record including the virtual call time and the preset callback number can be constructed. Among them, the number of the virtual call times generated above can be one or more, and the number of virtual call records constructed can also be one or more.

[0146] The preset callback number may be a randomly generated multi-digit sequence, a real phone number, or other real digital sequence, such as a device serial number, etc., and there is no specific limitation on this.

[0147] Optionally, at least one value array can be created, with each array used to store a virtual call record. The calendar implementation is then called to add the virtual call time to each data item. Then, for each array, call data containing at least the preset callback number is added to the data item to generate a virtual call record.

[0148] S740: Add the virtual call record to the number database.

[0149] After the virtual call record is constructed, the virtual call record can be added to the constructed number database. Since the virtual call record includes a virtual callback number, the callback number preset in the virtual call record can be used as the virtual callback number.

[0150] Optionally, when there are multiple virtual call records, since each virtual call record includes a virtual call time, when the controller of the display device reads the virtual callback number, it can read the callback number in the virtual call record whose virtual call time is closest to the current moment as the virtual callback number.

[0151] In this embodiment, when there is no number database in the preset access address of the display device, a number database and a virtual call record including a virtual call time and a preset callback number can be constructed to add a number virtual call record to the number database. Since the virtual call record includes the virtual callback number, the number database includes the virtual callback number to ensure that the controller of the display device can read the virtual callback number from the number database corresponding to the preset access address when responding to the callback instruction sent by the Bluetooth headset in the subsequent response, so that in the event that the voice remote control is damaged or lost, the Bluetooth headset can continue to realize voice interaction between the user and the display device.

[0152] Based on the technical solutions of the above embodiments, in some optional embodiments, the steps for establishing the voice channel are refined. Figure 8 The steps for establishing a voice channel include:

[0153] S810: When the virtual callback number is successfully obtained, a callback status is generated.

[0154] In the interaction between a Bluetooth headset and a mobile terminal with a phone call function, such as a mobile phone, if the mobile terminal obtains the most recently called number in response to a callback instruction sent by the Bluetooth headset, the mobile terminal will dial back the most recently called number and, based on whether the callback is successful, provide the Bluetooth headset with a callback result. If the callback result clearly indicates that the mobile terminal successfully dialed back the most recently called number, the Bluetooth headset can establish a voice channel with the mobile terminal in response to the callback result.

[0155] Accordingly, since the trigger condition for the display device to launch the voice assistant application is that the display device controller obtains the virtual callback number in response to the callback instruction sent by the Bluetooth headset instruction, and in order to realize the voice interaction between the user and the display device, it is necessary to establish a voice channel between the Bluetooth headset and the display device. Therefore, if the virtual callback number is successfully obtained, the display device controller can generate a callback status. The callback number is used to indicate that the virtual callback number is successfully obtained, thereby instructing the Bluetooth headset to establish a voice channel with the display device.

[0156] S820: Call the HFP service, encapsulate the callback status, and generate a channel establishment reminder.

[0157] Voice transmission between the Bluetooth headset and the display device is achieved based on the HFP function of the Bluetooth module of the display device. Therefore, after generating the callback status, the controller of the display device can call the HFP service, encapsulate the callback status, and generate a channel establishment reminder.

[0158] Optionally, the display device calls the HFP server, first encapsulates the callback status (indicating the successful acquisition of the virtual callback number), and then integrates the device information of the Bluetooth headset (such as name, address, etc.) into the status object. Thus, the device information of the Bluetooth headset is bound to the status data of the callback status and stored in a unified status object to obtain a channel establishment reminder.

[0159] S830: Send a channel establishment reminder to the Bluetooth headset, so that the Bluetooth headset establishes a voice channel in response to the channel establishment reminder.

[0160] After generating the channel establishment reminder, the controller of the display device may send the channel establishment reminder to the Bluetooth headset, so that the Bluetooth headset establishes a voice channel with the display device in response to the channel establishment reminder.

[0161] Optionally, after receiving the channel establishment reminder, the Bluetooth headset parses the channel establishment reminder through the HFP protocol to confirm that the display device has successfully obtained the virtual callback number, thereby triggering the voice channel establishment process. Specifically, it first obtains the device address of the display device, verifies the legitimacy of the device address, and then sends a response signaling to the display device, synchronously updates its own connection status to "connected", and initializes the voice channel, completing the preparations for voice interaction with the display device.

[0162] In some embodiments, as Figure 9 The figure shows the signaling diagram for establishing a voice channel between a Bluetooth headset and a display device, including:

[0163] S901: When the Bluetooth headset detects a user triggering operation corresponding to the callback function, it starts the callback function and sends AT and BLDM commands to the Bluetooth protocol stack of the Bluetooth headset.

[0164] Among them, the user triggers the Bluetooth headset button, and the Bluetooth headset's MCU (Microcontroller Unit) captures the button trigger signal through GPIO (General-purpose input / output). Then, the Bluetooth headset's MCU analyzes the button type and / or trigger method triggered by the user through the button trigger signal to determine whether it is to start the callback function (such as the trigger signal of a double-click operation). Then, the MCU converts the button trigger signal into AT and BLDM instructions. AT and BLDM instructions are AT commands used for Bluetooth device management. Their main functions are to query or operate the Bluetooth device list, connection status, etc. Among them, AT commands (Attention Command) are a standardized instruction set used to control modems or Bluetooth modules. They implement device configuration and function calls through simple text instructions; BLDM (Bluetooth Device Management) instructions are used to search, view, connect or disconnect Bluetooth devices, and are commonly used in the interaction between Bluetooth modules and main control devices (such as display devices).

[0165] S902: The Bluetooth protocol stack parses the AT and BLDM commands into ACTION_LAST_NUMBER_REDAIL.

[0166] ACTION_LAST_NUMBER_REDAIL is a broadcast action used to trigger redialing of the last dialed phone number.

[0167] S903: The Bluetooth protocol stack sends ACTION_LAST_NUMBER_REDAIL to the HFP service of the Bluetooth headset.

[0168] S904: The HFP service sends a replay request to the Telephony (telephony function) management module of the display device.

[0169] S905: The Telephony management module executes the replay logic (calls the TelephonyManager class) and starts the voice assistant service of the display device.

[0170] Among them, the TelephonyManager class is the core class for accessing telephone communication functions, providing functions such as device telephone status query, call control, and network information acquisition.

[0171] S906: The Telephony management module receives the replay broadcast (ACTION_LAST_NUMBER_REDAIL) and obtains the callback number according to the Telephony class requirements.

[0172] S907: The Telephony management module feeds back the callback status and callback number to the HFP service.

[0173] S908: The HFP service calls the underlying application to perform Bluetooth data encapsulation on the callback status and the callback number.

[0174] S909: The HFP service feeds back the encapsulation result to the Bluetooth protocol stack.

[0175] S910: The HFP service feeds back an AT response command and Bluetooth data to the Bluetooth headset according to the encapsulation result.

[0176] S911: The Bluetooth device establishes a voice channel with the display device.

[0177] In this embodiment, if the virtual callback number is successfully acquired, a channel establishment reminder can be issued by generating a callback status and calling the HFP service to encapsulate the callback status. A channel establishment reminder is then sent to the Bluetooth headset, causing the Bluetooth headset to establish a voice channel in response to the channel establishment reminder. This enables voice data transmission between the Bluetooth headset and the display device based on the voice channel, providing a voice data transmission channel for voice interaction between the user and the display device. This allows the user to continue voice interaction with the display device using the Bluetooth headset even if the voice remote control is damaged or lost.

[0178] After establishing the above-mentioned voice channel, the Bluetooth headset and the display device can transmit voice data through the above-mentioned voice channel to realize voice interaction between the user and the display device through the Bluetooth headset. Therefore, the display device can output voice prompt information to prompt the user to interact with the display device by voice, and then the Bluetooth headset collects audio data to send voice data to the display device. Among them, the so-called voice input reminder is a reminder information used to remind the user to interact with the display device by voice. The reminder information can be output in various forms such as voice, text, light, and graphics. The voice input reminder can prompt the user to speak, and then the Bluetooth headset starts to collect audio data.

[0179] Based on this, on the basis of the technical solutions of the above embodiments, in some embodiments, after the voice channel with the Bluetooth headset is established in the above S530, the voice interaction method may include at least one of the following:

[0180] 1) Control the hardware reminder module in the display device to output voice input reminders.

[0181] After establishing a voice channel with the Bluetooth headset, the controller of the display device can control the hardware reminder module in the display device to output a voice input reminder. For example, the display device can control the audio output device in the display device to play the voice input reminder. For another example, the display device can control the indicator light in the display device to flash to output the voice input reminder. In the above examples, the audio output device, indicator light, etc. can all serve as the hardware reminder module in the display device.

[0182] 2) Control the display to display a voice input reminder.

[0183] In some cases, after establishing a voice channel with the Bluetooth headset, the audio output device of the display device may be currently outputting audio, such as playing a song. In this case, in order not to interrupt the audio data being played by the display device, the controller of the display device may control the display of the display device to display a voice input reminder. For example, the voice input reminder may be displayed as text on the display screen, or as a flashing icon on the display screen.

[0184] 3) Send voice input reminder to Bluetooth headset.

[0185] In order to avoid interrupting the audio data currently being played by the audio output device of the display device and to avoid obstructing the video data currently being displayed on the display of the display device, the controller of the display device can send a voice input reminder to the Bluetooth headset, causing the Bluetooth headset to play the received voice input reminder to remind the user to interact with the display device by voice. Furthermore, since Bluetooth headsets are typically worn, the voice input reminder played by the Bluetooth headset is only heard by the wearer, thereby not causing interference to other people in the scene.

[0186] In this embodiment, the controller of the display device can select different output modes to output voice input reminders according to the needs of the actual scene. Thus, while ensuring the reminder effect of reminding the user to interact with the display device through voice, it does not interrupt the audio and video data being output by the display device and does not cause interference to other people in the scene.

[0187] Based on the technical solutions of the above embodiments, in some optional embodiments, such as Figure 10 As shown, the voice interaction method may include the following steps:

[0188] S1010: Receive a call-back instruction sent by the Bluetooth headset.

[0189] S1020: In response to the call-back instruction, obtain a virtual call-back number.

[0190] S1030: Start a voice assistant application based on the virtual callback number and establish a voice channel with the Bluetooth headset.

[0191] Among them, the specific implementation method of the above S1010-S1030 is the same as the specific implementation method of the above S510-S530, and will not be repeated here.

[0192] S1040: When the display device is in an audio or video playback state, control a hardware reminder module in the display device to output a voice input reminder.

[0193] After establishing a voice channel with the Bluetooth headset, the controller of the display device may first determine the state of the display device, and then select a suitable output mode of the voice input reminder according to the state of the display device.

[0194] When the display device is in an audio or video playback state, the controller of the display device can control the hardware reminder module in the display device to output a voice input reminder. The so-called audio or video playback state indicates that the display device is not currently playing audio or video. Therefore, the voice input reminder output by the hardware reminder module in the display device does not interrupt the audio or video data currently being played by the display device, and does not interfere with the user's viewing experience.

[0195] S1050, receiving voice data transmitted by the Bluetooth headset through the voice channel;

[0196] S1060: Call the voice assistant application, determine response data for the voice data, and control the display device according to the response data.

[0197] Among them, the specific implementation method of the above-mentioned S1050-S1060 is the same as the specific implementation method of the above-mentioned S540-S550, which will not be repeated here.

[0198] In this embodiment, when the display device is in the audio and video playback state, the hardware reminder module in the display device is controlled to output a voice input reminder. Since the so-called audio and video playback state indicates that the display device is not currently playing audio or video, the voice input reminder output by the hardware reminder module in the display device can fully attract the user's attention and improve the reminder effect. There is no terminal for the audio and video data being played by the display device, and there is no interference with the user's viewing experience.

[0199] Based on the above embodiments, in some embodiments, the step of determining the audio playback status is refined.

[0200] join Figure 11 The steps for determining the audio playback status shown include:

[0201] S1110, calling the voice assistant application to obtain the current playback status of the display device.

[0202] As mentioned earlier, the voice assistant application is an intelligent function that allows users to control the TV and obtain services in natural language through voice interaction technology. By calling the voice assistant application, the current playback status of the display device can be obtained.

[0203] The so-called playback status of the display device is used to indicate whether the display device has audio or video playback and the type of the current page of the display device.

[0204] S1120: When the current playback state is the first state, determine that the current mode is the audio playback state.

[0205] Among them, the first state is that there is no audio or video playback and the currently displayed page is the system homepage.

[0206] When the current playback state of the display device is no audio or video playback and the current display page is the first state of the system homepage, it can be indicated that although the display device is currently in the power-on state, it is not playing audio or video data. Therefore, the display device can set the current mode of the display device to the audio playback state. Then, when outputting the voice input reminder, the controller of the display device can control the hardware reminder module in the display device to output the voice input reminder.

[0207] In some optional embodiments, when the current playback state is not the first state, the current mode can be determined to be a non-audio playback state. The non-audio playback state can indicate that the display device is playing audio and video data and / or the currently displayed page is a video playback page rather than the system homepage. At this time, in order to avoid interrupting the audio and video data currently played by the display device, and in order to avoid blocking the content displayed on the currently displayed page, the controller of the display device can send a voice input reminder to the Bluetooth headset to send the voice input reminder through the Bluetooth headset. Optionally, when the display device is playing audio data and the currently displayed page is the system homepage, the controller of the display device can also control the display of the display device to display a voice input reminder.

[0208] In the above embodiment, by calling the voice assistant application, the current playback status of the display device is obtained, and when the current playback status is the first state, the current mode is determined to be the audio playback mode. The current playback mode of the display device can be determined in real time, thereby dynamically selecting the output method of the voice input reminder to better match the output requirements of the current scene, and enrich the output method of the voice input reminder to better improve the reminder effect.

[0209] Based on the technical solutions of the above embodiments, in some embodiments, such as Figure 12As shown, the voice interaction method may include the following steps:

[0210] S1210: Receive a call-back instruction sent by the Bluetooth headset.

[0211] S1220: In response to the call-back instruction, obtain a virtual call-back number.

[0212] S1230: Start a voice assistant application based on the virtual callback number and establish a voice channel with the Bluetooth headset.

[0213] S1240, receiving voice data transmitted by the Bluetooth headset through the voice channel;

[0214] S1250: Call the voice assistant application, determine response data for the voice data, and control the display device according to the response data.

[0215] Among them, the specific implementation method of the above S1210-S1250 is the same as the specific implementation method of the above S510-S550, and will not be repeated here.

[0216] S1260: If no voice data is received within a preset time period, close the voice assistant application; or, if a closing command for the voice assistant application is included in the response data, close the voice assistant application.

[0217] Typically, the voice interaction between the user and the display device will stop after a period of time. For example, the user only controls the display device through voice commands to complete operations such as volume adjustment and playback content switching. The control process may include an interaction process. For example, the user inquires about the weather, character information, etc. to the display device through voice commands. The inquiry process may include one or more interaction processes.

[0218] Based on this, in some optional embodiments, the controller of the display device can time the duration of the absence of voice data. When the timed duration reaches a preset time, it is determined that no voice data has been received within a preset time period. In this case, the controller of the display device can close the voice assistant application. The length of the preset time period can be set based on experience and the needs of actual application scenarios, and is not specifically limited to this.

[0219] In other optional embodiments, the user can control the display device to close the voice assistant application through voice commands to terminate the user's voice interaction with the display device. Therefore, when the controller of the display device detects that the response data of the voice data includes a closing command for the voice assistant application, the controller of the display device can close the voice assistant application.

[0220] In the above embodiment, the controller of the display device can close the voice assistant application in time to save the device resources of the display device occupied by the voice assistant application, and avoid the audio data of non-user commands in the environment collected by the Bluetooth headset from erroneous control of the display device, thereby avoiding interference with the user's viewing experience.

[0221] Based on the technical solutions of the above embodiments, in some embodiments, such as Figure 13 As shown, the above voice interaction process is explained by taking the interaction process between the display device and the Bluetooth headset as an example.

[0222] S1301: In response to a power-on instruction to the display device, the display device initializes a voice assistant application.

[0223] The Bluetooth headset sends a callback command to the display device.

[0224] S1302: During the initialization of the voice assistant application, if the number database does not exist in the preset access address, the display device constructs the number database.

[0225] S1303: The display device calls the calendar to generate a virtual call time and constructs a virtual call record including the virtual call time and a preset callback number.

[0226] S1304: The display device adds the virtual call record to the number database.

[0227] S1305: The Bluetooth headset sends a callback instruction to the display device.

[0228] S1306: In response to the call-back instruction, the display device accesses a number database corresponding to the preset access address and reads a virtual call-back number from the number database.

[0229] S1307 , when the virtual callback number is successfully obtained, the display device generates a callback status, calls the HFP service, encapsulates the callback status, and generates a channel establishment reminder.

[0230] S1308: The display device sends a channel establishment reminder to the Bluetooth headset.

[0231] S1309 , the Bluetooth headset establishes a voice channel with the display device in response to the channel establishment prompt.

[0232] S1310: The display device calls a voice assistant application to obtain a current playback state of the display device, and when the current playback state is a first state, determines that the current mode is an audio playback state.

[0233] S1311: The display device controls a hardware reminder module in the display device to output a voice input reminder.

[0234] S1312: The Bluetooth headset sends voice data to the display device via the voice channel.

[0235] S1313: The display device calls the voice assistant application to determine response data of the voice data.

[0236] S1314 , when the response data is inquiry response data, the display device outputs the response data; when the response data is control data, the display device controls the display or audio output device according to the response data.

[0237] S1315: If the display device does not receive voice data within a preset time period, the display device closes the voice assistant application; or, if the response data includes a closing command for the voice assistant application, the display device closes the voice assistant application.

[0238] The specific implementation of S1301-S1315 is the same as that in the above embodiments, and will not be repeated here.

[0239] Based on the same inventive concept, the present application also provides a voice interaction device for implementing the voice interaction method described above. The solution provided by this device is similar to the solution described in the method described above. Therefore, the specific limitations in the one or more voice interaction device embodiments provided below can be found in the above-mentioned limitations on the voice interaction method and will not be repeated here.

[0240] In an exemplary embodiment, Figure 14 As shown, a voice playback device is provided, which is applied to a display device and includes: an instruction receiving module 1410 , a number obtaining module 1420 , a number obtaining module 1430 , a voice receiving module 1440 and a voice response module 1450 .

[0241] The instruction receiving module 1410 is used to receive a call back instruction sent by the Bluetooth headset;

[0242] The number acquisition module 1420 is configured to acquire a virtual callback number in response to a callback instruction;

[0243] The channel establishing module 1430 is used to start the voice assistant application based on the virtual callback number and establish a voice channel with the Bluetooth headset;

[0244] The voice receiving module 1440 is used to receive voice data transmitted by the Bluetooth headset through the voice channel;

[0245] The voice response module 1450 is used to call the voice assistant application, determine the response data of the voice data, and control the display device according to the response data.

[0246] In some optional embodiments, the number acquisition module 1420 is specifically configured to:

[0247] Access the number database corresponding to the preset access address and read the virtual callback number from the number database.

[0248] In some optional embodiments, the voice interaction device further includes:

[0249] an initialization module, configured to initialize the voice assistant application in response to a power-on instruction to the display device;

[0250] The record adding module is used to add a virtual call record to the number database corresponding to the preset access address during the initialization process of the voice assistant application; wherein the virtual call record includes a virtual callback number.

[0251] In some optional embodiments, the record adding module is specifically configured to:

[0252] In the case that the number database does not exist in the preset access address, constructing the number database;

[0253] Call the calendar implementation to generate virtual call time;

[0254] Construct virtual call records including virtual call time and preset callback numbers;

[0255] Add virtual call records to the number database.

[0256] In some optional embodiments, the channel establishing module 1430 is specifically configured to:

[0257] When the virtual callback number is obtained successfully, a callback status is generated;

[0258] Call the HFP service, encapsulate the callback status, and generate a channel establishment reminder;

[0259] A channel establishment reminder is sent to the Bluetooth headset, so that the Bluetooth headset establishes a voice channel in response to the channel establishment reminder.

[0260] In some optional embodiments, the voice interaction device further includes a reminder output module, configured to perform at least one of the following after establishing a voice channel with the Bluetooth headset:

[0261] Control the hardware reminder module in the display device to output a voice input reminder;

[0262] Control the display to display a voice input reminder;

[0263] Send voice input reminders to Bluetooth headsets.

[0264] In some optional embodiments, the reminder output module is specifically used to:

[0265] When the display device is in an audio or video playback state, the hardware reminder module in the display device is controlled to output a voice input reminder.

[0266] In some optional embodiments, the voice interaction device further includes:

[0267] The status acquisition module is used to call the voice assistant application to obtain the current playback status of the display device;

[0268] The mode determination module is used to determine that the current mode is the audio playback state when the current playback state is the first state; wherein the first state is no audio or video playback and the current display page is the system homepage.

[0269] In some optional embodiments, the voice response module 1450 is specifically configured to:

[0270] When the response data is inquiry response data, the response data is output;

[0271] In the case where the response data is control data, playback control is performed on the display or audio output device according to the response data.

[0272] Each module in the above-mentioned voice playback device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of the processor of the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0273] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 15As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface, the display unit and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WiFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a voice playback method is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse.

[0274] Those skilled in the art will understand that Figure 15 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0275] In an alternative embodiment, Figure 15 The computer device shown may be the aforementioned display device, for example, a laser TV.

[0276] In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0277] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0278] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0279] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0280] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile memory and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a programmable logic unit (PLC), a data processing logic unit based on quantum computing, an artificial intelligence (AI) processor, and the like.

[0281] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0282] The above embodiments merely illustrate several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art may make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A display device, characterized in that: include: A Bluetooth module is configured to communicate with a Bluetooth headset; a display configured to display content; an audio output device configured to perform audio output; At least one controller, connected to the display and configured to: Receive a call back instruction sent by the Bluetooth headset; In response to the call-back instruction, obtaining a virtual call-back number; Based on the virtual callback number, start the voice assistant application and establish a voice channel with the Bluetooth headset; receive voice data transmitted by the Bluetooth headset through the voice channel; The voice assistant application is called to determine response data to the voice data, and the display device is controlled according to the response data.

2. The display device according to claim 1, wherein When executing the step of obtaining the virtual callback number in response to the callback instruction, the controller is configured to: Access a number database corresponding to a preset access address, and read a virtual callback number from the number database.

3. The display device according to claim 2, wherein The controller is further configured to: In response to a power-on instruction for the display device, initializing the voice assistant application; During the initialization process of the voice assistant application, a virtual call record is added to the number database corresponding to the preset access address; wherein the virtual call record includes the virtual callback number.

4. The display device according to claim 3, wherein When the controller adds the virtual call record to the number database corresponding to the preset access address, the controller is configured to: In the case that the number database does not exist in the preset access address, constructing the number database; Call the calendar implementation to generate virtual call time; Constructing a virtual call record including the virtual call time and a preset callback number; Add the virtual call record to the number database.

5. The display device according to any one of claims 1 to 4, characterized in that: When executing the establishing of the voice channel with the Bluetooth headset, the controller is configured to: If the virtual callback number is successfully obtained, a callback status is generated; Call the HFP service, encapsulate the callback status, and generate a channel establishment reminder; The channel establishment reminder is sent to the Bluetooth headset, so that the Bluetooth headset establishes the voice channel in response to the channel establishment reminder.

6. The display device according to claim 5, wherein: After executing the establishing of the voice channel with the Bluetooth headset, the controller is further configured to execute at least one of the following: Controlling a hardware reminder module in the display device to output a voice input reminder; Controlling the display to display a voice input reminder; Send a voice input reminder to the Bluetooth headset.

7. The display device according to claim 6, wherein: When the controller controls the hardware reminder module in the display device to output the voice input reminder, the controller is configured to: When the display device is in an audio or video playback state, the hardware reminder module in the display device is controlled to output the voice input reminder.

8. The display device according to claim 7, wherein: The controller is further configured to: Calling the voice assistant application to obtain the current playback status of the display device; When the current playback state is the first state, the current mode is determined to be the audio playback state; wherein the first state is no audio or video playback and the currently displayed page is the system homepage.

9. The display device according to claim 1, wherein When the controller controls the display device according to the response data, the controller is configured to: When the response data is inquiry response data, outputting the response data; In a case where the response data is control data, playback control is performed on the display or the audio output device according to the response data.

10. A voice interaction method, characterized in that: Applied to display devices, including: Receive the callback command sent by the Bluetooth headset; In response to the call-back instruction, obtaining a virtual call-back number; Based on the virtual callback number, start the voice assistant application and establish a voice channel with the Bluetooth headset; receive voice data transmitted by the Bluetooth headset through the voice channel; The voice assistant application is called to determine response data to the voice data, and the display device is controlled according to the response data.

Images